openresty

Introduction

OpenResty is a web server which extends Nginx by bundling it with many useful Nginx modules and Lua libraries. OpenResty excels at scaling web applications and services. For example, one module it includes enables you to write Lua code which will execute directly in an Nginx worker, enabling high-performance applications.

In this guide, you will set up OpenResty from source; the pre-built packages for some distros can be out of date. You'll also explore some simple example applications with OpenResty's unique features.

To follow this guide, you will need:

Note that Nginx should not be installed. It's included in OpenResty and having it installed ahead of time will conflict.

In this section, we will install OpenResty from source.

First, find the latest OpenResty source code release from the Download page on the OpenResty website. Download the tarball, making sure to replace the version number by the latest version if it has changed.

wget https://openresty.org/download/openresty-1.11.2.2.tar.gz

Download the PGP key file as well so we can verify the contents of the file.

wget https://openresty.org/download/openresty-1.11.2.2.tar.gz.asc

Next, we need to add the public key of the author as listed on the download page. At time of writing, this is public key A0E98066. However, do check if it has changed; it is listed on the same downloads page.

gpg --keyserver pgpkeys.mit.edu --recv-key A0E98066

You should see the following output (with your username in place of sammy):

Outputgpg: directory `/home/sammy/.gnupg' created
gpg: new configuration file `/home/sammy/.gnupg/gpg.conf' created
gpg: WARNING: options in `/home/sammy/.gnupg/gpg.conf' are not yet active during this run
gpg: keyring `/home/sammy/.gnupg/secring.gpg' created
gpg: keyring `/home/sammy/.gnupg/pubring.gpg' created
gpg: requesting key A0E98066 from hkp server pgpkeys.mit.edu
gpg: /home/sammy/.gnupg/trustdb.gpg: trustdb created
gpg: key A0E98066: public key "Yichun Zhang (agentzh) <agentzh@gmail.com>" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)

Check that the name on the public key (in this case, it's «Yichun Zhang») matches the name listed on the OpenResty website.

Now, check whether the signature file matches the downloaded .tar.gz file.

gpg openresty-1.11.2.2.tar.gz.asc

You'll see the following output:

Outputgpg: assuming signed data in `openresty-1.11.2.2.tar.gz'
gpg: Signature made Thu 17 Nov 2016 10:24:29 PM UTC using RSA key ID A0E98066
gpg: Good signature from "Yichun Zhang (agentzh) <agentzh@gmail.com>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 2545 1EB0 8846 0026 195B  D62C B550 E09E A0E9 8066

The warning you see is because you haven't personally verified whether this key belongs to the owner (i.e., you have not signed the public key with your own private key). There isn't an easy way to completely guarantee that this public key belongs to the owner, which it isn't completely trusted.

However, in this case, Good signature indicates that this file is indeed the file that the authors of OpenResty intended to distribute, so we can go ahead with the installation.

Next, extract the downloaded file and move into the newly created directory.

tar -xvf openresty-1.11.2.2.tar.gz
cd openresty-1.11.2.2

We will need to install the necessary tools to compile OpenResty. For more information about compiling programs from source, see this tutorial about using make to install packages from source.

sudo apt-get install build-essential

We will also need to install some other packages:

  • readline: This will be used by OpenResty for the command line interface.
  • ncurses: This is another piece of software that will be used by OpenResty for its command line interface.
  • PCRE: This software will provide OpenResty with regular expression capabilities.
  • OpenSSL: OpenSSL is used for secure communication, such as TLS (HTTPS).
  • Perl: Perl is a programming language that can be used in OpenResty.

To install these packages, execute the following command:

sudo apt-get install libreadline-dev libncurses5-dev libpcre3-dev libssl-dev perl

We now have all the components necessary to build and install OpenResty.

We will configure OpenResty with PCRE regular expression and IPv6 support. We will also parallelize part of the building process by supplying the -j2 flag, which will tell make that 2 jobs can be run simultaneously. This command will mostly test if all dependencies are available on your system and gather information that will be used by the build step later on. It will also already build some dependencies, such as LuaJIT.

./configure -j2 --with-pcre-jit --with-ipv6

Then you can build OpenResty, again by supplying the -j2 flags for parallelism. This will compile OpenResty itself.

make -j2

Finally, you can install OpenResty. Using sudo makes sure all files can be copied to the correct locations on the system so that OpenResty can find them when it is running.

sudo make install

You will need to allow HTTP connections in your firewall for the web server to work.

sudo ufw allow http

You can optionally also allow HTTPS with sudo ufw allow https if you are going to be using it. You can verify the change in the firewall by checking its status.

sudo ufw status

You should see HTTP traffic (port 80) allowed in the displayed output, as well as HTTPS (port 443) if you added it.

OutputStatus: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere
80                         ALLOW       Anywhere
443                        ALLOW       Anywhere
OpenSSH (v6)               ALLOW       Anywhere (v6)
80 (v6)                    ALLOW       Anywhere (v6)
443 (v6)                   ALLOW       Anywhere (v6)

You can now check whether the installation has worked. First, start OpenResty.

sudo /usr/local/openresty/bin/openresty

It will complete immediately without text output if the command is successful. In that case, you can visit http://your_server_ip in your browser. You'll see a page which says Welcome to OpenResty! with confirmation that it's fully installed and working.

You can now stop the OpenResty server.

sudo /usr/local/openresty/bin/openresty -s quit

OpenResty is installed, but you still need to configure OpenResty to run on startup so the server does not have to be started manually.

Here, we are going to set up OpenResty as a service so it starts automatically on boot. We will do this using the systemd init service. You can read this systemd basics tutorial for more information, and this unit file tutorial for information on unit files specifically.

Start by creating a new systemd file with nano or your favorite text editor.

sudo nano /etc/systemd/system/openresty.service

For this tutorial, we'll copy the default Nginx systemd file from a fresh installation and modify it for OpenResty. The complete file looks like this and should be pasted into the file we just opened. We'll walk through each part of the file to explain what it's doing.

/etc/systemd/system/openresty.service

# Stop dance for OpenResty
# A modification of the Nginx systemd script
# =======================
#
# ExecStop sends SIGSTOP (graceful stop) to the Nginx process.
# If, after 5s (--retry QUIT/5) OpenResty is still running, systemd takes control
# and sends SIGTERM (fast shutdown) to the main process.
# After another 5s (TimeoutStopSec=5), and if OpenResty is alive, systemd sends
# SIGKILL to all the remaining processes in the process group (KillMode=mixed).
#
# Nginx signals reference doc:
# http://nginx.org/en/docs/control.html
#
[Unit]
Description=A dynamic web platform based on Nginx and LuaJIT.
After=network.target

[Service]
Type=forking
PIDFile=/run/openresty.pid
ExecStartPre=/usr/local/openresty/bin/openresty -t -q -g 'daemon on; master_process on;'
ExecStart=/usr/local/openresty/bin/openresty -g 'daemon on; master_process on;'
ExecReload=/usr/local/openresty/bin/openresty -g 'daemon on; master_process on;' -s reload
ExecStop=-/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/openresty.pid
TimeoutStopSec=5
KillMode=mixed

[Install]
WantedBy=multi-user.target

In the [Unit] Section:

  • After=network.target makes OpenResty start after the network is up so that OpenResty can bind and listen to ports. This allows it to be reached from the outside.

In the [Service] section:

  • Type=forking tells systemd that the process we call in ExecStart will start the service in the background and that the process will stop itself after it has done so.
  • PIDFile=/run/openresty.pid tells systemd where to find the PID file OpenResty creates when it is started. This allows systemd to know whether OpenResty is still running.
  • ExecStartPre=/usr/local/openresty/bin/openresty -t -q -g 'daemon on; master_process on;' calls the OpenResty script without starting it. The -t flag tells OpenResty we only want it to test the configuration file; the -q flag tells it that we want to suppress any non-error output; the -g flag sets the global directives daemon on; master_process on that tell OpenResty we want it to start in the background as a daemon. We execute this script as ExecStartPre so that systemd will not try starting OpenResty when the configuration file is invalid, as it will error out on this command.
  • ExecStart=/usr/local/openresty/bin/openresty -g 'daemon on; master_process on;' actually starts OpenReesty. This is the same as ExecStartPre without the -t flag.
  • ExecReload=/usr/local/openresty/bin/openresty -g 'daemon on; master_process on;' -s reload tells systemd to run this command when we run systemctl reload openresty. The -s flag tells OpenResty to reload its configuration file.
  • ExecStop=-/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/openresty.pid tells systemd to run this command when OpenResty is stopped. It sents SIGSTOP to the process listed in the PID file. If it's still running 5 seconds later, systemd will take control via the following two options.
  • TimeoutStopSec=5 tells systemd we want the process stopped in 5 seconds. If it does not stop, systemd will forcibly try stopping OpenRest.
  • KillMode=mixed specifies how systemd should try stopping OpenResty when it has not stopped after 5 seconds.

In the [Install] section:

  • WantedBy=multi-user.target tells systemd when we want the service to be started if it is configured to be started at boot. multi-user.target means the service will only be started when a multi-user system has been started, i.e. we can run OpenResty as a different user.

That's all for the etc/systemd/system/openresty.service file. Next, we need to customize the OpenResty Nginx configuration file and enable the service.

Open the configuration file first.

sudo nano /usr/local/openresty/nginx/conf/nginx.conf

By default, it will look like this:

Default /usr/local/openresty/nginx/conf/nginx.conf

#user  nobody;
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;


events {
    worker_connections  1024;
}

. . .

Delete everything before the events { line, and replace it with the following three lines:

Updated /usr/local/openresty/nginx/conf/nginx.conf

user www-data;
worker_processes  auto;
pid /run/openresty.pid;

events {
    worker_connections  1024;
}

. . .

This file will make sure we are running as the www-data user and that systemd can recognize when OpenResty is running due to the pid line that will be created by OpenResty once it starts.

Save and close the file.

Next, create the log directory.

sudo mkdir /var/log/openresty

Reload the systemd service so that it can find our file.

sudo systemctl daemon-reload

Now, start OpenResty via systemd.

sudo systemctl start openresty

You can now visit http://your_server_ip again and see the same web page as before. The difference is that now, the process has been started by systemd.

The last step is to enable the service which will make sure that OpenResty is started on boot.

sudo systemctl enable openresty

You can learn more about managing systemd services and units in our services and units tutorial.

Now that we have configured the service, we can further configure OpenResty so it will e.g. log to a common location.

To configure OpenResty, we have used the default Nginx configuration as a reference, so that it will mostly match what you might be familiar with.

First, open the OpenResty configuration file again:

sudo nano /usr/local/openresty/nginx/conf/nginx.conf

This time, we are going to modify the http block and move the server block inside this http block to a new file to have a better structure. First, locate the http { line, and delete everything after it, except for the final line with the corresponding }.

Current /usr/local/openresty/nginx/conf/nginx.conf

user www-data;
worker_processes  auto;
pid /run/openresty.pid;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    . . .
}

Then, copy the following into the http block so that your entire file looks like this. We'll go over the changes one at a time.

/usr/local/openresty/nginx/conf/nginx.conf

user www-data;
worker_processes  auto;
pid /run/openresty.pid;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;
    tcp_nopush      on;
    tcp_nodelay     on;

    keepalive_timeout  65;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
    ssl_prefer_server_ciphers on;

    access_log /var/log/openresty/access.log;
    error_log /var/log/openresty/error.log;

    gzip  on;
    gzip_disable "msie6";

    include ../sites/*;
}

Save and close the file.

The changes we made to the default file are:

  • Uncommenting tcp_nopush on;, which tells OpenResty to send only full packets. This option is useful when using the sendfile option, which will allow OpenResty to optimize sending static files to a client.
  • Adding tcp_nodelay on;. This option will try sending packets as soon as possible, which may seem contrary to the above option, but it is used at a different time. tcp_nodelay is only used when using the keepalive option on HTTP requests, which is a connection to a web server by a web browser that will avoid the overhead of initiating an HTTP connection every time a request is made.
  • Adding and modifying the ssl_protocols and ssl_prefer_server_ciphers lines. These options configure the SSL options of OpenResty. We have removed old protocols that are vulnerable to known attacks on HTTPS, such as the POODLE attack.
  • Adding the access_log and error_log lines, which configures where the logs of the web server. We store the logs at the /var/log/openresty directory, which we created in the previous step.
  • Uncommenting gzip on and adding gzip_disable "msie6". These options will configure GZIP, which will compress web pages so that there is less data to transfer. We also add the last option because Internet Explorer 6 (and older) does not always process GZIP content properly.
  • Adding include ../sites/*;, which tells OpenResty to look for extra configuration files in the /usr/local/openresty/nginx/sites directory, which we will be created in a moment.
  • Removing all server blocks, which we'll relocate to a new file later in this step.

Next, create the new sites directory that we specified in the include line.

sudo mkdir /usr/local/openresty/nginx/sites

Create the default site.

sudo nano /usr/local/openresty/nginx/sites/default.conf

Add the following in this new file. This is the relocation of the original server block from nginx.conf, but has inline comments for more detail.

/usr/local/openresty/nginx/sites/default.conf

server {
    # Listen on port 80.
    listen 80 default_server;
    listen [::]:80 default_server;

    # The document root.
    root /usr/local/openresty/nginx/html/default;

    # Add index.php if you are using PHP.
    index index.html index.htm;

    # The server name, which isn't relevant in this case, because we only have one.
    server_name _;

    # When we try to access this site...
    location / {
        # ... first attempt to serve request as file, then as a directory,
        # then fall back to displaying a 404.
        try_files $uri $uri/ =404;
    }

    # Redirect server error pages to the static page /50x.html.
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root /usr/local/openresty/nginx/html;
    }
}

Save and close the file.

Now, create a new directory for this site.

sudo mkdir /usr/local/openresty/nginx/html/default

Then move the original index.html from its original location to the new directory.

sudo mv /usr/local/openresty/nginx/html/index.html /usr/local/openresty/nginx/html/default

Finally, restart OpenResty to use this new site.

sudo systemctl restart openresty

You can now again visit http://your_server_ip and see the same web page as before.

Now that OpenResty is fully configured, we can try some of the features introduced by OpenResty that are not available in Nginx by default.

In this section, we will look at a combination of different modules added by OpenResty which all exist to accommodate Lua scripting. We will modifying /usr/local/openresty/nginx/sites/default.conf throughout this step, so open it first.

sudo nano /usr/local/openresty/nginx/sites/default.conf

First, we are going to look at the content_by_lua_block configuration option. Copy the location block from the example configuration below and add it into the server block, below the two existing location blocks.

/usr/local/openresty/nginx/sites/default.conf content_by_lua_block example

server {
    . . .

    location /example {
         default_type 'text/plain';

         content_by_lua_block {
             ngx.say('Hello, Sammy!')
         }
    }
}

Save and close the file, then reload the configuration.

sudo systemctl reload openresty

If you visit http://your_server_ip/example now, you'll see a page which says Hello, Sammy!. Let's explain how this works.

The content_by_lua_block configuration directive executes everything within it as Lua code. Here, we used the Lua function ngx.say to print the message Hello, Sammy! to the page.

For another example, replace the contents of the location /example block with this:

/usr/local/openresty/nginx/sites/default.conf content_by_lua_file example

server {
    . . .

    location /example {
         default_type 'text/plain';

         content_by_lua_file /usr/local/openresty/nginx/html/default/index.lua;
    }
}

The content_by_lua_file loads the Lua content from an external file, so let's create the one we specified above: /usr/local/openresty/nginx/html/default/index.lua.

sudo nano /usr/local/openresty/nginx/html/default/index.lua

Add the following to the file, then save and close it.

/usr/local/openresty/nginx/html/default/index.lua

local name = ngx.var.arg_name or "Anonymous"
ngx.say("Hello, ", name, "!")

This is a simple piece of Lua which reads a query parameter in the URL, name, and customizes the greeting message. If no parameter is passed, it uses «Anonymous» instead.

Reload the configuration again.

sudo systemctl reload openresty

Now, visit http://your_server_ip/example?name=Sammy in your browser. This will display Hello, Sammy!. You can change the name query parameter, or omit it entirely.

Hello, Sammy!

You can also change the name query parameter to show any other name.

Warning: Do not place the Lua file you are loading in an accessible location from the web. If you do, your application code might be comprised if someone accesses this file. Place the file outside of your document root, for example by changing the document root to /usr/local/openresty/nginx/html/default/public and placing the Lua files one directory above it.

In this article you set up OpenResty, which will enable you to use Lua scripts in an Nginx worker. It is possible to create much more complex Lua scripts. You can also, for example, restrict access using Lua scripts or rewrite certain requests using Lua. You can find the documentation on the lua-nginx-module's GitHub page. There are even complete web frameworks which use Lua on OpenResty, such as Lapis.

If you want to learn more, you can visit the OpenResty website. Because OpenResty is just an extended Nginx installation, you can also learn how to set up server blocks in the Nginx server blocks tutorial, but make sure to replace the paths used in that tutorial by the paths used in this one.

2019/06/26 · Oleg Zvezdo4kin · 0 комментариев (-я)

Dynamic Memory Allocation in C using malloc(), calloc(), free() and realloc()

Изменение длины (размера) массива, где массив это набор элементов, хранящихся в ячейках памяти расположенных одна за другой.

arrays

Как вы можете видеть на изображении выше длина (размер) массива равна 9. Но что если требуется уменьшить длину, например, с 9 до 5 элементов, чтобы не расходовать память понапрасну?
Или наоборот, если массив из 9 элементов заполнен до конца, то увеличить его, например до 12?

В этом случае используется динамическое выделение памяти (Dynamic Memory Allocation).

Dynamic Memory Allocation - процедура изменяющая размер структуры данных во время выполнения программы.

C располагает для этого 4 функциями, которые подключаются заголовочным файлом <stdlib.h> и реализуют выделение памяти в С. Вот они:

Рассмотрим каждую из них подробнее:

malloc()

“malloc” или “выделение памяти” метод для динамического выделения одного большого блока памяти заданного размера.
Функция возвращает указатель типа void, который может быть присвоен указателю (pointer) любого типа.

Синтаксис:

 ptr = (cast-type*) malloc(byte-size)

Например:

 ptr = (int*) malloc(5 * sizeof(int));

Т.к. размер int равен 4-м байтам,
это выражение выделит 20 байт памяти.
Указатель ptr будет содержать
первый байт выделенной области памяти.

 malloc

Если же места недостаточно, выделение завершится неудачно и будет возвращён указатель NULL.

Пример:

#include <stdio.h> 
#include <stdlib.h> 
 
int main() 
{ 
 
    // This pointer will hold the 
    // base address of the block created 
    int* ptr; 
    int n, i, sum = 0; 
 
    // Get the number of elements for the array 
    n = 5; 
    printf("Enter number of elements: %d\n", n); 
 
    // Dynamically allocate memory using malloc() 
    ptr = (int*)malloc(n * sizeof(int)); 
 
    // Check if the memory has been successfully 
    // allocated by malloc or not 
    if (ptr == NULL) { 
        printf("Memory not allocated.\n"); 
        exit(0); 
    } 
    else { 
 
        // Memory has been successfully allocated 
        printf("Memory successfully allocated using malloc.\n"); 
 
        // Get the elements of the array 
        for (i = 0; i < n; ++i) { 
            ptr[i] = i + 1; 
        } 
 
        // Print the elements of the array 
        printf("The elements of the array are: "); 
        for (i = 0; i < n; ++i) { 
            printf("%d, ", ptr[i]); 
        } 
    } 
 
    return 0; 
} 

Вывод программы:

Enter number of elements: 5
Memory successfully allocated using malloc.
The elements of the array are: 1, 2, 3, 4, 5,

calloc()

“calloc” или “последовательное распределение” метод динамического выделения заданного кол-ва блоков указанного размера. При инициализации каждый блок получает значение по умолчанию - '0'.

Синтаксис:

 ptr = (cast-type*)calloc(n, element-size);

Например:

 ptr = (float*) calloc(5, sizeof(float));

Данное выражение выделяет непрерывное пространство в памяти
для 5-ти элементов, каждый из которых имеет размер 4 байта.

 calloc

Если места недостаточно, возвращается указатель NULL.

Пример:

#include <stdio.h> 
#include <stdlib.h> 
 
int main() 
{ 
 
    // This pointer will hold the 
    // base address of the block created 
    int* ptr; 
    int n, i, sum = 0; 
 
    // Get the number of elements for the array 
    n = 5; 
    printf("Enter number of elements: %d\n", n); 
 
    // Dynamically allocate memory using calloc() 
    ptr = (int*)calloc(n, sizeof(int)); 
 
    // Check if the memory has been successfully 
    // allocated by malloc or not 
    if (ptr == NULL) { 
        printf("Memory not allocated.\n"); 
        exit(0); 
    } 
    else { 
 
        // Memory has been successfully allocated 
        printf("Memory successfully allocated using calloc.\n"); 
 
        // Get the elements of the array 
        for (i = 0; i < n; ++i) { 
            ptr[i] = i + 1; 
        } 
 
        // Print the elements of the array 
        printf("The elements of the array are: "); 
        for (i = 0; i < n; ++i) { 
            printf("%d, ", ptr[i]); 
        } 
    } 
 
    return 0; 
}

Вывод программы:

Enter number of elements: 5
Memory successfully allocated using calloc.
The elements of the array are: 1, 2, 3, 4, 5,

free()

“free” - метод используемый для освобождения (de-allocated) памяти выделенной при помощи функций malloc и calloc, которые сами по себе не могут освободить её. Это помогает уменьшить потери памяти.

Синтаксис:

free(ptr);

 free de-allocated

Пример:

#include <stdio.h> 
#include <stdlib.h> 
 
int main() 
{ 
 
    // This pointer will hold the 
    // base address of the block created 
    int *ptr, *ptr1; 
    int n, i, sum = 0; 
 
    // Get the number of elements for the array 
    n = 5; 
    printf("Enter number of elements: %d\n", n); 
 
    // Dynamically allocate memory using malloc() 
    ptr = (int*)malloc(n * sizeof(int)); 
 
    // Dynamically allocate memory using calloc() 
    ptr1 = (int*)calloc(n, sizeof(int)); 
 
    // Check if the memory has been successfully 
    // allocated by malloc or not 
    if (ptr == NULL || ptr1 == NULL) { 
        printf("Memory not allocated.\n"); 
        exit(0); 
    } 
    else { 
 
        // Memory has been successfully allocated 
        printf("Memory successfully allocated using malloc.\n"); 
 
        // Free the memory 
        free(ptr); 
        printf("Malloc Memory successfully freed.\n"); 
 
        // Memory has been successfully allocated 
        printf("\nMemory successfully allocated using calloc.\n"); 
 
        // Free the memory 
        free(ptr1); 
        printf("Calloc Memory successfully freed.\n"); 
    } 
 
    return 0; 
}

Вывод программы:

Enter number of elements: 5
Memory successfully allocated using malloc.
Malloc Memory successfully freed.
Memory successfully allocated using calloc.
Calloc Memory successfully freed.

realloc()

“realloc” или “перераспределение памяти” - метод позволяющий перераспределять память. Если памяти выделенной до этого при помощи функций malloc и calloc недостаточно, при помощи realloc можно перераспределить её.

Синтаксис:

 ptr = realloc(ptr, newSize);

где ptr это перераспределяемая область с новым размером 'newSize'.

 realloc

Если же места недостаточно, указатель возвращает NULL.

Пример:

#include <stdio.h> 
#include <stdlib.h> 
 
int main() 
{ 
 
    // This pointer will hold the 
    // base address of the block created 
    int* ptr; 
    int n, i, sum = 0; 
 
    // Get the number of elements for the array 
    n = 5; 
    printf("Enter number of elements: %d\n", n); 
 
    // Dynamically allocate memory using calloc() 
    ptr = (int*)calloc(n, sizeof(int)); 
 
    // Check if the memory has been successfully 
    // allocated by malloc or not 
    if (ptr == NULL) { 
        printf("Memory not allocated.\n"); 
        exit(0); 
    } 
    else { 
 
        // Memory has been successfully allocated 
        printf("Memory successfully allocated using calloc.\n"); 
 
        // Get the elements of the array 
        for (i = 0; i < n; ++i) { 
            ptr[i] = i + 1; 
        } 
 
        // Print the elements of the array 
        printf("The elements of the array are: "); 
        for (i = 0; i < n; ++i) { 
            printf("%d, ", ptr[i]); 
        } 
 
        // Get the new size for the array 
        n = 10; 
        printf("\n\nEnter the new size of the array: %d\n", n); 
 
        // Dynamically re-allocate memory using realloc() 
        ptr = realloc(ptr, n * sizeof(int)); 
 
        // Memory has been successfully allocated 
        printf("Memory successfully re-allocated using realloc.\n"); 
 
        // Get the new elements of the array 
        for (i = 5; i < n; ++i) { 
            ptr[i] = i + 1; 
        } 
 
        // Print the elements of the array 
        printf("The elements of the array are: "); 
        for (i = 0; i < n; ++i) { 
            printf("%d, ", ptr[i]); 
        } 
 
        free(ptr); 
    } 
 
    return 0; 
}

Вывод:

Enter number of elements: 5
Memory successfully allocated using calloc.
The elements of the array are: 1, 2, 3, 4, 5, 

Enter the new size of the array: 10
Memory successfully re-allocated using realloc.
The elements of the array are: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,

Malloc C Function

Description The C library function void *malloc(size_t size) allocates the requested memory and returns a pointer to it.

Declaration Following is the declaration for malloc() function.

void *malloc(size_t size)

Parameters size − This is the size of the memory block, in bytes.

Return Value This function returns a pointer to the allocated memory, or NULL if the request fails.

Example The following example shows the usage of malloc() function.

#include <stdio.h>
#include <stdlib.h>
 
int main () {
   char *str;
 
   /* Initial memory allocation */
   str = (char *) malloc(15);
   strcpy(str, "tutorialspoint");
   printf("String = %s,  Address = %u\n", str, str);
 
   /* Reallocating memory */
   str = (char *) realloc(str, 25);
   strcat(str, ".com");
   printf("String = %s,  Address = %u\n", str, str);
 
   free(str);
 
   return(0);
}
2019/05/28 · Oleg Zvezdo4kin

Динамические массивы в C

Если вы используете относительно современный ЯП вроде JS, то массивы в С могут ввести вас в ступор.

Вступление

Массив в JavaScript:

let numbers = [];
numbers.push(1);
numbers.push(2);
numbers.push(3);
console.log(numbers); // [1, 2, 3]

Приведенный выше пример показывает, как бы мы создали массив в JS. Хорошо видно, что возможно добавить столько строк, сколько нам нужно.

Массив в C:

int numbers[3];
numbers[0] = 1;
numbers[1] = 2;
numbers[2] = 3;
printf("%d\n", numbers[0]); // 1
printf("%d\n", numbers[1]); // 2
printf("%d\n", numbers[2]); // 3

Первое выражение numbers[3] говорит компилятору, что массив сохранит в памяти 3 числа. Далее сохраним 1,2 и 3 под соответствующими индексами и выведем на дисплей.
Пока все прекрасно, но но нельзя добавить ещё элементы:

int numbers[3];
numbers[0] = 1;
numbers[1] = 2;
numbers[2] = 3;
numbers[3] = 4;
printf("%d\n", numbers[0]); // 1
printf("%d\n", numbers[1]); // 2
printf("%d\n", numbers[2]); // 3
printf("%d\n", numbers[3]); // should be 4

И что на это скажет gcc?:

array.c:8:5: warning: array index 3 is past the end of the array (which contains 3 elements) [-Warray-bounds]

Таким образом, мы получаем исключение за пределами границ памяти. Места в нашем массиве недостаточно, чтобы вместить ещё элементы.
Что же, если мы нуждаемся в динамическом массиве, в который можно добавить n элементов?

На С мы можем создать собственную имплементацию массива с динамически растущим размером.
Для этого используем блоки памяти.

malloc, realloc и указатели (pointers)

В С каждый тип данных имеет свой размер хранилища:

Тип Размер хранилища Диапазон значений
char 1 byte -128 до 127 или 0 до 255
unsigned char 1 byte 0 до 255
signed char 1 byte -128 до 127
int 2 или 4 bytes -32,768 до 32,767 или -2,147,483,648 до 2,147,483,647
unsigned int 2 или 4 bytes 0 до 65,535 или 0 до 4,294,967,295
short 2 bytes -32,768 to 32,767
unsigned short 2 bytes 0 до 65,535
long 8 bytes -9223372036854775808 до 9223372036854775807
unsigned long 8 bytes 0 до 18446744073709551615

В моей системе это 4 байта для целых чисел (integers). Просто имея эти данные можно создавать динамические массивы любого размера.
Размер типа данных можно получить при помощи функций sizeof(int), sizeof(double) или для тех типов данных, которые вам требуются.
Используя функции malloc и realloc мы можем создавать динамические блоки памяти.

Допустим, мы хотим начать с возможности хранить 3 целых числа (integers),это можно сделать, выделив блок памяти из 12 байт:

Единственным аргументом malloc является размер блока памяти в байтах.
Malloc возвращает указатель(pointer) на вновь созданный блок памяти.
#define INITIAL_CAPACITY 3
int main(){
     int* data = malloc(INITIAL_CAPACITY * sizeof(int));
}

Теперь у нас есть блок памяти, достаточно большой, чтобы вместить наши 3 целых числа - нам нужно сделать его динамическим. Сейчас мы все ещё не можем поместить больше 3-х элементов в наш блок памяти.

Если отслеживать размер и объемом используемой памяти, можно рассчитать, когда нужно изменить ее размер. Если блок памяти заполнен, удвоим размер этого блока памяти, при помощи вызвова realloc , который просто расширяет текущий блок памяти.

#define INITIAL_CAPACITY 3
void push(int *arr, int index, int value, int *size, int *capacity){
     int* ptr;
     if(*size > *capacity){
          ptr = realloc(arr, sizeof(arr) * 2);
          if(ptr == NULL)
               exit(0);
          else
               *capacity = sizeof(arr) * 2;
     }
 
     arr[index] = value;
     *size = *size + 1;
}
int main(){
     int size = 0;
     int capacity = INITIAL_CAPACITY;
     int* arr = malloc(INITIAL_CAPACITY * sizeof(int));
}

Теперь есть возможность добавлять элементы в блок памяти динамически.
Собрав все это вместе, получим следующую программу:

#include <stdio.h>
#include <stdlib.h>
 
#define INITIAL_CAPACITY 2
 
void push(int *arr, int index, int value, int *size, int *capacity){
     int* ptr;
     if(*size > *capacity){
          ptr = realloc(arr, sizeof(arr) * 2);
          if(ptr == NULL)
               exit(0);
          else
               *capacity = sizeof(arr) * 2;
     }
 
     arr[index] = value;
     *size = *size + 1;
}
 
int main(){
     int size = 0;
     int capacity = INITIAL_CAPACITY;
     int* arr = malloc(INITIAL_CAPACITY * sizeof(int));
     if(arr == NULL) {
          printf("Memory not allocated.\n");
          exit(0);
     }
     else {
          push(arr, 0, 1, &size, &capacity);
          push(arr, 1, 2, &size, &capacity);
          push(arr, 2, 3, &size, &capacity);
 
          printf("Current capacity: %d\n", capacity); // Current capacity: 2
 
          push(arr, 3, 4, &size, &capacity);
          push(arr, 4, 5, &size, &capacity);
          push(arr, 5, 6, &size, &capacity);
 
          printf("Current capacity: %d\n", capacity); // Current capacity: 16
     }
}
2019/05/27 · Oleg Zvezdo4kin · 0 комментариев (-я)

tcpdump sip analyzing

TCPdump is a powerful command-line packet analyzer, which may be used for a SIP message sniffing/analyzing. TCPdump is preinstalled on many linux distributions, or may be installed directly from debian repository:

apt-get install tcpdump TCPdump allows write sniff to a file or display it realtime. Its usage for SIP message analysis may look like:

1) Display real time to a console

 tcpdump -nqt -s 0 -A -i eth0 port 5060

where:

-n do not convert IP address to DNS names

-q be quite, print less output informations

-t do not print timestamps

-s capture number of bytes from a packet, 0 = default iptions which is max 65535, or simply a whole packet

-A prints each packet in ASCI

-v vv be very very verbose

-i use interface to capture on

port 5060 listen for traffic ort 5060 traffic for (source and destination)

Example of sniffing output using settings described above:

IP 158.193.139.51.5060 > 85.248.145.114.28444: UDP, length 252
E.......@......3U..r..o....uOPTIONS sip:85.248.145.114:28444 SIP/2.0
Via: SIP/2.0/UDP 158.193.139.51:5060;branch=0
From: sip:pinger@kamailio.org;tag=532c3365
To: sip:85.248.145.114:28444
Call-ID: 085dfe91-025420f2-2812823@158.193.139.51
CSeq: 1 OPTIONS
Content-Length: 0


IP 85.248.145.114.28444 > 158.193.139.51.5060: UDP, length 455
E.......u.!'U..r...3o.......SIP/2.0 200 OK
Via: SIP/2.0/UDP 158.193.139.51:5060;branch=0
Contact: <sip:192.168.1.103:28444>
To: <sip:85.248.145.114:28444>;tag=771cf100
From: <sip:pinger@kamailio.org>;tag=532c3365
CSeq: 1 OPTIONS
Accept-Language: en
Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, NOTIFY, MESSAGE, SUBSCRIBE, INFO
User-Agent: eyeBeam release 1102q stamp 51814
Content-Length: 0
or should be used with verbose extensions, which print some under layer protocol details, as check sums, header lengths…

tcpdump -nqt -s 0 -A -vvv -i eth0 port 5060 2) Second option to use tcpdump is to capture data and write them to a pcacp file, then do post analysis, using wireshark for example.

tcpdump -nq -s 0 -i eth0 -w /tmp/dump.pcap port 5060 Home page of the tcpdump tool is located here: http://www.tcpdump.org/tcpdump_man.html

Nice tutorial available at http://danielmiessler.com/study/tcpdump/

Man pages Usage More info from man pages:

Usage: tcpdump [-aAdDeflLnNOpqRStuUvxX] [-c count] [ -C file_size ]

              [ -E algo:secret ] [ -F file ] [ -i interface ] [ -M secret ]
              [ -r file ] [ -s snaplen ] [ -T type ] [ -w file ]
              [ -W filecount ] [ -y datalinktype ] [ -Z user ]
              [ expression ]

OPTIONS

-A Print each packet (minus its link level header) in ASCII. Handy for capturing web pages.

-B Set the operating system capture buffer size to buffer_size.

-c Exit after receiving count packets.

-C Before writing a raw packet to a savefile, check whether the file is currently larger than file_size and, if so, close the current savefile and open a new one. Savefiles after the first savefile will have the name specified with the -w flag, with a number after it, starting at 1 and continuing upward. The units of file_size are millions of bytes (1,000,000 bytes, not 1,048,576 bytes).

-d Dump the compiled packet-matching code in a human readable form to standard output and stop.

-d d Dump packet-matching code as a C program fragment.

-d dd Dump packet-matching code as decimal numbers (preceded with a count).

-D Print the list of the network interfaces available on the system and on which tcpdump can capture packets. For each network interface, a number and an interface name, possibly followed by a text description of the interface, is printed. The interface name or the number can be supplied to the -i flag to specify an interface on which to capture. This can be useful on systems that don’t have a command to list them (e.g., Windows systems, or UNIX systems lacking ifconfig -a); the number can be useful on Windows 2000 and later systems, where the interface name is a somewhat complex string. The -D flag will not be supported if tcpdump was built with an older version of libpcap that lacks the pcap_findalldevs() function.

-e Print the link-level header on each dump line.

-E Use spi@ipaddr algo:secret for decrypting IPsec ESP packets that are addressed to addr and contain Security Parameter Index value spi. This combination may be repeated with comma or newline seperation. Note that setting the secret for IPv4 ESP packets is supported at this time. Algorithms may be des-cbc, 3des-cbc, blowfish-cbc, rc3-cbc, cast128-cbc, or none. The default is des-cbc. The ability to decrypt packets is only present if tcpdump was compiled with cryptography enabled. secret is the ASCII text for ESP secret key. If preceeded by 0x, then a hex value will be read. The option assumes RFC2406 ESP, not RFC1827 ESP. The option is only for debugging purposes, and the use of this option with a true `secret’ key is discouraged. By presenting IPsec secret key onto command line you make it visible to others, via ps(1) and other occasions. In addition to the above syntax, the syntax file name may be used to have tcpdump read the provided file in. The file is opened upon receiving the first ESP packet, so any special permissions that tcpdump may have been given should already have been given up.

-f Print `foreign’ IPv4 addresses numerically rather than symbolically (this option is intended to get around serious brain damage in Sun’s NIS server — usually it hangs forever translating non-local internet numbers). The test for `foreign’ IPv4 addresses is done using the IPv4 address and netmask of the interface on which capture is being done. If that address or netmask are not available, available, either because the interface on which capture is being done has no address or netmask or because the capture is being done on the Linux “any” interface, which can capture on more than one interface, this option will not work correctly.

-F Use file as input for the filter expression. An additional expression given on the command line is ignored.

-G If specified, rotates the dump file specified with the -w option every rotate_seconds seconds. Savefiles will have the name specified by -w which should include a time format as defined by strftime(3). If no time format is specified, each new file will overwrite the previous. If used in conjunction with the -C option, filenames will take the form of `file<count>’.

-i Listen on interface. If unspecified, tcpdump searches the system interface list for the lowest numbered, configured up interface (excluding loopback). Ties are broken by choosing the earliest match. On Linux systems with 2.2 or later kernels, an interface argument of “any” can be used to capture packets from all interfaces. Note that captures on the “any” device will not be done in promiscuous mode. If the -D flag is supported, an interface number as printed by that flag can be used as the interface argument.

-I Put the interface in “monitor mode”; this is supported only on IEEE 802.11 Wi-Fi interfaces, and supported only on some operating systems. Note that in monitor mode the adapter might disassociate from the network with which it’s associated, so that you will not be able to use any wireless networks with that adapter. This could prevent accessing files on a network server, or resolving host names or network addresses, if you are capturing in monitor mode and are not connected to another network with another adapter. This flag will affect the output of the -L flag. If -I isn’t specified, only those link-layer types available when not in monitor mode will be shown; if -I is specified, only those link-layer types available when in monitor mode will be shown.

-K Don’t attempt to verify IP, TCP, or UDP checksums. This is useful for interfaces that perform some or all of those checksum calculation in hardware; otherwise, all outgoing TCP checksums will be flagged as bad.

-l Make stdout line buffered. Useful if you want to see the data while capturing it. E.g., “tcpdump -l | tee dat” or “tcpdump -l > dat & tail -f dat”.

-L List the known data link types for the interface, in the specified mode, and exit. The list of known data link types may be dependent on the specified mode; for example, on some platforms, a Wi-Fi interface might support one set of data link types when not in monitor mode (for example, it might support only fake Ethernet headers, or might support 802.11 headers but not support 802.11 headers with radio information) and another set of data link types when in monitor mode (for example, it might support 802.11 headers, or 802.11 headers with radio information, only in monitor mode).

-m Load SMI MIB module definitions from file module. This option can be used several times to load several MIB modules into tcpdump.

-M Use secret as a shared secret for validating the digests found in TCP segments with the TCP-MD5 option (RFC 2385), if present.

-n Don’t convert addresses (i.e., host addresses, port numbers, etc.) to names.

-N Don’t print domain name qualification of host names. E.g., if you give this flag then tcpdump will print “nic” instead of “nic.ddn.mil”.

-O Do not run the packet-matching code optimizer. This is useful only if you suspect a bug in the optimizer.

-p Don’t put the interface into promiscuous mode. Note that the interface might be in promiscuous mode for some other reason; hence, `-p’ cannot be used as an abbreviation for `ether host {local-hw-addr} or ether broadcast’.

-q Quick (quiet?) output. Print less protocol information so output lines are shorter.

-R Assume ESP/AH packets to be based on old specification (RFC1825 to RFC1829). If specified, tcpdump will not print replay prevention field. Since there is no protocol version field in ESP/AH specification, tcpdump cannot deduce the version of ESP/AH protocol.

-r Read packets from file (which was created with the -w option). Standard input is used if file is “-”.

-S Print absolute, rather than relative, TCP sequence numbers.

-s Snarf snaplen bytes of data from each packet rather than the default of 65535 bytes. Packets truncated because of a limited snapshot are indicated in the output with “[|proto]”, where proto is the name of the protocol level at which the truncation has occurred. Note that taking larger snapshots both increases the amount of time it takes to process packets and, effectively, decreases the amount of packet buffering. This may cause packets to be lost. You should limit snaplen to the smallest number that will capture the protocol information you’re interested in. Setting snaplen to 0 sets it to the default of 65535, for backwards compatibility with recent older versions of tcpdump.

-T Force packets selected by “expression” to be interpreted the specified type. Currently known types are aodv (Ad-hoc On-demand Distance Vector protocol), cnfp (Cisco NetFlow protocol), rpc (Remote Procedure Call), rtp (Real-Time Applications protocol), rtcp (Real-Time Applications control protocol), snmp (Simple Network Management Protocol), tftp (Trivial File Transfer Protocol), vat (Visual Audio Tool), and wb (distributed White Board).

-t Don’t print a timestamp on each dump line.

-t t Print an unformatted timestamp on each dump line.

-t tt Print a delta (micro-second resolution) between current and previous line on each dump line.

-t ttt Print a timestamp in default format proceeded by date on each dump line.

-t tttt Print a delta (micro-second resolution) between current and first line on each dump line.

-u Print undecoded NFS handles.

-U Make output saved via the -w option “packet-buffered”; i.e., as each packet is saved, it will be written to the output file, rather than being written only when the output buffer fills. The -U flag will not be supported if tcpdump was built with an older version of libpcap that lacks the pcap_dump_flush() function.

-v When parsing and printing, produce (slightly more) verbose output. For example, the time to live, identification, total length and options in an IP packet are printed. Also enables additional packet integrity checks such as verifying the IP and ICMP header checksum. When writing to a file with the -w option, report, every 10 seconds, the number of packets captured.

-v v Even more verbose output. For example, additional fields are printed from NFS reply packets, and SMB packets are fully decoded.

-v vv Even more verbose output. For example, telnet SB … SE options are printed in full. With -X Telnet options are printed in hex as well.

-w Write the raw packets to file rather than parsing and printing them out. They can later be printed with the -r option. Standard output is used if file is “-”. See pcap-savefile(5) for a description of the file format.

-W Used in conjunction with the -C option, this will limit the number of files created to the specified number, and begin overwriting files from the beginning, thus creating a ‘rotating’ buffer. In addition, it will name the files with enough leading 0s to support the maximum number of files, allowing them to sort correctly. Used in conjunction with the -G option, this will limit the number of rotated dump files that get created, exiting with status 0 when reaching the limit. If used with -C as well, the behavior will result in cyclical files per timeslice.

-x When parsing and printing, in addition to printing the headers of each packet, print the data of each packet (minus its link level header) in hex. The smaller of the entire packet or snaplen bytes will be printed. Note that this is the entire link-layer packet, so for link layers that pad (e.g. Ethernet), the padding bytes will also be printed when the higher layer packet is shorter than the required padding.

-x x When parsing and printing, in addition to printing the headers of each packet, print the data of each packet, including its link level header, in hex.

-X When parsing and printing, in addition to printing the headers of each packet, print the data of each packet (minus its link level header) in hex and ASCII. This is very handy for analysing new protocols.

-X X When parsing and printing, in addition to printing the headers of each packet, print the data of each packet, including its link level header, in hex and ASCII.

-y Set the data link type to use while capturing packets to datalinktype.

-z Used in conjunction with the -C or -G options, this will make tcpdump run ” command file ” where file is the savefile being closed after each rotation. For example, specifying -z gzip or -z bzip2 will compress each savefile using gzip or bzip2. Note that tcpdump will run the command in parallel to the capture, using the lowest priority so that this doesn’t disturb the capture process. And in case you would like to use a command that itself takes flags or different arguments, you can always write a shell script that will take the savefile name as the only argument, make the flags & arguments arrangements and execute the command that you want.

-Z Drops privileges (if root) and changes user ID to user and the group ID to the primary group of user. This behavior can also be enabled by default at compile time. expression selects which packets will be dumped. If no expression is given, all packets on the net will be dumped. Otherwise, only packets for which expression is `true’ will be dumped. For the expression syntax, see pcap-filter(7). Expression arguments can be passed to tcpdump as either a single argument or as multiple arguments, whichever is more convenient. Generally, if the expression contains Shell metacharacters, it is easier to pass it as a single, quoted argument. Multiple arguments are concatenated with spaces before being parsed.

2019/05/27 · Oleg Zvezdo4kin

Предыдущие записи >>

Только авторизованные участники могут оставлять комментарии.
  • blog.txt
  • Последние изменения: 2019/03/27