Nginx
Introduction
Nginx (pronounced engine-x) is a popular open-source web server, reverse proxy server, and load balancer. It is well-known for its performance, scalability, and versatility.
Getting Started
Compatibility
The Nginx O11ySource is compatible with all versions greater than 1.25.3
Data Collection Method
vuSmartMaps collects health and performance data from Nginx server using an internal data collector. This data collector collects data based on the source configuration. Nginx logs are collected by VuNet's Logbeat agent
Prerequisites
Inputs for Configuring Data Source
- Nginx Server: The IP Address/FQDN of the Nginx server. This field is the key to identify each server you add here.
- Period [in seconds]: How frequently data is gathered. The period should be between 60 seconds – 3000 seconds.
- URL: The server URL to Nginx server-status page. Eg: http://127.0.0.1, https://127.0.0.1, http://127.0.0.1:8080 or https://127.0.0.1:8080
- Access Log Path: Enter the Nginx access logfile's entire path
- Error Log Path: Enter the Nginx error logfile's entire path.
Firewall Requirement
To collect data from this O11ySource, ensure the following ports are opened:
| Source IP | Destination IP | Destination Port | Protocol | Direction |
|---|---|---|---|---|
| IP address of the Nginx server | Data Collector End Point | 9092* | TCP | Inbound |
*Before providing the firewall requirements, please update the port based on the customer environment.
Configuring the Target
Configure Metrics Collection from Nginx Server VuNet's internal data collectors collects Nginx metrics by connecting to the stub_status module, which provides the necessary functionality to display the status page. This module makes it simple to determine how well the server is doing.
- To check whether the stub_status module is enabled or not, use
nginx -V 2\>&1 | grep -o "with-http_stub_status_module". If you see a grep match output containing http_stub_status_module, then your module is already enabled and you can skip ahead. - If the status module is not enabled, you should check your Nginx configuration file located by default at
/etc/nginx/nginx.confor/etc/nginx/sites-available/defaultfor most systems, or within a specific virtual host file - As seen below, you must enable the status module and make sure that stub_status is active in order to gather metrics from Nginx. If you are using a prebuilt Nginx package, it should usually be included by default. To confirm, check if the
stub_status on; line is present in your Nginx configuration. To indicate which IP addresses should view the status page, you must alter the block that begins withlocation /server_status. In the example that follows, 127.0.0.1 access is permitted.
location /server_status {
stub_status;
allow 127.0.0.1;
deny all;
stub_status on;
server_tokens on;
}
Configure Logs Collection from Nginx Server The Nginx access log and error log offers even more specific information about each client request for Nginx Open Source version than the status module does. Your Nginx logs can be customized to include information that is more pertinent to your requirements. Nginx logs are collected by VuNet's Logbeat agent.
- Access log from Nginx server are collected in a specific format. Inculde the below format in your Nginx configuration.
log_format access_format 'time: $time_iso8601,' ' bytes_sent: $bytes_sent,' ' connection_time: $connection_time,' ' nginx_host: $host,' ' pid: $pid,' ' proxy_host: $proxy_host,' ' proxy_port: $proxy_port,' ' realip_remote_addr: $realip_remote_addr,' ' realip_remote_port: $realip_remote_port,' ' remote_addr: $remote_addr,' ' remote_port: $remote_port,' ' request: $request,' ' request_filename: $request_filename,' ' request_method: $request_method,' ' request_time: $request_time,' ' request_uri: $request_uri,' ' server_addr: $server_addr,' ' server_name: $server_name,' ' server_port: $server_port,' ' server_protocol: $server_protocol,' ' ssl_client_verify: $ssl_client_verify,' ' ssl_protocol: $ssl_protocol,' ' ssl_server_name: $ssl_server_name,' ' ssl_session_id: $ssl_session_id,' ' ssl_session_reused: $ssl_session_reused,' ' status: $status,' ' upstream_addr: $upstream_addr,' ' upstream_bytes_received: $upstream_bytes_received,' ' upstream_bytes_sent: $upstream_bytes_sent,' ' upstream_cache_status: $upstream_cache_status,' ' upstream_connect_time: $upstream_connect_time,' ' upstream_response_time: $upstream_response_time';
access_log <Path of Log File> access_format;
Configuration Steps
- Enable the O11ySource.
- Select the Sources tab and press the
+button to add a new Nginx host to be monitored. - Set up metrics and log collection configurations. Afterwards, select
Save and Continueto proceed with downloading the Logbeat agent. - The following packages will be available for download based on the OS:Download <OS> Logbeat Installation Binary - Downloads the full Logbeat package with required configurations for a fresh installationDownload <OS> Logbeat Update Configuration Binary - Downloads the agent configuration package to update an existing Logbeat installation
- Download the agent installation or update package, then click
Finishto close the data source window.
Metrics Collected
| Name | Description | Data Type |
|---|---|---|
| @timestamp | The original timestamp as a string, often representing the time the data was recorded or received. | String |
| timestamp | The precise timestamp of the event, with microsecond precision. | DateTime64 |
| host | The hostname or IP address of the machine where the event originated. | String |
| tenant_id | The unique identifier for the tenant (user or customer) associated with this event. | UInt64 |
| bu_id | The unique identifier for the business unit associated with this event. | UInt64 |
| server | The name or identifier of the server handling the requests. | String |
| src_host | The source host from which the request originated. | String |
| src_instance | The source instance or specific instance of the application sending the request. | String |
| name | The name associated with this particular set of metrics, often the service or application name. | String |
| port | The port number on which the server is listening for incoming requests. | UInt64 |
| requests | The total number of requests received by the server or application. | UInt64 |
| accepts | The number of accepted connections. | UInt64 |
| handled | The number of successfully handled connections or requests. | UInt64 |
| reading | The number of connections where the server is currently reading the request. | UInt64 |
| writing | The number of connections where the server is currently writing a response. | UInt64 |
| waiting | The number of connections that are currently idle, waiting for a request. | UInt64 |
| active | The number of active connections at the time of measurement. | Float64 |
| dropped | The number of connections that were dropped or refused. | Float64 |
| requests_diff | The difference in the number of requests since the last measurement period. | Float64 |
| accepts_diff | The difference in the number of accepted connections since the last measurement period. | Float64 |
| handled_diff | The difference in the number of handled connections since the last measurement period. | Float64 |
| dropped_diff | The difference in the number of dropped connections since the last measurement period. | Float64 |
| requests_rate | The rate of requests per second. | Float64 |
| period | The duration of the measurement period, typically in seconds. | Float64 |
| accepts_rate | The rate of accepted connections per second. | Float64 |
| handled_rate | The rate of handled connections per second. | Float64 |
| dropped_rate | The rate of dropped connections per second. | Float64 |
| telegraf_unique_id | A unique identifier for the metrics record, often generated by the Telegraf agent collecting the data. | String |
| nginx_host | The hostname or IP address of the NGINX server that is handling the requests. | String |
| timestamp | The precise timestamp of the event, with microsecond precision. | DateTime64 |
| message | The log message content. | String |
| beat | The name or identifier of the Beat (e.g., Filebeat, Metricbeat) that collected the log. | String |
| bu_id | The unique identifier for the business unit associated with this event. | UInt64 |
| bytes_sent | The total number of bytes sent in the request/response. | UInt64 |
| server_addr | The IP address of the server that processed the request. | String |
| target | The target endpoint or resource being requested. | String |
| beat_hostname | The hostname of the machine where the Beat is running. | String |
| connections_active | The number of active connections at the time of the log entry. | UInt64 |
| connections_reading | The number of connections currently reading requests. | UInt64 |
| connection_requests | The number of connection requests. | UInt64 |
| connections_waiting | The number of connections currently idle, waiting for a request. | UInt64 |
| connections_writing | The number of connections currently writing a response. | UInt64 |
| content_length | The length of the content in the request/response. | UInt64 |
| host | The hostname of the server processing the request. | String |
| log_file_path | The file path of the log file where this entry is recorded. | String |
| source_id | A unique identifier for the source of the log entry. | String |
| topic_name | The name of the topic, if logs are being collected and organized by topic (e.g., in Kafka). | String |
| http_response | The HTTP response code returned by the server. | String |
| limit_rate | The rate limit applied to the request, if any. | UInt64 |
| msec | The time in milliseconds at which the event occurred. | Float64 |
| nginx_host | The hostname of the NGINX server that processed the request. | String |
| nginx_port | The port on which the NGINX server is listening. | UInt64 |
| pid | The process ID of the NGINX process handling the request. | String |
| proxy_host | The hostname of the proxy server, if the request passed through one. | String |
| proxy_port | The port of the proxy server. | String |
| realip_remote_port | The real IP address's remote port from where the request originated. | Float64 |
| remote_addr | The IP address of the client making the request. | String |
| realip_remote_addr | The actual remote IP address, especially useful if behind a proxy. | String |
| remote_port | The remote port from which the request was made. | String |
| client_ip | The IP address of the client, potentially different from remote_addr if the request passed through a proxy. | String |
| connection | The connection details or ID associated with the request. | String |
| error_message | The error message, if any occurred during processing. | String |
| log_level | The severity level of the log (e.g., INFO, ERROR, WARN). | String |
| upstream_url | The URL of the upstream server or service that handled the request. | String |
| tid | The thread ID of the process handling the request. | String |
| request | The full HTTP request line sent by the client. | String |
| request_method | The HTTP method used in the request (e.g., GET, POST). | String |
| request_uri | The URI or path requested by the client. | String |
| request_length | The length of the request in bytes. | UInt64 |
| request_time | The time taken to process the request, in seconds. | Float64 |
| request_filename | The name of the file or resource requested, if applicable. | String |
| server_name | The name of the server handling the request. | String |
| server_port | The port number on which the server is listening. | String |
| server_protocol | The protocol used by the server (e.g., HTTP/1.1, HTTP/2). | String |
| ssl_session_id | The SSL session ID, if the request used SSL/TLS. | String |
| ssl_server_name | The server name indicated by the client during SSL handshake. | String |
| ssl_protocol | The SSL/TLS protocol used for the connection. | String |
| o11ysource_name | The name of the observability source, if part of a broader observability platform. | String |
| ssl_client_verify | The result of SSL client certificate verification. | String |
| ssl_session_reused | Whether the SSL session was reused (e.g., YES or NO). | String |
| status | The HTTP status code returned by the server (e.g., 200, 404). | UInt64 |
| tenant_id | The unique identifier for the tenant (user or customer) associated with this event. | UInt64 |
| type | The type of event or log entry. | String |
| upstream_addr | The IP address of the upstream server that handled the request. | String |
| upstream_connect_time | The time taken to establish a connection to the upstream server. | Float64 |
| upstream_response_time | The time taken by the upstream server to respond. | Float64 |
| upstream_bytes_sent | The number of bytes sent to the upstream server. | UInt64 |
| upstream_bytes_received | The number of bytes received from the upstream server. | UInt64 |
| upstream_header_time | The time taken to receive the response headers from the upstream server. | Float64 |
| upstream_response_length | The length of the response received from the upstream server. | UInt64 |
| upstream_cache_status | The cache status of the response from the upstream server (e.g., HIT, MISS). | String |
