Apache
Introduction
The Apache HTTP Server, commonly referred to as Apache ,is an open-source web server software that serves as the foundation for the majority of websites on the internet.
Getting Started
Compatibility
The Apache O11ySource is compatible with all versions greater than or equal to 2.2.31 and 2.4.16
Data Collection Method
vuSmartMaps collects health and performance data for Apache server using VuNet's Healthbeat agent. This agent collects data based on the source configuration. Logbeat agent is used to collect Apache Logs.
Prerequisites
Inputs for Configuring Data Source
- Apache Server: The IP Address/FQDN of the Apache server. This field is the key to identify each server you add here.
- Period [in seconds]: How frequently data is gathered. The period should be between 60 seconds – 3000 seconds.
- URL: The server URL to Apache server-status page. Eg: http://127.0.0.1, https://127.0.0.1, http://127.0.0.1:8080 or https://127.0.0.1:8080
- Access Log Path: Enter the Apache access logfile's entire path
- Error Log Path: Enter the Apache error logfile's entire path.
Firewall Requirement
To collect data from this O11ySource, ensure the following ports are opened:
| Source IP | Destination IP | Destination Port | Protocol | Direction |
|---|---|---|---|---|
| IP address of the Apache server | Data Collector End Point | 9092* | TCP | Inbound |
*Before providing the firewall requirements, please update the port based on the customer environment.
Configuring the Target
Configure Metrics Collection from Apache Server Apache metrics are collected by VuNet's Healthbeat agent by connecting to the server-status module. The Apache web server includes the mod status to allow web browsers to query the status of the server. This module makes it simple to determine how well the server is doing. Every report is produced in HTML format.
- To check whether the status module is active or not, use
apache2ctl -Morls /etc/apache2/mods-enabled. If not, you can enable it by runningsudo a2enmod status. - As seen below, you must enable the status module and make sure that ExtendedStatus is active in order to gather metrics from Apache. To indicate which IP addresses should view the status page, you must alter the block that begins with
<Location /server-status>(either in your status module's configuration file or the main Apache configuration file). In the example that follows, localhost access is permitted.
<IfModule mod_status.c>
<Location /server-status>
SetHandler server-status
order deny,allow
allow from localhost
</Location>
ExtendedStatus On
</IfModule>
- Save your modifications, then quit the window.
- The status modules configuration file is located in
<path-to-apache2\>/mods-enabled/status.conffor Debian users. Users of other platforms can find their primary configuration file in/\<path-to-apache\>/apache2.conf, //conf/httpd.confor/\<path-to-apache2\>/httpd.conf - To implement the changes without disrupting live connections, use a graceful restart
apachectl -k gracefulorservice apache2 gracefulConfigure Logs Collection from Apache Server The Apache access log offers even more specific information about each client request than the status module does. Your Apache logs can be customized to include information that is more pertinent to your requirements. Apache logs are collected by VuNet's Logbeat agent. - Find the section beginning with 'LogFormat' in your main Apache configuration file.
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
- Log variables can be used to modify an existing LogFormat or develop a unique format. We could modify the current logFormat as shown below and include the variable
%Dto add a new design that is identical to the combined log format but also logs the request delay in microseconds
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\" %D" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\" %D" combined
Configuration Steps
- Enable the Apache O11ySource.
- Select the Sources tab and press the
+button to add a new Apache host to be monitored. - Set up metrics and log collection configurations. Afterwards, select
Save and Continueto proceed with downloading the Healthbeat and Logbeat agent. - The following packages will be available for download based on the OS:Download <OS> Healthbeat Installation Binary - Downloads the full Healthbeat package with required configurations for a fresh installationDownload <OS> Healthbeat Update Configuration Binary - Downloads the agent configuration package to update an existing Healthbeat installationDownload <OS> Logbeat Installation Binary - Downloads the full Logbeat package with required configurations for a fresh installationDownload <OS> Logbeat Update Configuration Binary - Downloads the agent configuration package to update an existing Logbeat installation
- Download the agent installation or update package, then click
Finishto close the data source window.
Metrics Collected
| Name | Description | Data Type |
|---|---|---|
| @timestamp | The original string timestamp from the source log. | String |
| timestamp | The timestamp for the data point with high precision (64-bit). | DateTime64 |
| target | The target for the metrics, typically identifying the specific system or service being monitored. | String |
| host | Hostname of the server where the metrics were collected. | String |
| tenant_id | Tenant identifier for multi-tenant environments. | LowCardinality(String) |
| bu_id | Business Unit identifier for organizational grouping. | LowCardinality(String) |
| Type | Type of service being monitored (e.g., Apache, Nginx, etc.). | String |
| Uptime | Time in seconds that the server or service has been running since the last restart. | Float64 |
| Status | Current status of the server or service (e.g., "Running", "Stopped"). | LowCardinality(String) |
| service_type | The type of service provided (e.g., HTTP, HTTPS). | String |
| source_id | Unique identifier for the source from which the metrics are collected. | String |
| service_address | The address of the service (e.g., IP address or URL). | String |
| vublock_name | Block name or virtual unit block within a larger system. | String |
| metricset_name | The name of the specific metric set (group of related metrics). | String |
| type | Type of data point being recorded (e.g., gauge, counter). | String |
| period | Time period for which the metrics are being collected (in seconds). | Float64 |
| topic_name | Name of the topic related to the metric, such as a specific service or function. | LowCardinality(String) |
| requests_per_sec | Number of HTTP requests per second. | Float64 |
| hostname | The hostname of the server where Apache is running. | String |
| uptime_uptime | The uptime of the current Apache process in seconds. | Float64 |
| uptime_server_uptime | Total uptime of the Apache server since the last restart. | Float64 |
| cpu_load | The current CPU load (the average amount of work the CPU has to do). | Float64 |
| cpu_user | The percentage of CPU time spent in user space. | Float64 |
| cpu_system | The percentage of CPU time spent in kernel space. | Float64 |
| cpu_children_user | CPU time spent by child processes in user mode. | Float64 |
| cpu_children_system | CPU time spent by child processes in kernel mode. | Float64 |
| cpu_system_diff | Difference in system CPU usage between the last two metric collections. | Float64 |
| cpu_system_rate | Rate of system CPU usage. | Float64 |
| cpu_user_diff | Difference in user CPU usage between the last two metric collections. | Float64 |
| cpu_user_rate | Rate of user CPU usage. | Float64 |
| scoreboard_waiting_for_connection | Number of workers waiting for a connection. | UInt64 |
| scoreboard_reading_request | Number of workers reading requests. | UInt64 |
| scoreboard_dns_lookup | Number of workers performing DNS lookups. | UInt64 |
| scoreboard_closing_connection | Number of workers closing connections. | UInt64 |
| scoreboard_gracefully_finishing | Number of workers gracefully finishing connections. | UInt64 |
| scoreboard_idle_cleanup | Number of workers cleaning up idle connections. | UInt64 |
| scoreboard_keepalive | Number of workers handling keep-alive connections. | UInt64 |
| scoreboard_total | Total number of scoreboard slots used. | UInt64 |
| scoreboard_logging | Number of workers writing log entries. | UInt64 |
| scoreboard_open_slot | Number of open slots in the scoreboard. | UInt64 |
| scoreboard_starting_up | Number of workers in the process of starting up. | UInt64 |
| scoreboard_sending_reply | Number of workers sending replies to clients. | UInt64 |
| load_15 | System load average over the last 15 minutes. | Float64 |
| load_1 | System load average over the last 1 minute. | Float64 |
| load_5 | System load average over the last 5 minutes. | Float64 |
| bytes_per_sec | The number of bytes served per second. | Float64 |
| connections_total | Total number of connections handled. | Float64 |
| connections_async_writing | Number of asynchronous writing connections. | Float64 |
| connections_async_keep_alive | Number of asynchronous keep-alive connections. | Float64 |
| connections_async_closing | Number of asynchronous closing connections. | Float64 |
| workers_busy | Number of busy workers at the time of data collection. | Float64 |
| workers_idle | Number of idle workers at the time of data collection. | Float64 |
| total_accesses | Total number of accesses (requests) made to the server. | Float64 |
| bytes_per_request | Average number of bytes per request. | Float64 |
| total_kbytes | Total number of kilobytes served. | Float64 |
| total_accesses_diff | Difference in the number of total accesses since the last metric collection. | UInt32 |
| total_kbytes_diff | Difference in total kilobytes served since the last metric collection. | UInt64 |
| total_kbytes_rate | Rate at which kilobytes are being served. | Float64 |
| total_cpu | Total CPU time used by the server. | Float64 |
| total_cpu_pct | Percentage of total CPU utilization. | Float64 |
| busy_workers_pct | Percentage of workers that are busy. | Float64 |
| idle_workers_pct | Percentage of workers that are idle. | Float64 |
| timestamp | The timestamp when the log entry was recorded with high precision (64-bit). | DateTime64 |
| target | The target for the log or metrics, usually the application or server. | String |
| host | The hostname of the server where the log was generated. | String |
| apache2_access_referrer | The URL of the referring page from where the request originated. | String |
| apache2_access_response_code | HTTP response code returned by the server (e.g., 200 for success, 404 for not found). | UInt64 |
| apache2_access_body_sent_bytes | Number of bytes sent in the response body. | UInt64 |
| apache2_access_url | The URL that was accessed by the client. | String |
| apache2_access_user_name | The username of the authenticated user (if available). | String |
| apache2_access_remote_ip | The IP address of the client making the request. | String |
| apache2_access_method | The HTTP method used for the request (e.g., GET, POST). | LowCardinality(String) |
| apache2_access_user_agent_name | Name of the user's browser or client application (user-agent). | String |
| apache2_access_user_agent_minor | Minor version number of the user-agent. | UInt64 |
| apache2_access_user_agent_os_name | Operating system of the client accessing the server. | String |
| apache2_access_tat | Turn-around time for the request (time taken to process the request, in seconds). | Float64 |
| apache2_access_user_agent_device | The type of device making the request (e.g., mobile, desktop). | LowCardinality(String) |
| http-code | HTTP response code, similar to apache2_access_response_code. | UInt64 |
| response | The response body or message returned by the server, if applicable. | String |
| geo_country_name | Geographical location of the client, specifically the country name. | String |
| geo_timezone | Timezone of the client’s geographical location. | String |
| geo_city_name | Geographical location of the client, specifically the city name. | String |
| doc_type | Type of document or log entry (e.g., access log, error log). | LowCardinality(String) |
| source_id | Unique identifier for the source of the log entry. | LowCardinality(String) |
| vublock_name | Name of the virtual block or unit block within the system. | LowCardinality(String) |
| o11ysource_name | Name of the observability source or platform collecting the logs (e.g., a monitoring tool or service). | LowCardinality(String) |
| tenant_id | Identifier for the tenant in multi-tenant environments. | LowCardinality(String) |
| bu_id | Business Unit identifier, typically for organizational purposes. | LowCardinality(String) |
| type | Type of log entry, representing different kinds of data points (e.g., access logs, error logs). | LowCardinality(String) |
| message | The full log message or entry. | String |
