Send Apache 2.4 access log to Graylog over TCP

Posted on by

Graylog accepts a format called GELF. GELF is short for Graylog Extended Log Format. GELF appears like a straightforward JSON. Apache 2.4 logs are not in JSON format. This short blog helps explain two things:

  1. Configure Apache 2.4’s /etc/apache2.conf to log GELF
  2. Configure Apache 2.4’s site /etc/apache2/sites-enabled/000-default.conf to pass new log lines to a custom shell script
  3. Create shell script that sends information to Graylog server using TCP
Note: At the time of this writing, GELF TCP requires a null byte (which is 
Note: At the time of this writing, GELF TCP requires a null byte (which is \0) at the end of the string we intend to send
) at the end of the string we intend to send

Configure /etc/apache2.conf to log GELF format

  • Edit /etc/apache2.conf
  • Notice some existing LogFormat declarations. Below those declarations add this line
    LogFormat "{ \"version\": \"1.1\", \"host\": \"my-cool-vm\", \"short_message\": \"%r\", \"full_message\": \"%r, status: %>s, %O bytes, User Agent: %{User-Agent}i\", \"level\": 6, \"_user_agent\": \"%{User-Agent}i\", \"_source_ip\": \"%a\", \"_duration_usec\": %D, \"_duration_sec\": %T, \"_request_size_byte\": %O, \"_http_status\": %s, \"_http_request_path\": \"%U\", \"_http_request\": \"%U%q\", \"_http_method\": \"%m\", \"_http_referer\": \"%{Referer}i\" }" graylog_access
  • Apache will create a GELF log format that is consumed by our site’s custom logging

Configure /etc/apache2/sites-enabled/000-default.conf to consume GELF log

  • Edit /etc/apache2/sites-enabled/000-default.conf
  • Notice some existing ErrorLog and CustomLog declarations. Below those declarations, add this line
    CustomLog "||/home/me/test.sh" graylog_access
  • We are instructing Apache’s default site to consume logged information and send it to test.sh script

Create a shell script to send information to Graylog over TCP

/home/me/test.sh

#!/bin/bash

while read logline; do
        echo -e "${logline}
#!/bin/bash
while read logline; do
echo -e "${logline}\0" | nc 192.168.100.200 12201
done
" | nc 192.168.100.200 12201
done

Make this file executable:

chmod 755 /home/me/test.sh

Restart Apache (Ubuntu 16.04)

sudo systemctl restart apache2.service

Watch data in Graylog

Resources

Several resources helped in this journey