Celebrazio Net



Contact Us

Shipping logs - Fastly to Logz.IO

At the time of writing (Mar. 2019), no instructions existed for shipping CDN service logs (Fastly) to Logz.IO. Here is the way I found which worked.

Objectives of the Task

  • You have a fastly CDN service
  • You have a Logz.IO analytics service account
  • You want to review the Fastly logs in realtime using Logz.IO

Fastly Config

Setting Up the Listener - as follows:

Used the address provided by Logz.IO:  listener.logz.io  
  Used TLS port indicated for Syslog:  5001

Token and Domain information.
In Rsyslog over TLS Shipping instructions, we get useful information from Logz.IO: In Additional Configuration Code, Logz.IO provides

#   -------------------------------------------------------
#        File Logging Directives for Logz.io
#   -------------------------------------------------------
$ModLoad imfile
$InputFilePollInterval 10
$PrivDropToGroup adm
$WorkDirectory /var/spool/rsyslog
...
$ActionSendStreamDriverPermittedPeer *.logz.io
$template logzFormatFileTagName,"[GeGJm....Ie] <%pri%>%protocol-version% 
%timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [type=TYPE] %msg%\n"
if $programname == 'TYPE' then @@listener.logz.io:5001;logzFormatFileTagName
if $programname == 'TYPE' then ~

Your Token is found on the line beginning [ $template logzFormatFileTagName, ] The first field after the open double quote, brackets included, should go into the Fastly Token field.
Do not append a trailing space.

Choose TLS => Yes;
The TLS hostname is also included in the Syslog instructions shown above: the value at the end of [ $ActionSendStreamDriverPermittedPeer ] which is just "*.logz.io" goes into Fastly config as the TLS Hostname.

TLS Certificate => Yes;
Use the one provided by Logz.IO, including both the start

[ -----BEGIN CERTIFICATE----- ] 
and end 
[ -----END CERTIFICATE----- ]

Save and wait for your logs to start appearing.

Unfortunately, the format received at Logz.IO comes through unknown, so the next exercise will be to train the Logz.IO parsers, and create visualizations (outside the scope of this document).





1998-2024 Celebrazio.net