Logging a Forge server to Papertrail with encryption in transit
Recently I had to setup a central logging system for an application deployed with Laravel Forge. I decided to use Papertrail, since you can setup it using Forge. I've found a tutorial by Matt Stauffer, which helped me to get the ball rolling.
The only thing that bothered me was that logs transferred to Papertrail use UDP by default. As UDP is a stateless "fire-and-forget" transfer protocol, there's no security in place by default. This is good for performance, but a major drawback for mission critical applications, where you have to protect data as well as logs.
Transferring log data without encryption in transit means everyone can have a look at what's going on in your application and gains a certain amount of insight into your app. Just by looking at the contents of transferred log entries someone could collect information about usage patterns of your application, even if your log messages do not contain sensitive information.
This is especially a problem if you use an external service for logging like Papertrail, which is located outside your private network and therefore outside of your control. Keep in mind that logs should be kept free of personal identifiable information anyways. Remember to think about strategies to pseudonymize and anonymize sensitive data in logs ahead of time.
Luckily, you can encrypt your logs in transit using TCP and TLS, which is a bit slower than sending via UDP but worth the effort in terms of additional security.
Installing Papertrail on a Laravel Forge Server
Let me show you the basic steps on how to configure logging with Papertrail. Make sure that basic logging works before you add encryption to the stack. It might take a couple of minutes until generated log messages show up in Papertrail.
- Sign up for a Papertrail Account
- Go to "Settings > Log Destinations" and copy your log destintation's URL, e.g.
logN.papertrailapp.com:XXXXX
- Use Forge to install Papertrail on the "Monitoring" page of your server by specifying your log destination's URL
- Head over to Papertrail and check if logging works, e.g.
logger "test"
Encrypt log messages in transit
After you've got basic logging working, you can enable TLS-encryption in transit for log messages transferred to Papertrail. rsyslog
is installed on Laravel Forge provisioned servers by default.
- Run
sudo curl -o /etc/papertrail-bundle.pem https://papertrailapp.com/tools/papertrail-bundle.pem
to download Papertrail's CA to your server - Run
sudo apt install rsyslog-gnutls
to install the TLS extension forrsyslog
- Open
/etc/rsyslog.d/20-papertrail.conf
and replace its contents with the configuration provided below, keep in mind to update the last line with your log destination's URL*.* @@logsN.papertrailapp.com:XXXXX
$DefaultNetstreamDriverCAFile /etc/papertrail-bundle.pem # trust these CAs $ActionSendStreamDriver gtls # use gtls netstream driver $ActionSendStreamDriverMode 1 # require TLS $ActionSendStreamDriverAuthMode x509/name # authenticate by hostname $ActionSendStreamDriverPermittedPeer *.papertrailapp.com *.* @@logsN.papertrailapp.com:XXXXX
- Run
sudo service rsyslog restart
- Head over to Papertrail and check if logging still works
- Uncheck "Accept connections via UDP (Plain text)" in "Settings > Log Destinations > Destination Settings" and "Accept logs from unrecognized systems?" as soon as you have configured every system that should log to your destination.
- Test again
Papertrail recommends some tweaks to your configuration to queue log messages if the TLS connection drops in between, check out their help page to find out more. If logging is not working, use sudo tcpdump -n -s 1500 -X port XXXXX
to monitor your network transfer. A new entry should be generated everytime you send a message using logger "test"
.
I hope you've enjoyed this post on your journey towards more data protection. Tell me what you think, I'm @skollro on Twitter. If you're interested in data protection, check out my eBook GDPR for Developers.