Options for Auditing Your Servers (and why Log File Monitoring is not enough)

In my last blog post I discussed the business reasons for auditing your server infrastructure. In this blog post I will walk you through various options to do auditing of servers, and some of the pros and cons of each.

Most auditing regimens have involved the use of systems and security log file collection and aggregation. There are dozens and dozens of vendors offering log file management and security event management. One drawback to solely relying on a log management for auditing approach is that log files generated by systems and applications often provide an incomplete picture of what’s really happened on a system. This is because they contain large amounts of inconsequential event and management data and are often not detailed enough to determine which user performed specific actions on a system that resulted in a system failure or attack.

In addition, interpreting log files is time consuming and requires specialized skills held by only a very small subset of people in the organization. Log data is useful for top-level alerting and notification of potential problems but logged events are not tied to the actions of a specific user, so troubleshooting and root-cause analysis cannot provide the accountability that security best practices and compliance regulations demand. Another critical factor organizations must consider is lack of visibility because some applications have little or no internal auditing. This can be the case with custom-built applications where auditing capabilities may not be the highest priority and developers may not understand the organization’s audit needs, including the level of detail required and importance of securing access to log data itself. Also, many enterprise applications that are highly customized may not be logging critical events.

Another approach is implementing file change auditing. A change to a critical file can sometimes reflect a significant event such as improper access to a business’ sensitive file, such as a payroll spreadsheet. Specific regulatory requirements call for use of file change tracking, including HIPAA act section 164.312 and 164.316, as well as PCI section 11.5, that say you must “deploy file-integrating monitoring software personnel to alert personnel to unauthorized modification of critical system files, configuration files or content files.” The drawback to this approach is that a lot of your critical data is stored inside databases that generic OS-specific file change tracking can’t detect, and often times the overhead associated with auditing changes to files is too prohibitive.

A third and newer approach is user-level activity monitoring. Our DirectAudit solution is a great example of a product in this category, and it is fairly unique in that it supports UNIX, Linux and Windows. Products in this area exist to increase visibility and gain a clear understanding of the intentions, actions and results of user activity on systems. This approach can also generate higher-level alerts that will point to more detailed data on the actions, events and commands that the user actually performed on the system that lead up to the alert. This important metadata can only be collected by capturing the critical user-centric data (events and screen video) and cannot be reconstructed from log data generated by systems and applications. The downsides to this approach can include the need to capture vast amounts of data in a centralized database for playback, which like the other approaches requires additional infrastructure.

In the end, your most mission-critical server should have all three approaches — log, file and user level auditing — implemented to give you a 360-degree view of what is happening on your servers. The downside of not implementing auditing could include data security breaches and loss of reputation and business, significant fines for lack of compliance, as well as loss of visibility regarding what third parties are doing on your systems. The expression “better safe than sorry” definitely applies when it comes to auditing your servers.

Finally, some customers have asked me to compare DirectAudit to log file monitoring solutions. Using some of the points made above, let me give some quick and dirty differences:

  • If the application doesn’t log it, log files won’t contain it. For example, a user opens a file and copies credit card info to the clipboard. A log file shows he accessed the file. DirectAudit shows that he copied sensitive data from the file.
  • Log files show the perspective of the application or the operating system. DirectAudit shows the perspective of the user.
  • Log rollup and monitoring tools (like Splunk or ArcSight) can show a server that is malfunctioning by turning a dashboard light red. They can even drill down to the log event that triggered the alarm. DirectAudit tells you who was on the server at that time and what they were doing. This user context is a much more natural way to find out what happened and who did it.
  • Regulatory requirements (PCI, SOX, etc.) require that you are able to recreate what a user did with privilege on a system. Which is easier? Correlating across multiple incomplete and often confounding log files or watching a movie from the user perspective.
  • Log files provide forensic exhaust from systems — there might be an explanation as to what happened, but often there is either way too much information or not nearly enough. DirectAudit is the Goldie Locks solution that just shows you who did what on a system.

Again, in the end you need both, and in many cases you may even need file monitoring. Log files give you the rawest information about what happened from the point of view of the application or system. DirectAudit gives you the only user oriented view of what happened. And Centrify is working to better integrate its solution with log and event monitoring solution, with our Splunk integration as a good example. [Thanks to Corey Williams for help on this.]