In 2004, when I was starting a new job at the National Institute on Aging's Intramural Research Program I began evaluating products to meet FISMA requirements for file integrity monitoring. We already purchased a copy of Tripwire, but I was being driven mad by the volume of alerting from the system. I wanted something open source. I wanted something that would save me time, rather than waste 2 hours a day clicking through a GUI confirming file changes caused by system updates and daily operations.
At the time, I found two projects: Samhain and OSSEC-HIDS. Samhain is a great project that does one thing and does that one thing very well. However, I was buried in a mountain of FISMA compliance requirements and OSSEC offered more than file integrity monitoring; OSSEC offered a framework for distributed analysis of logs, file changes, and other anomalous events in the same open source project.
I now work at Booking.com and manage one of the world's largest distributions of OSSEC-HIDS. My team and I are active contributors to the OSSEC Community. After nearly a decade of experience deploying, managing, and extracting value from OSSEC, I was approached to write a book introducing new users to OSSEC. After 6 months of work, the book has been published!
Writing a book is hard, kids.
I write a lot. I thought, "writing a book would be fun and easy!" And, in some ways it is. It's also very challenging writing for an editor. It's also very humbling to receive feedback on your book from the technical reviewers. Most of my writing, never makes it anywhere public. Even the writing that ends up on my blog generates very little feedback. Now, imagine someone scrutinizing every single technical detail 50 pages of your writing. Every sentence fact checked and marked up. Every mistake you made highlighted and commented by multiple technical reviewers. That's hard.
It doesn't stop there. From the first draft, the editor points out every instance of awkward wording or incorrect grammar. Once your editors and technical reviewers are satisfied, the book moves on to the Technical Editor. This person is responsible for ensuring language consistency and compliance to the conventions of the dialect of which the book is written. If you're not comfortable with your work being torn apart and reassembled, writing a book may not be for you. It was grueling at times.
It does help to have a good group of people involved in the process. I was thankful to have a great team of editors and reviewers for this book at Packt Publishing. Now, that the job is done (side note: the job is never done), I am grateful to have been given this opportunity. It was a very rewarding experience, and like all rewarding experiences, it wasn't easy.
OSSEC-HIDS and so can you!
So, what's this OSSEC-HIDS this book is all about? I, like most of you, struggle with putting my logging data to good use. I have several posts on this blog dedicated to extracting value from logging. I also have compliance requirements, previously FISMA, currently PCI-DSS. OSSEC is an open source security project which incorporates a number of useful features that make it worth a look.
- Distributed log analysis
- File integrity monitoring
- Rootkit detection
- Policy auditing
- Alerting and active response system
All of these features interoperate in the same event system, providing richer context in an infinitely customizable environment.
Distributed log analysis
OSSEC runs best in a client/server model. You can have hybrid deployments to aggregate and analyze events at different points in your network. The clients run as agents, propagating log data to the servers. The servers than evaluate the log data using decoders to extract data and rules to analyze the data.
OSSEC ships with an impressive list of decoders and rules capable of analyzing and classifying events from standard system services including, but not limited to : ssh, ftp, email, web servers, LDAP, ActiveDirectory, Windows EventLogs, Windows Registry, Cisco PIX, Juniper NetScreens, and most Linux/BSD/Solaris system daemons and kernel messaging.
The OSSEC decoders and rules are extensible, so if your use case isn't addressed it's easy to add.
File integrity monitoring
If you work in IT security, you hate file integrity monitoring. It's because of products like Tripwire, which do file integrity monitoring, but are so loud and obnoxious about it, that the value is drowned in a sea of unending harassment of correct, but uninteresting events. I desperately wanted out of what I found largely unactionable alerts and incessant pointing and clicking.
OSSEC's implementation of FIM is similar to every other product in this space. You can monitor changes in modification times, access times, checksums (MD5 and SHA1), owners, groups, and permissions or any combination of those attributes. OSSEC's file integrity events occur inside the same framework as the log analysis, so it's possible to write rules to evaluate and correlate them. It's even possible to fire off scripts when they occur to perform validation against other databases.
Just this week I discovered this post on Improving file integrity monitoring with OSSEC that allows the servers to keep a validated list of checksums to ignore! It's also possible to audit the Windows Registry for changes using OSSEC's file integrity daemon.
Rootkit detection by itself isn't that valuable. In the context of the logging events and file integrity data, it can be invaluable in quickly identifying compromised servers. OSSEC uses a rootkit and trojan'd database of file checksums to detect bad files on your system. It also performs checks similar to chkrootkit or AIDE by looking for hidden files, programs, or open ports and checking for out of place files and directories.
Again, the events occur inside the OSSEC framework and can be customized using the OSSEC rules.
If you find yourself in an environment covered by a set of IT regulations, you may have to perform audits of critical daemons configurations to verify you are implementing "Industry Best Practices." While, as a rational human being, the phrase "industry best practices" makes my skin crawl, security professionals need to establish and ensure compliance with a security baseline. OSSEC provides a policy auditing framework which runs alongside it's file integrity monitoring daemon to look for both wanted and unwanted phrases in configuration files. OSSEC ships with policies for auditing according to the Security Benchmarks from the Center for Internet Security.
Alerting and active response
The real power comes from the flexibility of OSSEC's alerting system. Using rules you can modify alert levels and perform aggregate analyis. It's simple to write a rule that says "If an IP fails logins 5 times in 10 ten minutes, alert." You may recognize this functionality as provided by the popular Fail2Ban project. The real power is OSSEC's analysis can occur across every device on your network.
With the OSSEC active response system, it's possible to implement the Fail2Ban functionality and blacklist IP's that are misbehaving. Active response is a fancy way of saying "when an alert is triggered, run a script somewhere." You can specify running the script on the agent that generated the alarm, a specific agent on your network, or every agent on your network.
The active response system is incredibly powerful and flexible. I encourage you to check the project out, whether or not you buy my book.
This post was graciously translated into Russian by a volunteer transalation team.