Apache mod_security on CentOS 5 x86_64

In the journey of rebuilding my dedicated server to create a  secured cloud server, I was trying to find out what are the potential issues, besides network intrusion through various ports, and services on my server. I found numbers ranging from 70% to 85% of all intrusions actually occur by way of applications. Specifically, I am running many php and a couple Perl applications. Not to mention, I was allowing MySql to be accessed external to my machine in the past.

As I googled various options to secure Apache, I found mod_security module (http://www.modsecurity.org/).

So I started off by trying several different tutorials (listed below). So it all seemed fairly straight forward from the examples.

1. Install the plugin: yum, rpm or compile
2. Load module in mod_security.conf: or directly into httpd.conf
3. Load Rules: define them manually, and/or load them from additional configuration files

I spent several very long days trying to get this to work.

The first time I tried to get this to work, I just got the plugin from the CentOS yum repository. Then I configured the plugin, no errors where thrown, and Apache seemed to be working fine. I wanted to try to understand how to test or verify what I have done.

I first used a simple example from http://blog.bodhizazen.net/linux/how-to-mod_security-ubuntu-904/ that has you create simple php file on your server

<em>//bad.php
&lt;?  $secret_file = $_GET['secret_file']; include ( $secret_file); ?&gt;</em>

http://www.yourdomain.com/bad.php?secret_file=/etc/passwd

At first I kept getting a blank page, then I added some html that would print <h2>bad.php</h2> just to ensure the page was loaded. The page was succeeding, and not giving a 403 error as expected. I then tried to use the curl command that was given on the example

<em>[[code]]czoxMTM5OlwidXNlcjFAaG9tZSMgPHN0cm9uZz5jdXJsIC1pIFxcXCJodHRwOi8vd3d3LnlvdXJkb21haW4uY29tL2JhZC5waHA/c2VjcntbJiomXX1ldF9maWxlPS9ldGMvcGFzc3dkXFxcIjwvc3Ryb25nPg0KSFRUUC8xLjEgMjAwIE9LDQpEYXRlOiBUdWUsIDI4IEFwciAyMDA5IDIyOjJ7WyYqJl19NDoxMSBHTVQNClNlcnZlcjogQXBhY2hlLzIuMg0KVmFyeTogQWNjZXB0LUVuY29kaW5nDQpDb250ZW50LUxlbmd0aDogODYwDQpDb3tbJiomXX1udGVudC1UeXBlOiB0ZXh0L2h0bWwNCjxzdHJvbmc+cm9vdDp4OjA6MDpyb290Oi9yb290Oi9iaW4vYmFzaA0KZGFlbW9uOng6MToxe1smKiZdfTpkYWVtb246L3Vzci9zYmluOi9iaW4vc2gNCmJpbjp4OjI6MjpiaW46L2JpbjovYmluL3NoDQpzeXM6eDozOjM6c3lzOi9kZXY6L2J7WyYqJl19aW4vc2gNCnN5bmM6eDo0OjY1NTM0OnN5bmM6L2JpbjovYmluL3N5bmMNCmdhbWVzOng6NTo2MDpnYW1lczovdXNyL2dhbWVzOi9iaXtbJiomXX1uL3NoDQptYW46eDo2OjEyOm1hbjovdmFyL2NhY2hlL21hbjovYmluL3NoDQpscDp4Ojc6NzpscDovdmFyL3Nwb29sL2xwZDovYmlue1smKiZdfS9zaA0KbWFpbDp4Ojg6ODptYWlsOi92YXIvbWFpbDovYmluL3NoDQpuZXdzOng6OTo5Om5ld3M6L3Zhci9zcG9vbC9uZXdzOi9iaW57WyYqJl19L3NoDQp1dWNwOng6MTA6MTA6dXVjcDovdmFyL3Nwb29sL3V1Y3A6L2Jpbi9zaA0KcHJveHk6eDoxMzoxMzpwcm94eTovYmluOi9iaXtbJiomXX1uL3NoDQp3d3ctZGF0YTp4OjMzOjMzOnd3dy1kYXRhOi92YXIvd3d3Oi9iaW4vc2gNCmJhY2t1cDp4OjM0OjM0OmJhY2t1cDovdmFye1smKiZdfS9iYWNrdXBzOi9iaW4vc2gNCmxpc3Q6eDozODozODpNYWlsaW5nIExpc3QgTWFuYWdlcjovdmFyL2xpc3Q6L2Jpbi9zaA0KaXJjOnh7WyYqJl19OjM5OjM5OmlyY2Q6L3Zhci9ydW4vaXJjZDovYmluL3NoDQpnbmF0czp4OjQxOjQxOkduYXRzIEJ1Zy1SZXBvcnRpbmcgU3lzdGVtIHtbJiomXX0oYWRtaW4pOi92YXIvbGliL2duYXRzOi9iaW4vc2gNCm5vYm9keTp4OjY1NTM0OjY1NTM0Om5vYm9keTovbm9uZXhpc3RlbnQ6L2Jpe1smKiZdfW4vc2gNCmxpYnV1aWQ6eDoxMDA6MTAxOjovdmFyL2xpYi9saWJ1dWlkOi9iaW4vc2gNCnNzaGQ6eDoxMDE6NjU1MzQ6Oi92YXIvcnV7WyYqJl19bi9zc2hkOi91c3Ivc2Jpbi9ub2xvZ2luDQpwb3N0Zml4Ong6MTA0OjEwNzo6L3Zhci9zcG9vbC9wb3N0Zml4Oi9iaW4vZmFsc2U8L3tbJiomXX1zdHJvbmc+XCI7e1smKiZdfQ==[[/code]]</em>

Ok, so I setup mod_security what’s going on then?

Well, the example I ended up did not Load any libraries as documented in the modsecurity.org install guide. I thought, these examples (several of them), all professed to work, why does mine not work?

The answer has not been easy, and unfortunately, not 100% clear either. I will list through the 3 main items I had trouble with and the solution I found to finally get this to work

1. Correctly loading x86_64 libraries.

As I described an issue with several different libraries in the blog http://baselogic.com/blog/archives/294 This was effecting many items in my server configuration. I also found that my libxml2 was having issues while I was hacking different configurations to get something to either work, or to throw some errors. Once this issue was solved, I go to thinking about my second issue:

2. Get correct mod_security module for x86_64.

As with my libxml2 and other libraries, I started tracking down what version of mod_security I was actually running. I was running a version I got through some means in one of these tutorials. Then I found an updated module from Jason Litka (http://www.jasonlitka.com/2007/08/24/mod-security-packages-now-available/). The install went fine, and even came with a new configuration file and rules. But then I kept getting errors trying run a configcheck on Apache:

<em>Starting httpd: httpd:
Syntax error on line 210 of /etc/httpd/conf/httpd.conf:
Syntax error on line 5 of /etc/httpd/conf.d/mod_security.conf:
Cannot load /etc/httpd/modules/mod_security2.so into server:
/etc/httpd/modules/mod_security2.so:
<strong>undefined symbol: ap_get_server_banner</strong></em>

The solution alluded to trying to rebuild apache and other modules, and I was not interested in compiling Apache to get this to work. I wanted to use yum as much as possible. So I kept researching, and quickly realized that the latest version I could find via any yum repository, was 2.5.0, yet on modsecurity.org, the latest available version was 2.5.9 ion source version, as I did not see x86_64 CentOS listed for the update version.

Well, I downloaded this package and installed this manually because I had no other choice. But it was very easy. Now even though at the time I did not know it, this issue was resolved.

3. Find the Configuration that properly works.

After installing x86_64 version of mod_security, I still was unable to get even a simple test to work, thus my module was not working.

After several really long days hacking at various options over and over again. I was quite frustrated that, what I thought to be a simple configuration and module was such an issue to resolve. This is the part of the solution that is not 100% clear to me. I believe I had tried this configuration before, but somehow on m=one of my hacks, this configuration worked:

LoadFile /usr/lib64/libxml2.so
LoadFile /usr/lib64/liblua.so.5.0

#Load mod_unique_id and mod_security 2
LoadModule unique_id_module modules/mod_unique_id.so
LoadModule security2_module modules/mod_security2.so
<IfModule mod_security2.c>
Include modsecurity.d/*asl*.conf

#Enable mod_security
SecRuleEngine On
SecDefaultAction log,auditlog,deny,status:403,phase:2,t:lowercase,t:replaceNulls,t:compressWhitespace

#Log Configuration
SecAuditEngine RelevantOnly
SecAuditLogParts ABCFHZ
SecAuditLogType Serial
SecAuditLog logs/mod_security2.log

#SecAuditLogStorageDir logs/audit
SecAuditLogRelevantStatus ^(?:5|4\d[^4])

#General Settings
#SecTmpDir /temp
#SecUploadDir /temp/uploads
#SecUploadKeepFiles RelevantOnly

#Rules
</IfModule>

<em>[[code]]czo5ODpcInVzZXIxQGhvbWUjIDxzdHJvbmc+Y3VybCAtaSBcXFwiaHR0cDovL3d3dy55b3VyZG9tYWluLmNvbS9iYWQucGhwP3NlY3JldHtbJiomXX1fZmlsZT0vZXRjL3Bhc3N3ZFxcXCI8L3N0cm9uZz5cIjt7WyYqJl19[[/code]]</em><em>
HTTP/1.1 403 Forbidden
Date: Thu, 30 Apr 2009 17:20:03 GMT
Server: Apache
Content-Length: 283
Content-Type: text/html; charset=iso-8859-1</em>

<em>&lt;!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"&gt;
&lt;html&gt;&lt;head&gt;
<strong>&lt;title&gt;403 Forbidden&lt;/title&gt;</strong>
&lt;/head&gt;&lt;body&gt;
&lt;h1&gt;Forbidden&lt;/h1&gt;
&lt;p&gt;You don't have permission to access /bad.php
on this server.&lt;/p&gt;
&lt;hr&gt;
&lt;address&gt;Apache Server at yourdomain.com Port 80&lt;/address&gt;
&lt;/body&gt;&lt;/html&gt;</em>

Conclusion:

Although x86_64 seems to not be widely used and supported with many tutorials, I can say that once I was able to get this plugin working, I was able to quickly and easily see the benefits from various web application intrusions. I also gained a better insight as to how to identify OS issues and how to notice them next time.

References:

• ModSecurity: http://www.modsecurity.org
• How-To Forge: http://www.howtoforge.com/apache_mod_security
• Jason Litka: http://www.jasonlitka.com/2007/08/24/mod-security-packages-now-available/
• My Whiteboard: http://www.my-whiteboard.com/linux-admin/protect-your-web-server-from-security-attacks-using-modsecurity.html
• Atomic: http://www.atomicorp.com/wiki/index.php/Atomic_ModSecurity_Rules

the end…