... please wait
Several months ago I started receiving the occasional alert email in the middle of the night warning that my server was hitting a threshold. Reviewing the server logs I found the usual suspects. Hits to Wordpress; admin-ajax, wp-login.php, xmlrpc.php... But also, there were a ton of hits from a web crawler that was hitting the same content over and over and over. There are plugins available to cache websites and throttle repeat content, but a hit to the CMS is still a hit and will require PHP, several image requests, file requests and more than a few hits to the database.
I decided to try to kill the traffic at the server before the CMS was even loaded.
Note: It makes more sense to add these changes to the Apache config files, if you have the ability as they will be available to all your websites in one configuration, but they will work just as well in your .htaccess file.
1. Add a deny.conf configuration file
Apache on many servers id often located in /etc/httpd, but your configuration may differ.
# sudo vi /etc/httpd/conf/deny.conf
2. Add some directives to block specific web crawlers
We will be using the Apache directive BrowserMatchNoCase which will basically just run a string comparison against the referrer header.
The format is:
bbBrowserMatchNoCase "nameofcrawler" alias
Here are a list of a few bots that always seem to hit my servers with little regard for the affect on the server:
BrowserMatchNoCase "Baiduspider" bots BrowserMatchNoCase "Yandex" bots BrowserMatchNoCase "SemrushBot" bots
This is not a definitive list...Its just a few examples to get you started.
3. Add some IP lists to block known countries where malicious traffic tends to originate.
Its true that its possible to use proxies and a well funded and determined attacker will have access to and use them...But, lets face it a well funded and determined hacker will likely find a way in if they want to. This is just a way to alleviate hits from some of the run of the mill hackers using off-the shelf tools.
We all know the countries in question, we see them on the news all the time. Do a little research and you can find resources that will give you a relatively up to date list of IP addresses to block.
Sample:
Order Allow,Deny Deny from 192.243.55.0/24 Deny from 163.172.65.0/24 Deny from 174.34.135.0/24 ... ... ... Deny from 91.200.12.0/24 Allow from all Deny from env=bots
4. Include the configuration file into your Apache configuration file.
# sudo vi /etc/httpd/conf/httpd.conf
Jump to the end of the file by typing Shift + G
Enter editing mode by typing the letter i.
Paste this code into the configuration file:
<Directory />
Options FollowSymLinks
AllowOverride All
Include /etc/httpd/conf/deny.conf
</Directory>
After making this change restart Apache.
One more thing.
I have noticed that the majority of malicious attempts on a server generally tend to target Wordpress. I can give some theories as to why, but my opinion is my own and might simply piss some people off. Lets just say that Wordpress is fun to build with...But it would NOT be my first choice.
Anyways these are directives that (if you are not using Wordpress are great) And if you are using Wordpress they can still be very useful (based on what you are using).
1. wp-login.php
I would suggest using a plugin that hides wp-admin. My favorite is WP-Hidelogin.
With this installed on a Wordpress website and the following directive bots will just find a 403 error taking up a tiny little bit of memory.
<Files ~ "wp-login.php$">
Order allow,deny
Deny from all
</Files>
2. xmlrpc.php
This article from Wordfence suggests that xmlrpc.php isn't a threat and is useful to Wordpress. But,...if I'm not using it, then I block it. And if I'm not using Wordpress then I definitely block it. I would rather a 403 than a 404.
<Files ~ "xmlrpc.php$">
Order allow,deny
Deny from all
</Files>
3. wp-comments-post.php
I rarely see any hits to this anymore. It used to get hit all the time. But, I rarely have customers who even want comments turned on. So I just tend to block it.
<Files ~ "wp-comments-post.php$">
Order allow,deny
Deny from all
</Files>
readme.html
I have seen Wordfence modify the name of this file with a hash and one time when it did it broke my installation when I upgraded, so I set the settings to NOT change this file and I simply block it this way.
The logic of renaming this file has been questioned by some
<Files ~ "readme.html$"> Order allow,deny Deny from all </Files>
wp-config.php
I see hits to this file on occasion.
<Files ~ "wp-config.php$">
Order allow,deny
Deny from all
</Files>
admin-ajax.php
As a plugin developer I use this file a lot...I see hits to this file as often I see hits on wp-login.php. The admin-ajax.php is well designed and a well designed plugin following the rules should be secure. If the attacker doesn't know what POST data to send, then this file will return a 0 with a very small footprint. If the attacker knows what plugins are on the website and knows what to pass it however, this file can make a large footprint, if the the plugin isn't designed to handle these situations.
Example:
I wrote a plugin that needed to search a set of data based on a number of parameters. The data to be returned was a relatively small amount but the previous developer hadn't really given any thought (or had any understanding) of how to store the data. A vague query or one across a large date might really begin to slow things down. Aware of this I wrote code to handle these situations, but that doesn't mean that other developers will take that into account when building a plugin.
I experimented with various server tricks to try to kill admin-ajax.php if the referrer were from an external source without much success:
<Files ~ "admin-ajax.php$'>
order deny,allow
deny from all
allow from xxx.xxx.xxx.xxx # the server IP ( unfortunately this IP will still be the user who made the request...close but no cigar )
</Files>
# this looks promising as it checks that the domain is the same as the referrer, but it just never seemed to work, despite several sources stating that it should work
<Files ~ "admin-ajax.php$">
RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_REFERER} .
RewriteCond %{HTTP_REFERER} !www.domain.tld[NC]
RewriteRule .? - [F]
</Files>
Of course, if you are not using Wordpress then this might be a good idea, just to avoid yet another 404 hit.
<Files ~ "admin-ajax.php$'>
order deny,allow
deny from all
</Files>
- Log in to post comments