Alleviating Some of the Crap Thats Slowing Down Your Server at the Source
Several months ago I started receiving the occasional alert email in the middle of the night warning that my server was hitting a threshold. Reviewing the server logs I found the usual suspects. Hits to Wordpress; admin-ajax, wp-login.php, xmlrpc.php... But also, there were a ton of hits from a web crawler that was hitting the same content over and over and over. There are plugins available to cache websites and throttle repeat content, but a hit to the CMS is still a hit and will require PHP, several image requests, file requests and more than a few hits to the database.