How To Temporary Ban Bad Bots with Fail2Ban + CSF

Those of you who have run your own server in the past know the importance of banning bad bots as they can overwhelm your server with too many hits. You can always ban them permanently with your .htaccess or NGINX configuration file. For certain bots, you may want to ban them temporarily after a certain number of hits. Fail2Ban is a neat tool that can combine with your firewall to take care of it. For starters, you will want to find out which bots are hitting your site the most. This command will help with that:

The next step is to install Fail2Ban and CSF and configure them to work together. You are going to have to set up your filter and action settings. For my jail.conf, I defined the following restrictions. It looks for 15 attempts and bans them for 18000 seconds. You can play around with these depending how heavily your site is being visited.

You will also have define your csf.conf action in action.d folder. For some reason, my auto unban action didn’t work no matter what I did, so I decided to pass temporary bans to CSF.

Finally, in my filter.d folder, I changed my gptbot.conf to include the bots that I want to monitor for. In my particular case I am targeting GPTBot, Amazonbot, and a few others.

By using this approach, you can limit how often your site gets visited by certain bots without permanently ban them. After doing this, you may be able to drop your server load significantly. My restrictions are quite moderate but with a more aggressive approach, you can further keep these bots in check.