IPTables against SSH dictionary attacks
Like everybody who has a Linux server running an SSH daemon connected to the Internet, I regularly get attacked by people (well, botnets probably) trying to do a brute-force attack against the server. Such attempts can take many hours, during which they simply try many thousands of possible username/password combinations.
As long as you have your SSH server configured properly, the most important thing being to only allow SSH access to accounts which actually need it, this is more an annoyance than a problem. Nonetheless, it is an annoyance, if only because of all the crap in your logfiles.
There are many ways to detect and block such attacks. sshdfilter works well, and a good detailed overview of the various options can be found here. One that particularly appealed to me, however, was a very simple netfilter-based technique consisting of only two lines of iptables code. It uses the recent netfilter extension, and the idea of using it to combat SSH attacks was apparently first conceived by Andrew Pollock.
Here’s my version:
iptables -A INPUT -p tcp –dport ssh -m state –state NEW \
-m recent –set –name SSH -j ACCEPT
iptables -A INPUT -p tcp –dport ssh -m recent –update \
–seconds 600 –hitcount 6 –rttl –name SSH -j DROP
This will blacklist a given IP address as soon as six TCP connections to the SSH port are detected from that address within ten minutes, and then the blacklist will remain in effect until no further connection attempts have been made from that address during the past ten minutes. This is the most basic version: it does not support whitelisting and there’s no logging, but it works very well at what it does. As I can see from my logs, all crack attempts are indeed broken off after the first six attempts, and they generally don’t come back.
This is just one of the many reasons why Netfilter/IPTables is such a great tool. Sure, it’s a bit intimidating at first, but once you understand the basics, you can fine-tune your server’s routing and firewalling configuration to an almost limitless degree.
What I’m still looking for, however, is a way to control the ‘detection period’ and the ‘blacklist period’ separately. For example, if six SSH connections are made from a given address within four minutes, that address is blacklisted for the next twelve hours. Anybody know of a good way to do that?
February 27th, 2007 at 8:53 pm
Thanks for posting this. I’ve looked at several different ways of doing this very thing. I like this the best from the ones I have seen so far. For about the past year I’ve been addressing this with a blacklist of networks that are foreign in origin, and keeping track of the US ones only blocking the *really* bad ones originating in the US. I have a little over 3000 networks in my blacklist and it has definitely helped in cutting down the number of attempts. The drawback is the weekly maintenance of updating the list based on the past week’s attacks. Also, I was noticing a significant degradation in network performance on my system because there is *so* much information in the iptables based firewall on my system. Hopefully this will be the final solution I’m looking for.
A note for people: A direct copy from this web page does NOT work because of the way hyphens are handled in your web browser. For example, a copy-paste will produce -dport instead of –deport on the console. Same goes for –state, –seconds, etc.
Also, I did not find the option of using –dport ssh in the man pages on my system, but my firewall script ran fine none the less. I only saw references to –dport 22 for example or specifying a list of ports comma separated (up to 15 max). In the man pages it uses numbers only. So my question then becomes would I just specify ssh,ftp to also knock out the scripting attempts to my proftpd server?
February 28th, 2007 at 12:22 am
Thanks for your feedback! The mangling of dashes seems to be Wordpress trying to be too smart for its own good. I’ll look into that later, it’s way past bed-time now..
Yes, instead of numeric port numbers, you can use any of the aliases defined in /etc/services. So wherever you can use a comma-separated list of port numbers, using named services should work as well.
Don’t get too enthusiastic, though, or you might accidentally lock yourself out if you ever legitimately have a succession of short SSH/FTP sessions to your machine for whatever reason.. If you follow the link to Andrew Pollock’s site in my post, there’s some information on whitelisting which may be useful for you.
February 28th, 2007 at 3:07 am
Okay. I could not get the comma separated list to work by using mport or multport with –dports. I ended up specifying port 21 and 22 in my case ftp and ssh like this: ftp:ssh
The order is important because the port range must start with the minimum first and then end with the maximum last. Here is your script again with the slight modification:
iptables -A INPUT -p tcp –dport ftp:ssh -m state –state NEW \
-m recent –set –name ATTACKS -j ACCEPT
iptables -A INPUT -p tcp –dport ftp:ssh -m recent –update \
–seconds 600 –hitcount 6 –rttl –name ATTACKS -j DROP
I figured I should change the name from SSH to ATTACKS since I’m using it for ftp attacks as well-)
I haven’t been attacked yet but I’m actually waiting anxiously for an attack to see how this is going to work
February 28th, 2007 at 3:24 am
Sheesh!!! I posted just after 7pm Mountain Time and I saw the 3am timestamp on the post! Then I did a look up and you’re in the Netherlands? That’s amazing! I’m in Scottsdale, Arizona in the US. The world is getting smaller
It’s 8 hours later there, so I guess you’re sleeping. I’ll post back when I’ve been sufficiently attacked so you know how wells it’s working. Thanks again!
February 28th, 2007 at 5:33 am
Something went wrong. It’s not blocking the ftp attempts. That’s too bad. None tried on ssh yet. I’ll let you know whether that’s successful too.
Do you see anything wrong with my code above? I’m going to have to study this some more to understand what it’s trying to do exactly. If it works for the ssh attempts and not the ftp attempts, then I will have more information as to what I should focus on next. I’m almost thinking I need to block port 20 as well for ftp as I think that’s some control stuff, but I can’t remember for sure. It’s been to long since I’ve looked at it.
Here’s an entry from my proftpd.log file that just happened this evening:
Feb 27 20:15:50 webhost1 proftpd[11062] localhost (ns.1risekabu.com[210.166.217.45]): no such user ‘Administrator’
Feb 27 20:15:50 webhost1 proftpd[11062] localhost (ns.1risekabu.com[210.166.217.45]): USER Administrator: no such user found from ns.1risekabu.com [210.166.217.45] to 192.168.0.2:21
Feb 27 20:15:50 webhost1 proftpd[11062] localhost (ns.1risekabu.com[210.166.217.45]): mod_delay/0.5: delaying for 56 usecs
Feb 27 20:15:51 webhost1 proftpd[11062] localhost (ns.1risekabu.com[210.166.217.45]): mod_delay/0.5: delaying for 38 usecs
This is one entry, but the frequency of all of them was enough that the iptables code should have knocked it down. I ended up with 185 entries in the “secure” log file from this one all spaced apart by like 2 or 3 seconds.
I’ll let you know what I come up with on the ssh side in a day or 2.
February 28th, 2007 at 3:50 pm
Looks like it’s not working for ssh either. I will have to investigate more later to try and figure out why. I was really hoping this would be it. I’m not going to give up on this though. I have to go to work now but I’ll look into it more later this evening if I can.
February 28th, 2007 at 9:03 pm
Looks like it should work. One think to check for, though: are these your only iptables rules, or are you integrating it into a larger firewall script? In the latter case, make sure that you put those two rules *before* any rule which may do an ACCEPT on the incoming packets!
Another thing you could try is to have two lines for the ssh port and two lines for the ftp port, instead of trying to combine them; just to see if that makes a difference.
By the way, speaking about weird: your posts were accepted yesterday, and they were visible to anonymous users — and then this morning, apparently Spam Karma decided to flag them as possible spam after all! Looks like that needs some fine-tuning as well..
March 1st, 2007 at 8:46 am
Okay, your script worked for ftp and ssh just like I posted above. I had to take out all my other firewall rules and try it. Since I like blocking all but certain ports I specify, I’m going to have to investigate why iptables works this way. I’m certainly *not* an iptables expert, maybe not even a novice, but I thought you would like to know it was *my* fault.
I’ve always tried to learn the minimum to get iptables to do what I want it to. Using it effectively is like mastering another programming language. And then you only write one “program” with it, namely your firewall rules for your server.
I’m really beat, so I’m going to have to take this up later. I will post back my findings when I get that far. Thanks again. It was really beautiful watching that scripting attack die after six tries. I tailed the logs while it was happening and then ran the script to see it in action real time!
March 4th, 2007 at 12:56 am
Martin, you hit the nail on the head! I made the changes you suggested, that is moving your code to the beginning of the firewall script. It works for both ftp and ssh. I had to wait until I had an attack on both before I posted back. to be sure everything was working as intended. So this means that your rules are applied first so there is no way that either users from my whitelist or blacklist for that matter can violate your limit rules. I then have all ports closed and I reopen only the ones I specify. Lastly, I drop inbound startup requests (SYN drop). This has the effect of making port scans take *much* longer because they must operate in passive mode.
I would like to have an exempt whitelist though. But for now it’s not really needed and therefore I’m not going to spend the time to figure it out. I think it would be a matter of !192.168.0.0/24 or something added to your portion of the firewall script. You could loop through the whitelist in the shell script with the other networks similarly.
Let me know if you are interested in the rest of the script. I actually got most of it from Rob Flickenger in a book he wrote called “Linux Server Hacks” I think.
I also wrote a Perl script a couple of years ago for mining out ip addresses from the logs. This gives me a summary (with number of hits from each address) of all the ip addresses that have hit me based on the log files passed from the script. The script does *not* try to figure out multiline entries. It simply finds an entry and then increments that ip address as having occurred again in the log file. Let me know if your are interested in this too. It enables you to go through logs very quickly.
Thanks again for this little hack. I have over 500,000 entries from the last 4 weeks in my logs. This is ftp and ssh attacks. I hope in another 4 weeks that this number will be around 2000 or so thanks to your firewall addition. An added bonus: my system is *much* faster because my blacklist has been completely eliminated in both iptables and hosts.deny. Now I’m only going to need this stuff you wrote:-D
March 22nd, 2007 at 4:58 pm
Okay, I’ve added some functionality that others might be interested in. I did some more studying of iptables and came up with a method of including a whitelist along with this limiting behavior for the dictionary attacks. In addition, I have added the notion of listing other services. For example, right now I’m limiting ssh, ftp, and pop3. I recently got a pop3 attack that caused me to look at that too.
In summary:
- I have a whitelist that is *not* limited by the number of login attempts.
- I have this hack presented here that limits the number of attempts from *all* other people for the 10 minute period, same as presented above.
- I have a blacklist, which is no longer being used but it’s still nice that I can use it if I change my mind.
- I close all ports except the ones I specify to open.
- And finally, I drop SYN packets which are inbound start up requests.
- Another note is that all the adjustable parts listed above are either set in variables at the top of the script or they are included in external files (eg the whitelist and blacklist). Open ports are in an easily editable list and the limiting ports are in an easily editable list at the top of the firewall script. So in order to open up another port, just add it to the list. To make a port limited to use per time, just add it to the LIMITED list. Once something has been adjusted, the firewall script just gets re-run. The ports are treated as a group so 3 ssh attempts, 2 ftp attempts, and then 1 pop3 attempt will trigger the 10 minute blocking (unless you’re on the whitelist of course
For now, I think this is all the functionality I have ever wanted out of iptables or a any other firewall for that matter. I’m thinking I might want to eventually adjust it for variables on the number of attempts, the time frame for those attempts, and then the blocking time.
If you’re interested I’ll post the script. Otherwise, I’ll likely post it on a how to once I get my own server co-located. Thanks for all the help.
March 23rd, 2007 at 5:26 pm
Hi Scott,
Cool! I’m very interested to see your improvements. If you don’t have a server to publish it on, feel free to use mine. If you prefer, you can mail it to me at “martin” at this domain, and I’ll publish it for you as a separate article.
March 25th, 2007 at 4:30 am
I made the improvements I described earlier. I also added some descriptions to what’s happening. The other thing is the logging switch. I put that in, but I have it turned off. Since I’m not interested in logging when the block occurs, I did not put that ability in script, but I don’t think it would be too hard to add. I’m e-mailing the script to you now.
This is my personal e-mail address you are receiving this from. I don’t give this one out to businesses or the like, only people I know. Please respect that and use meetscott@netscape.net if you want to distribute some correspondence to someone else. Thanks.
March 25th, 2007 at 5:25 pm
Thanks!
I have made the script you sent me available for download:
http://mwolf.net/misc-files/rc.firewall
It’s linked to from a new article:
http://mwolf.net/archive/firewall-script-from-scott/
May 4th, 2007 at 1:09 am
The reason it didn’t work right most likely is because you added -j ACCEPT to the first command. just strip that off.
February 3rd, 2008 at 9:11 pm
[...] In response to my article about using the recent IPTables module to fight brute-force password attacks, based on an idea from Andrew Pollock, a reader worked out the idea into a complete firewall script, with configurable whitelisting, the ability to block multiple ports, and several other enhancements. Read his post for the details. [...]