One classic topic for this kind of discussion, which came up recently at work, is the use of exceptions for error handling. Every modern programming language offers an exception mechanism for this purpose, and presumably it is there to be used. However, ever since they were first introduced, there has been a large and vocal subset of the community arguing that exceptions do more harm than good and you’ll be writing better code if you just use good old return values to report whether a method succeeded.

One representative example comes from Joel Spolsky, one of my favorite authors. Another oft-quoted article making the same arguments is found in the “Frequently Questioned Answers” by Yossi Kreinin. They both make the same basic points: exceptions do not reduce complexity but merely hide it, and when complexity is hidden people tend to forget about it.

These arguments have merit, but I still feel that (when properly used) exception handling delivers enough value to be worth the cost. So I am going to be arguing for the status quo here, for a change. Executive summary: the dangers of exceptions are real, but code readability trumps almost everything.

Let’s say that we have a method of, say, 15 lines of code. This method is straightforward and easy to read and to modify. I can understand at a glance what it does; it forms a clear picture in my mind which I can easily reason about. There is just one problem with it: We haven’t added any error handling yet. For which we have basically two options: use return values, or use exceptions.

If we use return values, our nice clean 15-line method will suddenly blow up to easily five times as large, with lots of conditional statements and lots of ugly, repetitive boilerplate code. Such code is annoying to write and to read, easy to make mistakes in, and very tempting to get sloppy with. Suddenly, I can no longer hold a mental picture in my mind of what the method actually does. I’d have to go through the code line-by-line and trace all the possible code paths. Even then, I am quite likely to miss some subtlety. And since the method is now too large to fit on a screen, I should refactor it into multiple smaller methods, which requires adding even more errorhandling boilerplate.

On the other hand, if I use exceptions instead, most of my methods will become only a little larger, and all of the errorhandling code will be nicely contained in one place where I can reason about it separately. The rest of the code stays nice and clean and easy to understand at a single glance.

“Aha!” say the exception skeptics (exceptics?). “But that reduced complexity is only illusionary! If every line of code in your program might thrown an exception, you still have an enormous amount of possible code paths to worry about — they’re just not explicitly visible in your source code anymore, which is even worse!

Well yeah, that’s true. I still have to worry about those exceptions even if I can’t see them. But the nice thing is, I can think about them, because we are now back in the scenario where I have just 15 lines of code which I can easily turn into a mental picture which I can reason about. I can look at that code and ask myself: “What happens if an exception is thrown on this line?” “Will the resource I allocate on line 4 be properly cleaned up in all possible scenarios?” “If this function fails because of an invalid input file, how will we recover from that?” And the big difference is that I can reason about these questions at a much higher level of abstraction.

It’s like the difference between an amateur chess player and an expert. When a novice player thinks about his next move, he works his way through the possible options as a computer would: “I could do this, but then my opponent will do either that or that. Or I could do this, but then she will do …” This is not a good way to play chess, since there are a lot of different combinations and the human brain is really bad at systematically working its way through such a huge decision tree.

An expert chess player, on the other hand, thinks at the level of strategy and tactics, rather than individual moves. “I am on the offensive, but my opponent has stronger control over the center field. It looks like I could attack over the left flank and win a few pawns, but I should probably do something about that exposed bishop first..” Only after analyzing the board at this high level, will the expert identify a small number of specific opportunities and threats to be investigated at the level of individual moves and counter-moves.

From a given starting position, the number of moves and counter-moves available to the expert is the same as to the novice. But by thinking about the game at a high level of abstraction first, the expert can dismiss the vast majority of possibilities at a glance, without conscious thought, and focus on the half-dozen interesting cases which need to be investigated in detail. It could happen that the expert overlooks some subtle trick which the plodding novice would have discovered, but the smart money will bet against it.

(As an aside, research has shown that experts are much better than novices at memorizing the position of the pieces of a chess board, but only when dealing with a position from a real game. When the pieces are placed on the board randomly, the experts are not much better at memorizing them than people with no knowledge of chess at all.)

I am hardly a chess master, but I’m fairly good at reading code. When I try to read a large block of ugly code where every “real” function call is followed by a dozen lines of boring error handling code, I feel like the novice chess player in the story above, missing the forest for the trees. When reading a nice clean piece of code where the error handling is provided through exceptions, I feel like the expert player, spending most of my time thinking at a higher level of abstraction. The complexity of the situation is the same either way, but now I have a mental picture of the situation which allows me to visualize it from different angles, dismiss all of the boring straightforward cases at a glance, and quickly zoom in on the ones which require my detailed attention.

At this point, assuming you’re still reading, you are probably either nodding along with me or you’re getting red in the face and starting to sputter in protest. If you’re nodding along, that likely means you belong to the camp which views programming as an essentially artistic activity, where intuition and the undefinable concept of ‘elegance’ play a big role. You believe that there is an unprovable, but very real, connection between the subjective beauty of a piece of code and the chance that it is correct.

On the other hand, if you’re sputtering and getting angry, you probably belong to the camp which feels that software development is an engineering discipline in which feelings and emotions have no place. Your standpoint is that if a piece of code has 200 different possible execution paths, then somebody should bloody well go over all of those 200 paths and verify that each of them is correct — anything else is sloppy and amateurish and would never be allowed in your team.

And to be fair, I have a lot of sympathy for that standpoint, as I like to think of myself as a left-brain type of person as well. In fact, if I were in charge of developing the control software for a nuclear reactor, I would change sides: I would disallow exceptions, make sure that all conditionals are written out explicitly, and then use a variety of formal proof techniques to guarantee the correctness of every single path through the code. My team’s productivity may be less than one line of code per day, but there will be no mushroom clouds on my watch!

At this point, the people in the second group puff up their chest and say, well maybe you think that your project is not important enough to be worth using proper software engineering techniques on, but I take pride in my work and all my code is written to the same level of quality as a nuclear reactor! Well, maybe. Good for you. In the meantime, I have a deadline to meet.

So, dear reader, to determine which of these two camps you believe yourself to be in: when was the last time your used mathematically rigorous formal proof techniques to validate your program’s correctness? Yeah, me neither. Well, since we have just found out which camp you are not in, you had better make sure that your code is readable and elegant enough to be able to reason about it intuitively. It’s the only chance you have of delivering reasonably bug-free code at an acceptable level of productivity.

(Oh, and adding some automated testing won’t hurt either way. As Donald Knuth once famously wrote: “Beware of errors in the above code; I have only proved it correct, not tested it.”)

So I was visiting my parents this week-end, and my Dad asked me to help him with a little macro job on an Excel spreadsheet. It sounded simple enough. However, I had forgotten just how astonishingly horrible Visual Basic For Applications, the sorry excuse for a programming language built into Excel (and the other Office applications), can be.

As far as I remember, the last time I did anything with VBA was probably somewhere in the late nineties. Even by the standards of back then, VBA is a really shitty programming language. By the standards of 2009, it’s spectacularly bad. The only explanation I can think of is that somewhere high up in Microsoft Strategic Command, somebody decided to spend a lot of effort on making it as useless and infuriating as possible, while still keeping it just barely functional enough to be able to do the things you want to do with it, if you’re willing to go through a lot of pain. God only knows why they made that decision, but surely a language as bad as this cannot be created by accident.
What surprised me most is that it was still exactly as bad as I remembered. We’re talking about Office 2007 here, the all-new singing and dancing one with the cool magic fluffy ribbons and shit. Surely they could have made some minor improvements to the scripting language while they were hard at work at hiding the ‘File’ menu behind a big round decorative window-corner ornament? In fact, I would have expected them to have simply integrated the .NET framework into Office by now, so that you could write your Excel macros into F# if you felt like it?

Apparently, that is not the case. The language is still filled with lots of little inconvenience such as the fact that, where every other programming language on Earth lets you return a value from a function by doing something like

def thisFunctionReturnsThree()
    return 3
end

VBA, for reasons which I’m sure seemed like a good idea to somebody at some point, expect you to do it like this:

Function ThisFunctionReturnsThree()
    ThisFunctionReturnsThree = 3
    Exit Function
End Function

But minor inconveniences like that are just a scratch on a gaping shotgun wound. Where VBA really starts driving the red-hot splinters of aggravation under your fingernails, is when you try to use the Collection object. The VBA Collection is some kind of schizophrenic bastard datatype with a horrible identity crisis, which can never quite decide whether it wants to be a fancy-schmancy associative array (a.k.a. hash or map, for those of my readers who are used to non-braindead programming environments) or just an ordinary variable-sized array. Anyway, back in 1997 or whatever, the Collection type supported a grand total of four methods:

Add
Remove
Count
Item

And here in 2009, it still supports the same four methods! Note that there is no way to enumerate all the keys in the collection (when using it as a hash), nor is there a way to ask the collection object if it contains a given value (when using it as a resizeable array). Doing myCollection.Item(”foo”) will throw an error if foo is not present in the collection. So there is no way, at least without grossly abusing the exception-catching mechanism as a flow control technique, which is evil, to access a value of which you are not certain whether it exists.

Nor does VBA seem to natively offer any more powerful container types. The standard trick, apparently, is to bring in the Scripting.Dictionary from the Windows Scripting Library, which is not a standard part of either VBA or MS Office, but which can kinda-sorta generally be expected to probably be present on most Windows systems which have been installed and occasionally updated during the past five years. (The ones which have not been regularly updated are, of course, so bogged down with malware by now that there won’t be any CPU power left to run Excel anyway, so no need to worry about them.) Scripting.Dictionary is hardly the equal of, say, the Hash class from Ruby’s standard library, but at least it has a contains method and it allows you to enumerate its keys and values. Welcome to the 1990’s, Microsoft!

By the way, have I already mentioned that VBA, again unlike every other programming language invented after the death of Blaise Pascal, does not do lazy evaluation on Boolean expressions? So if you want to write something along the lines of, say

If myDict.Contains(”foo”) And myDict(”foo”) = “bar” Then
    ’ Do something cool
Else
    ’ Do something not-quite-as-cool
End If

then your code will break because, after having cleverly determined that myDictionary does not contain an item named “foo”, VBA will then shrewdly try to access it anyway, and promply die.

Oh, and neither Collection nor Dictionary has a built-in ability to sort its contents. If you search for e.g. “vba sort container” you will find a lot of people who have helpfully written their own functions to perform this rather basic task.

We did eventually get my Dad’s spreadsheet to do what we wanted it to. I’d estimate that I spent about 10% of that time on actually writing the functionality we were after, and the other 90% of the time swearing at the absurd, arbitrary limitations of VBA and trying to come up with stupid workarounds for things which should be absolutely trivial, and which would be absolutely trivial in any other language, and which anybody who wants to write a non-trivial piece of code is going to need so why isn’t it present by default in this supposedly mature product? Gah.

Now, it is very possible, even quite likely, that many of the things I ran up against were simply a result of my unfamiliarity with the programming environment. I am more than happy to admit that I am not an Excel guru or a VBA guru. Probably then, there are much better solutions to each of the problems mentioned above. Please feel free to teach me about them and laugh at my ignorance!

However, I do know how to use Google, and what I found were not solutions but just a lot of other people having the same problems and complaining about how incredibly limited the language is. So, Microsoft, could we perhaps ask you to take perhaps one-half of a developer off the task of making Office 2010’s ribbons even sparklier, and bringing the facilities of VBA up to the level of an average scripting language from ten years ago? Thanks in advance.

One of the open challenges is to work out the prime factors of a given number. To make things a little more difficult, the output must be printed in a specific format:

7000: 2^3 5^3 7
123456789: 3^2 3607 3803

Here is one of my attempts at a Ruby implementation, which is 97 bytes after removing the final trailing newline:

print”#{n=gets.to_i}: ”
(2..n).each{|x|i=0;n,i=n/x,i+1while n%x<1
print x,i<2?’ ‘:”^#{i} “if i>0}

Then we switched to Perl, and together Dirk-Jan and I managed to come up with a version which is just 82 bytes after removing all the newlines (just pipe it through “perl -pe chomp“).

print$n=pop,’:';for(2..$n){
$i=0;$n/=$_,$i++until$n%$_;
print” $_”.”^$i”x($i>1)if$i}

(Be aware that the Ruby version takes its input from stdin, while the Perl program looks at its first command-line argument.)

That’s pretty compact, right? Algorithmically, the code is actually fairly straightforward; probably the most evil trick we use is to employ the ‘x‘ operator, which does string concatenation a given number of times, in combination with the fact that any Boolean expression can be treated as a numeric value of either 0 or 1, as in C (but not in Ruby).

Unfortunately, both of the versions given above are horrendously slow. They have a running time of O(n), while any self-respecting factorization algorithm should be at most proportional with the largest prime factor of n. It took my Pentium-4 system almost 45 minutes to calculate the factors of 2,000,000,000, which a reasonably intelligent high school student could probably have done in half a minute or so. Here is a version which can do it in a fraction of a second, but at the cost of being a whole 10 bytes larger:

print$n=pop,’:';
for($x=2;$n>1;$x++){
$i=0;$n/=$x,$i++until$n%$x;
print” $x”.”^$i”x($i>1)if$i}

So, how are we doing in the contest? Well, we’re not even competing. The leader is currently at an astonishing 76 bytes! I have no idea how they do that. The winning programs are all in Perl, which tends to do very well in this type of contest despite the fact that all variable names are at least two characters. The best Ruby program is 82 bytes, with Python coming in at 100 and PHP at 122.

Another task in Code Golf is to write a program which can solve the famous Towers of Hanoi puzzle, given a random starting position with up to nine disks. The input consists of three lines of text, each giving the sequence of disks for one of the three pegs. For example:

975
864
321

The goal is to print the series of moves needed to get all disks together on peg C, the third one, following the usual rule that at no time may a larger disk be on top of (to the right of) a smaller one. Here is a very simple example run with only three disks:

$ cat simple.txt
31

2

$ ruby hanoi.rb simple.txt
2 to B
1 to B
3 to C
1 to A
2 to C
1 to C

We haven’t spent as much time yet on this one as on the prime factorization problem. Here is my best effort so far:

T,D=[],[:A,:B,:C]
D.each{|t|gets.chomp.each_byte{|x|T[x-48]=t}}
def m s,t
if s>0
m s-1,(D-[t,T[s]])[0]
puts”#{s} to #{t}”
m s-1,T[s]=t
end
end
m T.size-1,:C

This is an embarassing 157 bytes. In the actual contest participants, Ruby is leading the pack this time, with 104 bytes, while the best Perl entry so far is a whole six bytes larger. Go Ruby!

The Code Golf contest is open to participants using Ruby, Perl, Python and PHP. Nonetheless, Jeroen, Jeroen and Raimond, I look forward to you trying to beat the above programs in Java or C#.. 

In a little over a month, I will be

years old.

What’s that, you didn’t get it? Here, I’ll repeat it for you in terms you may understand more easily:

At work, it has become a bit of a tradition that when people announce their birthday, they do so in an at least somewhat obfuscated format. Hexadecimal, binary and more obscure number formats are always popular, of course, as are silly descriptions of the form “my age is the ninth distinct biprime“. But last year I decided to take it to the next level, and write a little generator in Ruby for expressions such as the ones you see above. As you can probably guess, the expressions are generated using TeX.

You can play with it for yourself, if you want to, and also download the latest version of the code. But please be gentle with my server, as you can probably guess it’s a rather heavy application and I’m running this site on a little home PC..

A question which I like to use when interviewing C++ programmers: what is the range of a 32-bit integer?

I don’t use this question very often anymore, or at least I don’t let it influence my decision very much (which is why I don’t mind spilling the beans here), because the correlation with other technical skills turns out to be not as strong as I thought it would be. Still, there are some interesting patterns in the kind of people who know the answer versus those who do not.

Among the people who do not know the answer, some of them react quite affronted that they would even be expected to. What is the point, they will ask, in having memorized some little piece of trivia which they could Google up in a few seconds? Isn’t “knowing where to find it” a much more useful skill? Ask me about architecture! Ask me about design patterns and data structures! All of these objections have some validity, and indeed we will certainly ask about those other things during the interview. Still, I believe that it is perfectly reasonable to expect a good developer to know the answer to the above question by heart.

There are two reasons why I believe that. The first is simple: if you don’t know the range of the data types you’re working with, how can you write reliable code? “I will look it up when I need it” is a common excuse, but that doesn’t fly with me. It would be a perfectly valid answer if I had asked you to recite from memory the arguments to some obscure library function, but we’re talking about the integers here! The most commonly used datatype in any programming language. Are you seriously telling me that, every single time you write

for (int i = 0; i < n; ++i)

you look up in the compiler documentation whether the range of i is large enough that you can safely assume n to fall within it? And then you promptly forget about it so that you need to look it up again the next time? So that, when you arrive in my interviewing room with a CV that says you have five years of C++ experience, you have looked up that particular bit of trivia many thousands of times by now, and you still don’t remember it? What’s more likely is that you have never looked it up, which means that you are basically, as Joel Spolsky would call it, programming by superstition.

The other reason why you need to know this stuff is that you will be a more effective programmer if you are able to think of what you are doing in terms of the underlying processor instructions — a skill which is useful even when working in very high-level languages, but which is absolutely necessary when working in C or C++.

For example, even a non-programmer may be aware that the reason why the world is rapidly moving to 64-bit processors and operating systems right now, is because the 32-bit ones are unable to use more than 4GB of memory. But as a professional programmer, I would expect you to know why that is. The answer is simple: it’s because 32-bit machines use 32-bit memory addresses, and with those you can only access 2³² = 4,294,967,296 different bytes. If you don’t know that, you are lacking a fairly basic piece of understanding of how computers work.

Now, of course I don’t expect people to know the exact number 4,294,967,296 by heart, but “a little over four billion” (or two billion for a signed integer) would not be too much to ask. If you cannot tell whether it is more or less than a million, as was the case with several ’senior’ candidates I had the pleasure to interview, you’d better hit the ball out of the ballpark on every other part of the interview. Which they did not. (Please note that I’m using the American usage of ‘billion’ to mean a thousand million.)

Another point is that C++ has largely fallen out of favour nowadays for general application programming. So when it is still used, it is mostly to support performance-critical algorithms or very high-volume data processing. In that kind of code, having a good mental picture of the data structures you’re using is really crucial. If you want to have millions of instances of a given object in memory simultaneously, shaving a few bytes off each instance can really matter — which means that you need to know when you can safely use a char or a short instead of just blithely using ints for everything.

So, here are the facts which I expect every C++ programmer, as well as the better C# and Java ones, to know more readily than their mother’s birthday:

• A byte is 8 bits, and can hold 256 different values. Examples: the number of different characters in a non-Unicode text file, the number of different shades of red, green and blue in a true-colour image. Also the maximum filename length on ext3 and NTFS, and an unfortunate limit on path length in many programs.
• A ’short’, on most C/C++ compilers, is 16 bits, which means it ranges from -32,768 to 32,767 when signed, or from 0 to 65,535 when unsigned. A decade or two ago, it was quite common to have the default size of int be 16 bits on PC-based C compilers, and you may still encounter this in the embedded world. On many systems today the wchar type is 16 bits, so that it supports all characters in the UCS-2 subset of Unicode, which contains all of the characters you actually care about. Beware of the difference between UCS-2 and UTF-16, though. Before true-colour displays became the norm, there were video cards which supported 16-bit colour; usually this was actually 15-bits, with 5 bits (32 different shades) each for red, green and blue, although sometimes the remaining bit was used to allow 64 different shades for green, which is the colour which the human eye can perceive most accurately. 16 bits is also the default integer size on some database systems, which can give you some nasty surprises when your e-commerce system has been in production for a few months and a customer submits the 65536th order.
• 24 bits (three bytes) gives you about 16.8 million (16,777,216) different combinations. Among other things, this is the number of different colours on a true-colour display. In November 2006, Slashdot had to do an emergency update of their database, since they used the MySQL ‘mediumint’ type for comment IDs, so the system broke when the 16777216th comment was posted.
• 32-bits is the default integer size for almost any modern programming language — yes, even on 64-bit platforms, which will break any code which assumes that a pointer can be cast into an integer and back. As stated above, a signed integer ranges from minus to plus 2.1 billion, while the unsigned range goes up to almost 4.3 billion. This is a fairly generous range for most problem domains, but sooner or later it will bite you if you’re not aware of it. Among many other things, this is why many programs have trouble with data files larger than 2GB. Also the absolute upper limit on the number of unique IP4 addresses (the actual limit is much lower because of the way the address space is organized).
• A float value uses 24 bits for the mantissa, which translates into 7 decimal digits of precision, which is less than most programmers intuitively expect. A double uses 53 bits, which is plenty for most purposes. Anyway, the most important thing you need to know about working with floating-point numbers is that it is fraught with gotchas and special cases which will trip you up sooner or later. And never, ever use them for monetary values.

I would also expect a serious programmer to know the powers-of-two table at least up to 2¹⁶ = 65,536. You simply run into those numbers so often that you can’t help memorizing them after a while, and it’s surprising how often this knowledge can help you in debugging various kinds of overflow issues.

So what about larger integer sizes? Well, unless you work in cryptography or some other specialized field (in which case I really hope that you didn’t learn anything new from this article), you can safely assume that 64 bits is enough for anything you will ever want to do. But here is a handy little rule-of-thumb I use to convert between bits and decimal digits: 10 bits is 1,024 and 3 digits is 1,000, so to go from number-of-bits to number-of-digits you just divide by 10 and then multiply by 3. (Or for an even rougher approximation, just divide by 3.)

For instance, an unsigned 64-bit address represents roughly 3 * 6.4 = 19.2 decimal digits, so it can safely be used to store numbers well over 1,000,000,000,000,000,000. Likewise, if you want to know how many bits you need to store a 12-digit number, you divide by 3 and multiply by 10, which tells you that you need approximately 40 bits.

As you can see, the ADSL router and anything connected to it through the WiFi is considered untrusted: the real access point to my internal network is the Linux machine.

Among other things, the Speedtouch has the ability to support Voice-Over-IP by attaching an analog phone. Unfortunately, this functionality does not work in combination with the “assign public IP to a machine on the local network” setting. Which is a pity, because behind the router is my Linux server, running a web- and mailserver among other things, and I really want that server to have my public IP address. Partly because having the server NATted could cause problems with mail, in particular, in the sense that when I send out mail to another server, some suspicious spamblocker software may take offense if the address reported in the headers of my outgoing mail does not match my actual IP. But mostly because having a web/mail/FTP/whatever server hidden behind a NAT, just feels wrong.

So, I have two conflicting desires here: I want to use VOIP over a phone connected to the Speedtouch, and I want my server to believe that it is listening directly on my public IP. A little googling confirms that a lot of other people have this same problem, but if anybody has found a solution already, I didn’t see it.

There does exist a way to expose the server to the outside world without breaking VOIP: use the “game and application sharing” feature to forward all TCP and UDP ports except port 5060 (which is the SIP port used by the VOIP service) to the server. But then we are using NAT again, which is not what we want. What I want is: port 5060 is handled by the Speedtouch, all other packets are sent straight to my Linux server, which should receive them on the public IP address of my Internet account. Unfortunately, it seems that there isn’t any way to configure the Speedtouch like that.

How I eventually solved this problem is by adding an iptables rule on my server, which uses DNAT to translate the source address of each packet back into my public IP before the server sees it.

Like this:

• A packet comes in from the outside world on my public IP address (82.95.250.5).
• The router sees it, and if the packet is not aimed at the VOIP port (5060), sends it on to my server, which has a local IP of 10.0.0.1 (assigned to it by the router through DHCP).
• An iptables rule on my server intercepts the packet and changes the source address back to 82.95.250.5. This way, all services running on my server can pretend that they are connected directly to the Net, without any special configuration needed.
• As my server sends a response to the packet, the destination address is changed back to the router’s address (10.0.0.138).
• The router performs a second layer of NAT, translating the destination address of the response packet back into 82.95.250.5 again, before sending it along to its final destination.

So every packet exchanged between my server and the rest of the world, is NATted twice: once by the router, once by the server. Not a particularly elegant solution, but it will have to do until somebody comes along with a better way to bend the Speedtouch to his will (or until I buy a better ADSL router).

Here’s the magic iptables statement:

iptables -A PREROUTING -t nat -i $EXTERNAL -d $FAKEPUBLICIP -j DNAT –to-destination $PUBLICIP

In my case, $EXTERNAL would be eth1 and $FAKEPUBLICIP is 10.0.0.1.

As I said, it’s not a particularly elegant solution. One annoying consequence is the fact that if you try to browse to mwolf.net from the WiFi network, you’ll get an error because the router will get confused about where to send packets for 82.95.250.5. This could be solved with a bit of DNS magic.

But maybe somebody has already found a nice, clean way to configure the Thomson router to do what I want, and I just missed it. Any ideas would be appreciated.

You can download his firewall script here. You can contact the author at the address meetscott at the domain netscape.net.

As long as you have your SSH server configured properly, the most important thing being to only allow SSH access to accounts which actually need it, this is more an annoyance than a problem. Nonetheless, it is an annoyance, if only because of all the crap in your logfiles.

There are many ways to detect and block such attacks. sshdfilter works well, and a good detailed overview of the various options can be found here. One that particularly appealed to me, however, was a very simple netfilter-based technique consisting of only two lines of iptables code. It uses the recent netfilter extension, and the idea of using it to combat SSH attacks was apparently first conceived by Andrew Pollock.

Here’s my version:
iptables -A INPUT -p tcp –dport ssh -m state –state NEW \
-m recent –set –name SSH -j ACCEPT
iptables -A INPUT -p tcp –dport ssh -m recent –update \
–seconds 600 –hitcount 6 –rttl –name SSH -j DROP

This will blacklist a given IP address as soon as six TCP connections to the SSH port are detected from that address within ten minutes, and then the blacklist will remain in effect until no further connection attempts have been made from that address during the past ten minutes. This is the most basic version: it does not support whitelisting and there’s no logging, but it works very well at what it does. As I can see from my logs, all crack attempts are indeed broken off after the first six attempts, and they generally don’t come back.

This is just one of the many reasons why Netfilter/IPTables is such a great tool. Sure, it’s a bit intimidating at first, but once you understand the basics, you can fine-tune your server’s routing and firewalling configuration to an almost limitless degree.

What I’m still looking for, however, is a way to control the ‘detection period’ and the ‘blacklist period’ separately. For example, if six SSH connections are made from a given address within four minutes, that address is blacklisted for the next twelve hours. Anybody know of a good way to do that?