Last part of the DDOS story:

how well did the evil plan work and what could we do about it?

Of course, we didn’t have to turn off our spam filters! 🙂 Nevertheless, the attack has caused some excitement.

A strange picture emerged on the SecuMail^® servers:
Hardly any mail throughput, an ever-increasing mail jam but almost no load.
Only after some pondering and a very useful tool called TCP dump did we manage to determine the cause:
The machines were bored while waiting in vain for DNS responses from the Russian fake domains.
After a very short time, all SecuMail^® processes were busy with “doing nothing” in this way, so that hardly any emails could be delivered.

Can a DNS query stop the entire operation?
No, it’s not that simple. But small animals also make a mess and even heaps of it!
So what exactly happened:

When trying to resolve all link URLs contained in the mail body via DNS, the SecuMail^® processes
never received an answer and gave up after about 3 seconds. However, if not even error messages from the DNS server come back, it is classified as “unreachable” and then luck is tried at the next DNS server entered in the system, which of course could not have any chance of success in this case either.
As a rule, three DNS servers are registered on our servers.

~~~

http://keineantwort.russischedomain.ru http://auchkeineantwort.russischedomain.ru
http://nieantwort.andere-russischedomain.ru
http://stumm.andere-russischedomain.ru
http://sagkeinwort.andere-russischedomain.ru
~~~

According to Adam Riese, this would be a total
of (on average) five link URLs in the text (see above), for each of which three name servers wait for DNS responses for at least three seconds.
That makes a whole 45 seconds waiting time per mail (!). This would explain the slow mail throughput under low load.

How did we fight this adversity?
As a first immediate measure, we have reduced the number of entered DNS servers to one and have thus been able to significantly reduce the waiting time per mail
. After that, it became more difficult to find further measures. Blacklists or manual domain entries on our DNS server were ruled out because of the large number of entries that would have been necessary. Deactivating the “Url Resolver” completely was out of the question for us, as it is hardly dispensable as an important part of spam detection.

The solution is called “negative caching”.
The results of the DNS queries are to be simply stored temporarily, so that only exactly once per link URL has to be queried. Unfortunately, it turned out that our name servers (bind) were not able to cache requests aborted by timeout despite activated negative caching. The chaching only works for regularly received DNS responses, but not if the DNS server does not respond.
However, there is still an inconspicuous service called nscd (Name Service Caching Daemon) on Linux systems, which has hardly ever attracted our attention so far. And this is also able to remember for a while whether a server responds at all. Because the number of URLs linked in the text cannot be infinite, they have almost completely found their way into the NSCD caches after a short time, thus enabling a scanning process at normal speed.

A little later, all e-mails in the queue were processed. Apart from the delayed delivery, our customers have not noticed any of this.

Conclusion:
The fight against spam remains a tedious “cat and mouse game”.
However, with a flexible and open filter system in your own hand, which you can adapt and expand in time, you are well equipped for battle.

Best regards,
Hannes Wilhelm