Recently there has been a lot of discussion and reports about an increasing amount of NDR messages triggered by spam. Since the NDR problem become a major factor in the spam world, and I noticed that there are some confusion about it, I thought it’s important that I’ll give a short overview of the problem and how Commtouch approaches it. Commtouch has developed a mechanism to cope with this nuisance problem, but before I go and explain about the solution I think it’s essential to understand the problem and its complexity.
First, let’s define what NDR means. An NDR is a bounce message notifying that a message did not reach the intended recipient. Since there is no industry standard for these bounce messages, there are several names for this type of scenario. Here are a few that I know of:
- NDR – Non-delivery report
- Bounce Message
- DSN -(failed) Delivery Status Notification
- NDN -Non-Delivery Notification
For the purpose of this post I will use the term “Legitimate-NDR”, for NDR messages that were sent to the original sender of the message (e.g. if they had a typo in an email address), and “Spam-NDR”, for NDR that was triggered by a spam message, where usually people get them since spammers sent spam “on their behalf” and the unsuspecting user receives the NDR.
To be honest, one can claim that any NDR is legitimate, since a legitimate MTA is issuing a notification to the sender that the message did not reach its intended recipient, or that the quota has exceeded or any other status notification. From a user perspective, considering the amount of NDR messages that an organization receives and the fact that it did not send any of these original messages – it is spam, and if not, it’s definitely not legitimate.
Types of NDRs
Unfortunately, there is no industry standard for how NDR email messages should look nor how they should be treated. When an MTA issues an NDR, it usually sends it in one of the following forms:
- Full NDR – NDRs with original message info as attachment
- Partial NDR – NDRs with original headers and/or some parts of the original message in the body
- Empty NDR – NDRs with no recognizable data from the original message
Although there is no definitive evidence, from what we see in our detection centers, most MTAs return Partial NDRs. Although Empty NDRs is not the most common method, they really complicates the problem since it is very difficult to distinguish between “Legitimate NDR” from “Spam NDR”.
It is important to understand that Full NDR poses real security threats rather than just annoying spam messages, since it may contain malware attachments intended to infect the machine with malicious code.
Spam NDR has been around for years but has only recently gained recognition as a major spam issue, most of it can be associated to the massive trend of using zombie armies to propagate spam, and the never-ending endeavors of spammers to come up with new techniques to evade anti-spam solutions.
MTAs have two ways to inform each other about a message bounce situation: synchronous and asynchronous. Synchronous bouncing occurs when the receiving MTA denies the acceptance of a message during the SMTP session, and the message is not even received by the MTA. In this case, the sending MTA issues an internal message, usually by the “System Administrator” notifying the sending account that the message did not received.
The problem is that this method made it easy for spammers to engage in Directory Harvest Attacks (DHAs). DHAs are a common way for spammers to gather existing corporate email accounts. The idea is that the spammer connects to the corporate MTA and starts sending thousands of email addresses with the corporate domain (like john@, david@ marketing@, etc.) and just keeps track of which emails bounced back. Of course this type of attack could take some time, but from the spammer perspective, it’s an effective way to build an accurate database.
In order to deflect this problem, some MTAs have adopted the asynchronous approach. In this method, instead of telling the sending MTA right away that the recipient does not exists, it says in the session level that all recipients are valid. At some point, the MTA does its own checking and according to its own policy issues an NDR message to the recipient that appears in the FROM address. The idea behind this method was that spammers are just wasting their time with directory harvest attacks, since the MTA accepts all recipients. Another reason for having NDRs is that some delivery status, like quota size, cannot be checked on-session and must be triggered a delivery report.
Of course the asynchronous approach helped prevent the DHAs, but it created a new NDR problem. Along the way, anti DHA techniques were introduced, such as tarpit or Teergrube, which identified a DHA attack and delayed or blocked connection from sources associated with the attack. These methods are considered much more effective against DHAs rather than asynchronous bouncing, but unfortunately, there are still MTAs that implement the asynchronous method- which is the infrastructure for the Spam NDR.
Recent NDR Spikes
Recently, the NDR problem raised its head, and more and more incidents about the problem were reported. But what’s interesting is that we spotted a pattern, in which an individual user and sometimes a domain had a dramatic increase in the amount of spam NDRs and after a short while it got to the usual low-levels or even disappeared completely.
Analyzing the new patterns in the Spam NDR email messages showed that only a few accounts (in some cases even single accounts) were the target of the Spam NDR. So although it would be expected that every account in a domain will get the same amount of NDRs (this would have explain a theory that maybe spammers are using NDR to pass their spam through security solutions), only a few accounts got those messages.
We believe that this type of behavior relates to the increase of use of reputation techniques to block spam. Some reputation systems (depending on the vendor) use reputation scores in their decision process of determining if a given source is bad or good. One common technique is based on a simple fact that usually a legitimate MTA does not send email messages for more than a few domains. On the other hand, a compromised MTA or a zombie computer will send from a single IP address hundreds if not thousands of domain in a short timeframe. This is a behavior that some reputation systems track and analyze. Since its becoming a common technique, a spammer who wishes to deceive such a mechanism would use a small amount or even a single domain as the sender in order to avoid its IP from being considered bad.
Commtouch NDR Solution
Commtouch’s solution comes from a deep understanding of the problem and a proven expertise in the messaging technology space. As a result of the complexity of the NDR problem, Commtouch offers different approaches to tackle the problem in order maximize the best solution for each need.
Commtouch has applied a built-in logic to its RPD detection engine in order to distinguish NDR notifications from other types of inbound email messages. These NDR notifications are classified as “good” email by the detection engine and are redirected to the Inbox folders of targeted recipients. The functionality is enabled by default.
To prevent recipients from receiving Spam NDR emails, Commtouch has developed an NDR solution implemented in our recent released SDK (Ver 5.06). This solution can be enabled remotely by Commtouch on a per license key basis. The solution can be used to match different customer needs.
There are several ways to implement the NDR solution, depending on organizations’ needs:
- Full blocking of all NDR email messages – users that are not concerned about the potential loss of Legitimate NDR notifications can simply block all NDR messages, both legitimate and Spam. Although it may be considered a harsh policy, it is a fast and easy way to handle the NDR problem – No additional cost or effort is required.
- Blocking of spam NDRs only – to distinguish between the good and bad NDRs, a small configuration change to the mail server is necessary; the mail server needs to stamp outbound email messages according to our specifications (this tactic is also known as BATV). This stamped value is used as an identifying token for the detection engine when some of these messages are rejected and bounced-back as NDR notifications. The combination of auto-detecting a message as an NDR notification and the existence of the stamped token will result in classifying the message as Legitimate NDR and allow forwarding it to the Inbox folder. Any other type of NDR notification will be classified as Spam NDR and blocked.
Since implementing the latter mechanism in the MTA may require resources and time, for organizations that suffer from NDR attacks and are searching immediate relief, a two-phase approach needs to be considered. In this case, at first the NDR enhancement will be enabled without stamping at the MTA, in which case all NDRs will be blocked, both legitimate and spam NDRs. When possible the mechanism will be implemented to allow Legitimate NDRs to be received.
Licensing partners who are interested in more information about these solutions should contact their technical account manager.