Diving into DMARC: Can it really end spam, or at least phishing?

As the publisher of a photography newsletter, and the recipient of seemingly endless spam messages, I’ve got more than one reason to be interested in any new spam-fighting technique. After the fairly disappointing results from initial industry efforts — Sender Policy Framework (SPF) and Domain Keys Identified Mail (DKIM), the announcement of DMARC (Domain-based Message Authentication, Reporting, and Conformance) has rightfully been greeted with skepticism. Aside from glowing headlines repeating promises from the specifications promoters, there hasn’t been much real information reported on it. So I plunged into researching and implementing it, so I could pass along how it works and what it can and can’t do for you.

From reading initial press reports — citing glowing quotes from DMARC sponsors including PayPal and Microsoft — DMARC sounded like a whole new arsenal in the war on spam, and a lethal blow to the specific form of spam known as phishing, where mail purports to be from a trusted sender like [INSERT BANK NAME HERE], but is really from Nigeria or someplace else outside the reach of the DOJ. None of the articles said much about why DMARC would succeed where SPF and DKIM hadn’t, so I decided to dig in further and implement DMARC on my own mail server to judge the results. The first thing I learned is that DMARC is entirely dependent on SPF and DKIM, so a recap of how they work is in order.

SPF and DKIM: Today’s tools in the war on spam

In addition to various crowdsourced and AI-based tools for detecting spam from message contents, and campaigns to shut down major spammers, there are two primary tools used by mail hosts and ISPs to attempt to minimi e spam, and in particular, phishing: SPF and DKIM. SPF is older and simpler. It attempts to address a simple problem in the internet’s mail protocol (SMTP) — that any mail sender can claim to be sending mail from any other mail sender — simply by setting a flag or a header on their message. This quirk is necessary since often the machine eventually sending an email (perhaps a mail server deep in the bowels of Google or Yahoo) is not the same one that originated it (a trendy boutique sending out a newsletter, or your company that uses Google Apps, for example). But SMTP can’t tell whether it is you or a Nigerian Prince sending out that mail about your latest windfall or mugging, claiming to be from you, really.

With SPF, to avoid being spoofed, domains simply add a DNS text (TXT) record — or a newer, true SPF record — enumerating which mail servers are allowed to send mail on their behalf. They can list them by domain or IP address. The domain can further indicate what the recipient should do with mail not coming from one of the listed mail servers. It can recommend that the mail be rejected or that it be accepted (although presumably treated with considerable suspicion by any spam filters down the line).

The virtue of SPF is its simplicity. Anyone with access to their DNS records can implement it in a matter of minutes. And since DNS is hard to fake out, SPF records are reliable. In principle, wide adoption of SPF was supposed to eliminate phishing and reduce overall spam. Of course, that assumes that the spammers don’t use their own domains. If they do, they can simply add an SPF record authori ing themselves. Or spammers will often use accounts on otherwise trusted domains (like large webmail providers, hacked servers, or open mail relays) to circumvent SPF protection. This reduces the “whitelist” utility of SPF, since simply having a valid SPF record doesn’t mean much about the legitimacy of the sender.

An SPF record can be as simple as example.com. IN TXT "v=spf1 a mx -all" which allows only hosts with an MX (Mail eXchange) record for example.com to send mail for example.com, and all other mail should be rejected.

One obvious problem with a strict SPF policy is that it can make life difficult for mobile and traveling users. For example, if an employee wants to send an email from a shared computer or mobile phone using a server that isn’t the company’s, they would be unable to use their corporate email address. That may actually be a desired result for the company, but can cause a lot of confusion for employees. A similar problem can arise for those with lifetime, “portable” email addresses, often given to alums or organi ation members, and used on many different ISPs.

In my case, I implemented SPF a couple years ago on our mail server, hoping it would allow more of our newsletters to be received by our subscribers. To be honest, we didn’t detect any increase in the amount of mail getting through, which points to one of the problems with SPF — it doesn’t include any feedback on whether it is working or how well. One of the goals of DMARC is to fix that problem.

DKIM: High-tech anti-phishing

SPF works very poorly with forwarded mail. Because all it does is verify the immediate sender of a message, once the message has been forwarded there is no longer a way to tell whether the original message is authentic or fraudulent. This creates a huge hole for spammers to exploit, by claiming to have forwarded messages from legitimate domains. SPF is also hampered by the need to explicitly allow mail servers to act on a domain’s behalf — a hassle for large organi ations that might be adding and moving mail servers frequently. SPF isn’t very granular, either, operating only at the domain level, making it a little vague for large ISPs with millions of customers. DKIM was designed to address these issues.

With DKIM the actual message is digitally signed with the private half of a unique key pair generated for a domain. The public half is stashed in a DNS record, so any recipient can verify the message envelope, and the original sender. The message isn’t encrypted — for compatibility with non-DKIM recipients — but the signature verifies that the sender and contents are unaltered. DKIM addresses the forwarding issue head on, but at the cost of complexity. Both the sender and recipient have to put digital signature libraries and verifiers in their mail pipeline for it to work. Many major mail sending houses and ISPs will do this for you, but smaller sites can be stuck with an IT hassle.

A sample DKIM header looks like this:

DKIM-Signature a=rsa-sha1; q=dns;

d=example.com;

i=user@eng.example.com;

s=jun2005.eng; c=relaxed/simple;

t=1117574938; x=1118006938;

h=from:to:subject:date;

b=d dVyOfAKCdLXdJOc9G2q8LoXSlEniSb

av+yuU4 GeeruD00ls VoG4 HRNiY R

Note the public key for the example.com domain is visible for any mail recipient to use for verifying the origin of a mail message.

Almost everyone agrees that if SPF and DKIM really worked, and implementing them ensured that legitimate email was received, they would be worth the work they require. Unfortunately both standards are effectively black holes. A site implements them and sits back and waits to see if they can figure out if they are working. There really isn’t any simple way to tell if they are having an effect. That’s where the new DMARC comes in.

DMARC: Hearing a voice in the wilderness

DMARC, despite all the engaging headlines written quickly after the press release came out, doesn’t actually provide any new technical tools along the lines of SPF and DKIM. What it does do, which is just as important and has been missing until now, is give users of SPF and DKIM the ability to recommend policies to those receiving mail claiming to be from their domains, and also get reports back from compliant sites about mail from their site — including suspicious messages often from phishers or spammers attempting to trade on their brand or reputation.

Like SPF, DMARC is trivial to implement, as long as you have access to DNS records for your domain. It is simply a TXT record that advises DMARC-compliant sites what to do with mail that claims to come from your domain. The actual verification is done by SPF and DKIM (DMARC does nothing useful if you have neither of those deployed), but until DMARC there was no way to give instructions on how to handle passes and fails. A DMARC record can indicate that failing emails should either be passed anyway (although clearly the recipient may still choose to further examine, and delete or quarantine them), flagged as suspect, or failed.

An example DMARC record that simply requests reports from a site that receive mail from a domain is as simple as:

_dmarc IN TXT ( "v=DMARC1; p=none; rua=mailto:dmarc-feedback@example.com" )

For sites with tight control over their email and their brand, a “lockdown” policy of recommending deleting any suspect mail from their domain may be the best policy. For sites with many users of various kinds and less control over all the possible uses (a common case is a business user sending mail from their personal ISP attempting to use their business address as the from address — or attempting to use their personal email address from their business account) domain owners might choose a more lenient policy.

It’s important to restate that all the DMARC record does is give a recommendation to the recipient from the (claimed) sending domain. It is in no way binding on the recipient. DMARC doesn’t provide a report on what the recipient actually does with the mail, but it does offer informative reports on mail that has been received claiming to come from the sending domain, along with the result of the SPF and DKIM checks.

The reporting capability of DMARC is most of what makes it interesting. Senders can now find out how much mail is coming from their domain (or claiming to come from their domain), where it originated, and whether their SPF and DKIM policies are correctly flagging it as legitimate or suspect. A sender can specify it wants to receive reports for all failed messages, or a daily digest of results, or both.

The reports come in XML format, which has not coincidentally already generated a set of commercial report-processing tools from chief DMARC touter Agari, but even looking at them in a text editor can be revealing (that’s Agari’s sumo at the top of the story, incidentally). After implementing DMARC on a couple of sending domains, I immediately started getting reports from Google (which is apparently the only major ISP that has actually implemented DMARC so far, judging by the silence from the others). From the reports I could see not just that the SPF records were working correctly for several different hosted domains all using the same sending domain, but which other IP addresses were claiming to be sending mail from that domain. Since other major mail providers, including Hotmail, AOL and Yahoo!, have also signed onto the DMARC organi ation, hopefully coverage will begin increasing quickly.

Some of the other domains listed in the reports as having sent email originally from one of our domains were easy to explain — clients of ISPs like Yahoo or hotmail forwarding mail — but some were suspicious. A quick check on a spam-catching site like the Project Honey Pot site can then reveal something about the spammer, as this screenshot of part of a report on a suspect site shows. For small senders or even mid-si e businesses, this information may not be much more than a curiosity. For the big — and often phished — brand names behind DMARC, like AOL, Paypal, Bank of America, Facebook, and Fidelity, these reports might well be the basis for further black-listing or even other types of actions against phishing sites.

DMARC, like SPF and DKIM before it, is less exciting than the announcements trumpet, but it is another step in getting anti-spam and anti-phishing tools deployed, at least allowing organi ations to get a better handle on the problem and judge whether their tools effectiveness in addressing the issues.

Diving into DMARC: Can it really end spam, or at least phishing?