The Goal of SPF Records
Sender Policy Framework (SPF) is a simple system to prevent forgery of the sender’s address, by providing a mechanism for domain administrators to define a policy of who is authorised to send mail ”From” their domain (eg. your users, like [email protected]), using simple DNS (Domain Name System) records.
Syntax and Best Practice
Spam, phishing & virus/malware-infected mails almost always use a forged sender-address, and because SPF is so easy to implement & protects both the sender’s reputation and the safety of their customers (from “phishing” activities), it’s a no-brainer; everyone with a domain, who doesn’t want anyone else to forge mail from their domain, should have an SPF record.
The list of hosts/IPs authorized to send from the domain, is published in the DNS records for that domain, in the form of a specially formatted DNS text (TXT) record - here’s an exciting example, in “bind” zone-file format - see http://bit.ly/Zone_File
@ 300 IN TXT “v=spf1 mx a ip4:18.104.22.168 ip4:22.214.171.124/22 include:emailprovider.example.com -all”
The SPF record is within the quotes, and comprises the initial SPF version declaration (v=spf1), followed by any ”mechanisms” that form part of the allow policy, followed by a final ”qualifier” that detarmines what to do if none of the listed mechanisms, match with the sending IP address (it’s best to set this to “-all”, which will cause the SPF check to Fail any non-listed IP – therefore protecting the domain owner’s reputation, and any customers who receive legitimate communications from that domain).
Although SPF is quite simple, there are some pitfalls that can be avoided, if you know about them in advance of designing & implementing an SPF record for your domain.
Before implementing an SPF record, it is essential that you gather a complete list of IP addresses that need to send From your domain. This can be as few as 0 (zero) - if you do not use the domain for email, but can often contain many IP addresses from systems like webservers, printers, IoT devices, and your primary email-provider’s mail platform.
Once the SPF record is deployed, you can easily modify it to correct any missing elements, but it’s best to have as full a list as possible, to avoid any unexpected email rejections.
At the initial point of deployment, it’s a good idea to start with a short DNS record TTL (Time To Live) – the example above, is set to 300 seconds – so that any changes you need to make, only take a short time to take effect. Once you’re more confident of the record’s validity, you can then increase the TTL to something a little less taxing on your DNS server (3600s).
Using the Vamsoft SPF Policy Tester Advanced mode, to test your proposed SPF record, should prevent you from deploying a syntactically incorrect SPF record.
SPF’s whole reason for being, is to prevent unauthorised entities from forging email From your domain, in the SMTP envelope of emails. The only qualifier that does this, is the “-all” combination.
Many online resources will suggest “~all” as an appropriate qualifier, but this actually does nothing for anti-forgery, and should only be used for testing purposes & be switched to “-all”, in production.
Too many DNS lookups
It is little-known rule, that SPF records must be entirely resolvable, within 10 DNS lookups, or fewer; if more DNS lookups are required, then, randomly, there will be spurious SPF failures, as DNS responses are delivered with the records ordered in a round-robin fashion, meaning that sometimes, a valid sender IP will be validated, and sometimes, it won’t.
The best indicator of this issue is the dmarcian SPF Surveyor, mentioned in the troubleshooting section, below.
The biggest culprit of out-of-control DNS mechanisms in an SPF record, is where a domain has “include” mechanisms, for 3rd-party email-service providers.
With the plethora of email-based services (sales/marketing/telemetry/etc) that might be used by an entity, it is becoming increasingly likely that a domain might have to include a 3rd-party’s email platform within their SPF record. These providers are not normally concerned about the DNS-efficiency of their SPF records, which they require their customers to include in their SPF record, to enable proper mail flow. This often leads to needlessly nested includes, and off-the-charts (>10) DNS lookups.
SPF Checks Are An “Edge” Technology ONLY
By design, SPF checks are meant to be performed at the edge of your mail service – typically on the servers that comprise your MX records, where inbound mails arrive, from external parties – NOT, after mails have passed the edge MTAs (Mail Transfer Agents), and certainly NOT at the point of being delivered into your mailbox.
If a service is performing SPF checks, but it is not the MX, or at the mail-service’s “edge” (where it meets the raw Internet), it is highly likely that almost all SPF checks will be failures, and they will almost certainly be incorrect failures (false positives).
For instance, if you are a user of the Microsoft Office 365 suite, and also use a cloud-based email-scanning service from a 3rd-party – such as Symantec, you will need to make sure that your Office 365 Exchange Admin Center → Protection → Spam Filter → Default → Advanced Options → “SPF record: hard fail:” is set to “Off”. This is because for some reason, it is impossible to switch off SPF checking, within the Exchange Online service.
Troubleshooting & Diagnostics
There are many tools to help domain administrators to create and check SPF records. These are the ones I’ve found most useful.
SPF Policy Tester (by Vamsoft)
The output from this tester is the best I’ve found, and is very helpful in diagnosing false-positive/negative SPF outcomes.
Use the Basic tester to determine if a particular sender should be allowed to send From their domain, via a particular IP address.
Use the Advanced tester to override the SPF policy retrieved from DNS - perhaps for testing proposed changes to your SPF record, before putting them live.
SPF Surveyor (by dmarcian)
The SPF Surveyor is an SPF diagnostic tool that presents a graphical view of SPF records. It’s like an SPF record health-check.
It is particularly useful at finding and highlighting syntax errors and brilliant at identifying more esoteric issues like there being too many DNS lookups required to resolve the SPF record. It even provides hints at how to configure your DNS SPF records, to alleviate the problem, using “record flattening”.
The dmarcian SPF Record Flattener rewrites the surveyed record by removing duplicate netblocks, collapsing any overlapping netblocks, and using 0 DNS-querying mechanisms / modifiers. Each SPF record is kept to less than 512 bytes to fit into a single UDP packet (assuming no other TXT records are sharing the DNS label).
There are so many corporate entities out there that because of their lack of SPF record or worse, because of their ignorance of how SPF works, are leaving their reputations and their customers data/privacy at risk (usually from phishing activities). This makes me sad - sigh.
Whenever a company apologizes to their customers, because they have received phishing emails that appear to come from the company’s domain, I check to see if they have properly implemented SPF. There are usually just 2 states in play: -
Embarrassing - there’s an SPF record, but it’s configured to be completely ineffectual for SPF’s raison d’être (protecting the domain from forgery)
Disgraceful - there’s no record at all, as if they don’t even care about it (though more likely they’re just completely ignorant of SPF)
The number of banks and financial institutions that fall into this category, is eye-opening.
Entities warning its customers about scam emails that have been reported to them, which would not be possible, if the domain owners had been using SPF, or had configured their SPF record correctly!
Here are some examples…
% host -t TXT just-eat.co.uk
just-eat.co.uk descriptive text “v=spf1 include:_spf.google.com include:spf1.just-eat.co.uk include:spf2.just-eat.co.uk include:_spf.salesforce.com include:mail.zendesk.com ~all”
% host -t TXT joker.com
joker.com has no SPF record