Tag Archives: email

Email infrastructure made right. Part 3: feedback loops.

In the previous posts (part 1, part 2) we covered some required items which should be implemented to send emails safely. Here’s one more item you should consider before starting sending emails: feedback loops. It works next way. You register on FBL and if some of your recipients mark your mail as spam you will get a special message about it. So you can take necessary actions. You can get a list of available feedback loops on corresponding Wikipedia page. It’s free of charge. For MSN, Zoho and Yahoo you will need their accounts, Google doesn’t have FBL.

Email infrastructure made right. Part 2: handling bounces.

In our previous post on email infrastructure we were talking about minimum requirements which should be implemented. Today we will cover bounces handling. What do the bounces mean? In short words, it’s a non delivery report. You could get if you send a mail to non-existing user (error code 5.1.1 ) or a user which has full mailbox (5.2.2).

There are kinds of the bounces:

  • soft bounces;
  • hard bounces.

If you get soft bounces it means that you can try to send an email later. And hard bounces mean that email account you are trying to send to is invalid. Such email accounts should not be used for further mail sending. And if you have a lot of subscribers you need a way to get rid of invalid email accounts. But how can you do it? Fortunately there’s a simple way. Probably you’ve heard about VERP. It stands for Variable envelope return path. Here’s how it works (we will cover Sendmail and Dovecot based setup on Centos).

Let’s assume we want to send a mail to joe@example.com from no-reply@serverbeep.com. If there’s no user joe@example.com we will get a bounce message to no-reply@serverbeep.com. But why don’t we collect all bounces in one mailbox? For simplicity of handling bounces we can setup some mail account. Let’s called it bounces@serverbeep.com and setup our LDA to deliver all mails to some user account (it could be system user bounces or some other account depending on your setup). Set up Sendmail to forward all incoming mail sent to bounces@serverbeep.com to system account bounces. Open /etc/aliases files and add next line:

bounces@serverbeep.com: bounces

Tell Sendmail to use Dovecot as LDA:

FEATURE(`local_procmail', `/usr/libexec/dovecot/deliver',`/usr/libexec/dovecot/deliver -d $u')
MODIFY_MAILER_FLAGS(`LOCAL', `-f')
MAILER(procmail)dnl

It should be placed to /etc/mail/sendmail.mc.

Apply Sendmail changes:

cd /etc/mail ; make all restart

If you don’t have Dovecot installed it’s time to install it:

yum install dovecot

Here’s an example of Dovecot configuration file working as LDA:

# 2.0.9: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-279.1.1.el6.x86_64 x86_64 CentOS release 6.3 (Final)
disable_plaintext_auth = no
lda_mailbox_autocreate = yes
listen = 127.0.0.1
mail_location = maildir:~/Maildir
mbox_write_locks = fcntl
passdb {
driver = pam
}
postmaster_address = postmaster@serverbeep.com
ssl = no
userdb {
driver = passwd
}
protocol lda {
auth_socket_path = /var/run/dovecot/auth-master
info_log_path = /var/log/dovecot/delivery-info.log
log_path = /var/log/dovecot/delivery.log
postmaster_address = postmaster@serverbeep.com
}

(Make sure you have /var/log/dovecot and dovecot has the permissions to write there).

Well, since we prepared everything let’s get back to VERP. To get a bounces while sending a mail to joe@example.com to bounces user maildir we need to add Return-Path header to our mail. Short example in PHP:

<?php
mail("joe@example.com", "Message subject", "Message body", "From: no-reply@serverbeep.com\n", "-fbounces+joe=example.com@serverbeep.com");
?>

So, there will be additional header in the email:

Return-Path: bounces+joe=example.com@serverbeep.com

In this case if there’s no user joe@example.com we will get a bounce message in our POP3/IMAP4 account bounces from where the messages could be easily fetched with simple Python script.

import imaplib
M = imaplib.IMAP4('127.0.0.1')
M.login('bounces', 'strongest_password')
M.select()
typ, data = M.search(None, 'ALL')
for num in data[0].split():
typ, data = M.fetch(num, '(RFC822)')
print 'Message %s\n%s\n' % (num, data[0][1])
M.close()
M.logout()

Once you get the messages you can parse it to get a list of invalid email accounts and make a modification of your users table in the database.

Email infrastructure made right. Part 1: basic requirements.

If you send emails quite often like we do at ServerBeep.com (we send email notifications if something bad happen with our customers’ websites or services), you had better to prepare your email setup before you start. It’s better to do it before than to solve the issues after. There are few items described below which should be consider as required before you start.

Check you IP addresses history

Sender Score and Sender Base allow you to check your IP history. If you find any issues you’d better consider changing it.

MX and PTR DNS records

To send a mail you have to setup PTR and MX DNS records properly so your IP address should point to your domain. A lot of SMTP server would reject your mail if you don’t have correct PTR record.

[root@sb1 ~]# host 88.198.20.28
28.20.198.88.in-addr.arpa domain name pointer serverbeep.com.
[root@sb1 ~]#

SPF

SPF stands for Sender Policy Framework. It allows you to tell other SMTP servers which IP and/or domain are allowed to send a mail from your domain. And it’s quite easy to setup. There’re a lot of online services which could help you with it.

Here’s for instance, our SPF record:

serverbeep.com. 42008 IN SPF "v=spf1 ip4:88.198.20.28 ip6:2a01:4f8:130:32a4::28 ~all"

Besides, it would be better to have the same one in TXT record. Just in case.

If you check the headers from the mail you received to your Gmail account you can see how Google treats SPF:

Received-SPF: pass (google.com: domain of no-reply@serverbeep.com designates 2a01:4f8:130:32a4::28 as permitted sender) client-ip=2a01:4f8:130:32a4::28;
Authentication-Results: mx.google.com; spf=pass (google.com: domain of no-reply@serverbeep.com designates 2a01:4f8:130:32a4::28 as permitted sender) smtp.mail=no-reply@serverbeep.com; dkim=pass header.i=@serverbeep.com

Using Gmail is the simplest way to check if you set up SPF properly. Besides there are a lot of other ways to check online. Just google for ‘spf check’.

DomainKeys Identified Mail

To setup DKIM you need to generate public and private keys. Public should be added to your domain zone while private key should be securely stored on you server. SMTP server, Sendmail for instance, uses private key to add a digital signature which can be used for email validation.

Here’s an example of DKIM record:

default._domainkey.serverbeep.com. 41765 IN TXT "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDXFOEq2OCK1Bpp7YpId3nSgJX8fcfl5bx3a9aiDHm5nCAx0ZmAPacOoo+dmFlBfbcAIbi2BPons6w/uIW4rSeFAuybGw04/wMOkVit1OJPoiCsGW9BJLPeMnez+6m32zv3drVjgeywxtVDNbQphOIJdk4S88O2hlWOsusYv4sEdQIDAQAB"

The same as for SPF, you could check your DKIM by sending a mail to your Gmail account. Look for DKIM-Signature and Authentication-Results headers.

Necessary mail aliases: abuse@, postmaster@ and fbl@

If there are any issues other postmasters would send a mail to some of these aliases. So again, you should definitely have them set up.

Smart Network Data Services by Microsoft

Register on SNDS to be informed if there’re any issues with Hotmail.

Of course, it’s not all required steps. But probably the most important ones.