how to build a virtual user email server

This guide shows you how to build an email server that can send and receive emails for multiple domains on behalf of users who do not have shell accounts on the server, a.k.a. "virtual users."

These steps were developed and tested on a fresh installation of Ubuntu Server 12.10.

about the examples

In the examples that follow, we will configure a new virtual user email server to accept and queue emails for two distinct virtual users who receive emails for different domains. Neither user will require a shell account, and both users will be able to securely send and receive emails through this server.

Step 1: Install Ubuntu Server.

This guide will best reflect your setup experience if you are starting from scratch. Obtain an installation CD or DVD for Ubuntu Server 12.10 and install Ubuntu Server. The default settings should work just fine; just be sure you opt to install Postfix and Dovecot.

After you have completed the installation, install all outstanding updates.

Details on how to install Ubuntu and its updates are not provided here. (Frankly, if you need help with this, you probably are not ready to build a virtual user email server.)

Step 2: Create a special user to own all virtual email accounts.

Since your server will be receiving messages for users who do not have shell accounts, you'll need to designate a special user to own those users' mailboxes. We'll create a system user called vmail for this purpose. Issue the following command:

sudo useradd -mrs /bin/bash vmail

This command is interpreted as follows:

• "useradd" tells Ubuntu that you want to create a new user.
• "-m" tells useradd to create a new home folder for vmail. This will be created at /home/vmail.
• "-r" tells useradd that the new account is a system account. This mainly influences the selection of values for the new UID and GID. In Ubuntu, the IDs for regular users begin at 1000, so if no other system users have been created since Ubuntu was installed, useradd will probably choose 999 for both the UID and GID.
• "-s /bin/bash" tells useradd that you want vmail's default shell to be /bin/bash. This switch isn't strictly necessary, but if you are a bash user, this option will make you more comfortable when you are later shelling around as user vmail.

As previously mentioned, if your Ubuntu installation is fresh, this command will probably set the UID and GID of the new system user to 999. However, if the IDs are different, then you'll need to make a note of them for later. Type the following commands to confirm and note the IDs:

tail /etc/passwd
tail /etc/group

The last line of the output of both commands will reveal the UID and GID of the new user vmail.

Step 3: Copy the default Postfix configuration.

Postfix settings are stored primarily in the /etc/postfix folder. Put your terminal into that folder now:

cd /etc/postfix

By default, when Postfix is installed, a configuration file is not created. Let's create a basic Postfix configuration by copying one of the sample configuration files, which contains reasonable defaults:

sudo cp -a /usr/share/postfix/main.cf.debian main.cf

We'll customize this file in a bit.

Step 4: Declare the virtual hosts.

We will begin our virtual host configuration by creating a file listing all the domains your new mail server will accept messages for. This file is called vhosts, and it resides in the Postfix configuration folder. Open the file in an editor:

cd /etc/postfix
sudo nano vhosts

List the name of each host, one per line. For example:

# Postfix will send and receive mail for the following domains:
sharkysoft.com
hippiechickpea.com

As demonstrated above, if you want to include comments in this file, precede them with "#". After you have entered all of your virtual domains, save the file and exit the editor.

Step 5: Declare the virtual users.

Next, let's create a list of email addresses and the folders where those addressees' messages will be stored. Start a new file using the following command (your current folder is still /etc/postfix, right?):

cd /etc/postfix
sudo nano vmaps

Enter your virtual user email addresses and their respective delivery folders, one per line, as demonstrated below. (Real email addresses are not used in this example, for obvious reasons.) As before, comments begin with "#".

# Remember to execute "sudo postmap vmaps" every time you update this file!
# Trailing slash is required to ensure Maildir storage format is used

spam@sharkysoft.com      sharkysoft.com/spam/mail/
spam@hippiechickpea.com  hippiechickpea.com/spam/mail/

The trailing slash is very important. It signals to Postfix that you want email stored in the "Maildir" format. (If you omit the slash, Postfix will use the "mbox" format instead, which is not only an older and inferior format, but it is also quite fragile. If you're going to store email as files on your server, Maildir is really your best option.)

Storing the maildir files in a dedicated subfolder, such mail in the example above, while not strictly necessary, is a wise practice. This is because future expansion of your system's capabilities for virtual users may result in a need to store files that don't belong to the maildir structure.

Truth be told, after you finish this guide, the locations you enter into this file won't matter anymore, because we'll rely on a delivery agent other than Postfix. For now, however, it's nice to know we can still use Postfix as the delivery agent.

After you save this file, execute the following command:

sudo postmap vmaps

This command compiles vmaps into a database format (stored in vmaps.db) that is more efficient for Postfix to use. Be sure to re-run this command every time you edit vmaps.

If you see the following error message...

postmap: fatal: open /etc/postfix/main.cf: No such file or directory

...it means you jumped the gun a bit. Return to Step 3 to copy the default Postfix configuration file.

Step 6: Enable virtual delivery.

With our Postfix data files in place, it's now time to point Postfix in their direction. Open the configuration file we copied earlier.

cd /etc/postfix
sudo nano main.cf

At the bottom of the file, append the following configuration:

home_mailbox = Maildir/
myhostname = your_server_name
virtual_mailbox_domains = /etc/postfix/vhosts
virtual_mailbox_base = /home/vmail
virtual_mailbox_maps = hash:/etc/postfix/vmaps
virtual_minimum_uid = 999
virtual_uid_maps = static:999
virtual_gid_maps = static:999

Replace "your_server_name" with the host name of your mail server. This name must not be the same as any of your virtual hosts. Also, if necessary, replace 999 with the appropriate UID and GID for user vmail.

Postfix is now configured to accept messages for virtual users. Activate your changes by restarting Postfix:

sudo service postfix restart

Step 7: Test virtual delivery.

Before we go any farther, we should generate some test emails to see if they get delivered correctly. You can do this easily from the terminal using a command like the following:

mail -s "virtual delivery test" spam@sharkysoft.com spam@hippiechickpea.com

After you enter this command, the terminal will accept your plain text email. Type as many lines as you please, and when you are finished, end your message by entering a line containing only a period ('.'). mail will then ask you for CC recipients. Just press enter to skip this prompt.

After you exit the mail command, your messages should be delivered immediately. When Postfix delivers your test emails, it will automatically create the Maildir folders required to store them. Inspect these folders now, by assuming the identity of the vmail user.

sudo su vmail
cd
ls
# From here, start diving into the folders and looking around to verify that mail was delivered.

If you're using the Maildir mail storage format, you'll find the newly delivered messages in the new folder for each virtual user.

When you're finished poking around, return to your normal identity with the following command:

exit

Step 8: Point Dovecot to the virtual user mailboxes.

If you've verified that Postfix is delivering mail correctly to the virtual user mail folders, then it's time to point Dovecot to the same folders. We'll do this in a custom Dovecot configuration file, local.conf, which is automatically processed by Dovecot when Dovecot starts, if the file exists. Open this file now:

cd /etc/dovecot
sudo nano local.conf

You are probably creating a new file, so on't be alarmed if the file is empty. Enter the following line to point Dovecot to the virtual user mailboxes:

mail_location = maildir:/home/vmail/%d/%n

Save your work and exit the editor.

Step 9: Create the virtual users password file.

Even though Dovecot knows where your users' mail is, it won't do anyone any good if they can't read it. Dovecot needs to know how to authenticate users who want to read their mail.

In a typical Linux system, user information is stored in /etc/passwd and /etc/shadow. These files list physical users who have shell accounts. In our system, however, we want to provide access to virtual users who don't have shell accounts.

We will accomplish this by storing virtual users' credentials in a similar fashion, but in a file that only Dovecot uses. We'll store these credentials at /etc/dovecot/users. You'll have to build this file from scratch. When you are finished, it will look something like this:

spam@sharkysoft.com:{CRAM-MD5}bce47bb8…e6eae0c0:999:999
spam@hippiechickpea.com:{CRAM-MD5}6a80066e…66f9c335:999:999

As you may have gathered, each line lists one user. The format for a single line is:

email_address:password_hash:UID:GID

The passwords are generally stored as hashes that begin with the name of the hashing scheme, surrounded by curly braces. To generate a password hash, use the following command:

doveadm pw

Now that you know what you need to build, let's get started. Open a second terminal that you can use to generate passwords. (You can copy them into the first terminal.) In the first terminal, open your (new) users file now.

cd /etc/dovecot
sudo nano users

Add one line for each virtual user. When you are finished, save your work and exit the editor.

Step 10: Direct Dovecot to the new password file.

With your virtual user password file built, let's instruct Dovecot to start using it. Open Dovecot's authorization configuration file:

cd /etc/dovecot
sudo nano conf.d/10-auth.conf

Locate the following disabled configuration directive (near the end of the file):

#!include auth-passwdfile.conf.ext

Enable it by deleting the '#' character at the beginning.

Also look for this line:

!include auth-system.conf.ext

Disable it by inserting a '#' character.

Save your changes and exit the editor.

Finally, activate your changes by restarting Dovecot:

sudo service dovecot restart

Step 11: Verify IMAP access.

At this point, it should be possible to connect to Dovecot with a mail reader and download the test messages you previously sent. I recommend Thunderbird, an easy-to-use mail reader that supports a variety of configurations.

Configure Thunderbird to connect to your new mail server. See if you can read your messages. By default, Dovecot does not permit any authentication method that sends your password in the clear. Keep this in mind as you configure your connection in your mail reader.

For IMAP, supported security settings include:

• STARTTLS
• SSL/TLS

I haven't bothered checking POP3 connection settings, as I never use them.

Step 12: Enable SASL authentication.

It's a safe bet that at least some of your virtual users will not be content to just read their mail; some of them will probably want to send mail, too. If you want to allow outside users -- including your virtual users -- to send mail through your server, you should demand authentication. This is because leaving your server open to just anyone will expose your server to abuse by spammers.

In this step, we'll enable a service in Dovecot, called "SASL," that Postfix can use to validate user credentials.

Enable SASL service in Dovecot.

Dovecot provides a service that Postfix can leverage to authenticate your virtual users. Let's turn that service on. Reopen local.conf:

cd /etc/dovecot
sudo nano local.conf

Add the following lines:

service auth {
  unix_listener /var/spool/postfix/private/dovecot-auth {
    mode = 0660
    user = postfix
    group = postfix
  }
}

Activate your changes:

sudo service dovecot restart

Connect Postfix to SASL service.

Now that the authentication service is enabled in Dovecot, let's tell Postfix to use it. Reopen main.cf:

cd /etc/postfix
sudo nano main.cf

Add the following lines:

smtpd_sasl_auth_enable = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/dovecot-auth
smtpd_sasl_authenticated_header = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain = $myhostname
broken_sasl_auth_clients = yes
smtpd_recipient_restrictions =
	reject_unknown_sender_domain,
	reject_unknown_recipient_domain,
	reject_unauth_pipelining,
	#permit_mynetworks,
	permit_sasl_authenticated,
	reject_unauth_destination
smtpd_sender_restrictions = reject_unknown_sender_domain

Save your changes and exit the editor. Activate your changes:

sudo service postfix restart

Step 13: Test mail delivery from a MUA.

In the previous step, you secured your mail server so that only authenticated users can relay mail through it. However, authentication should not be necessary to deliver mail directly to addresses managed by your server -- even if the sender is untrusted. Otherwise, how will anyone ever send your users messages? In this step, we'll test both direct delivery and SASL-authenticated relay access.

Test direct delivery.

To verify this, configure a MUA on a separate machine to use your new server as the outgoing mail server. Thunderbird is a great choice for testing your server, because it is easy to configure and is available on many platforms. Configure Thunderbird to use the following settings for outgoing mail:

Server Name:	any name that resolves to your server's IP address
Port:	the standard SMTP port, 25
Connection security:	None
Authentication method:	No authentication
User Name:	blank

If you're using another MUA, look for similar parameters and try to configure them the same way.

Now see if you can send a few messages using this configuration. Because you have configured your MUA to bypass authentication, you should only be able to send mail directly to addresses that your server is configured to receive messages for. Try that first. Verify that they were delivered by checking the appropriate inboxes.

Test unauthenticated relay access.

Next, try to send a message to an address not managed by your server, such as your favorite Gmail account. Since you are not authenticating, the server should refuse, with a message such as, "relay access denied." If it does not refuse, then you have a serious configuration problem and you should take immediate steps to resolve it, before spammers enslave your machine.

Test authenticated relay access.

Now let's enable authentication to see if you can send messages to the rest of the world. This will also allow you to test your SASL configuration. Update your MUA's outgoing server configuration to use authentication. In Thunderbird, do this by setting Connection security to Password, transmitted insecurely. You'll also need to set User Name to one of your virtual user email addresses.

Note that this test will require you to send passwords unencrypted over the network. If the network between your test machine and your new server is trusted, this should not present any risk. However, if you have concerns, you should not perform this part of the test until we have enabled encryption.

Apply this new configuration and try sending another message to your favorite Gmail account. Your MUA should prompt your for a pssword, and the message should be delivered. If this was your experience, then Postfix is successfully authenticating through SASL.

Step 14: Create and install security certificates.

Right now, your Postfix configuration does not offer any of the benefits of encrypted communication. This means that when your virtual users authenticate to relay a message, their passwords will be transmitted in the clear. This is bad news if the network path between all of your users and your server is not totally trusted, and it is considered bad practice even in a trusted network.

Your Dovecot configuration is in slightly better shape. My testing has led me to believe that Dovecot's default configuration will not allow user authentication in the clear. When Dovecot is installed, it automatically generates self-signed security certificates for secure communications. These are, perhaps, not the security certificates you want to use long-term, but they'll get you by in a pinch.

We will upgrade your Postfix installation to use encrypted authentication. Before we can do that, however, we need to create and install some security certificates.

Details on how this is done is provided by many guides freely available on the internet. The examples in the following steps are based on the premise that certificate files are stored at /etc/ssl/certs/sharkysoft.com.crt and /etc/ssl/private/sharkysoft.com-insecure.key.

Step 15: Enable TLS in Postfix.

Since many mail servers relay messages in the clear, the primary benefit you will gaim from configuring encrypted communications with Postfix is the protection of your users' passwords.

STARTTLS is a protocol that allows client/server communications that have started on an unencrypted channel to escalate to an encrypted format. Most halfway decent mail clients support this. Let's enable this in Postfix now:

cd /etc/postfix
sudo nano main.cf

Append the following:

# TLS parameters
smtpd_use_tls = yes
smtpd_tls_cert_file = /etc/ssl/certs/sharkysoft.com.crt
smtpd_tls_key_file = /etc/ssl/private/sharkysoft.com-insecure.key
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_tls_received_header = yes
smtpd_tls_mandatory_protocols = SSLv3, TLSv1
smtpd_tls_mandatory_ciphers = medium
smtpd_tls_auth_only = yes
tls_random_source = dev:/dev/urandom

I apologize for not taking the time to explain all of these configuration directives right now. To be honest, I'm not sure I understand them all well enough to explain them anyway. But I know they work. Google is your friend.

Save your changes, exit your editor, and restart Postfix.

sudo service postfix restart

Then see if you can still send messages to your favorite Gmail account. This time, your communications should be secure.

Step 16: Configure Dovecot to use your certificates.

You can improve Dovecot's secure communications by telling it to use your certificates, rather than the default certificates that it automatically created when the program was installed.

Open local.conf again:

cd /etc/dovecot
sudo nano local.conf

Add the following lines, which will override the default certificates:

ssl_cert = </etc/ssl/certs/sharkysoft.com.crt
ssl_key = </etc/ssl/private/sharkysoft.com-insecure.key

(Don't forget to change the certificate names to the names of your own certificates.)

In Ubuntu, it appears that the following settings are enabled by default, and that you should have no need to specify them. However, just in case, here are some additional settings it won't hurt to add:

ssl = yes
disable_plaintext_auth = yes
auth_mechanisms = plain login

Restart Dovecot:

sudo service dovecot restart

Use your MUA to verify that you can still access your virtual users' email. Don't be alarmed if your MUA gives you some sort of security warning, since you previously connected to Dovecot using different certificates.

What's next?

Congratulations! If you testing went well, then you have successfully built a virtual user email server. This is what you have accomplished:

• Your server can receive and deliver emails on behalf of your virtual users.
• Your virtual users can securely connect to your server with a MUA to read or download their messages.
• Your virtual users can securely connect to your server to send messages to the outside world.

This is a pretty good start. However, there's still more you need to do if you want a top-notch email server. At the very least, you should install some spam protection on your server. A program called Spamassassin is a good choice for this. We'll get to that in another guide, which is now officially on my to-write list. Thanks for reading, and have fun!