VoiceMail IMAP Integration

“Unified messaging” has been a buzzword in the telecommunications industry for ages. It’s all about integrating services together so users can access the same types of data in multiple locations, using different methods. One of the most touted applications is the integration of email and voicemail. Asterisk has been doing this for years, but many of the larger companies are still trying to get this right. Asterisk has had the ability to send users voicemails via email, using the Mail Transport Agent (MTA) in your Linux distro (this always used to be sendmail, but Postfix has become increasingly popular as an MTA). Voicemail to email is one of the oldest features in Asterisk, and it normally works without any configuration at all.[158]

Internet Message Access Protocol (IMAP) integration has existed in Asterisk (and been steadily evolving) since version 1.4. IMAP voicemail integration means your users can access their voicemails via a folder within their email accounts, which gives them the ability to listen to, forward, and mark voicemail messages with the same flexibility that the Asterisk VoiceMail() dialplan application gives. Asterisk will be aware of the statuses of those messages when the users next log in via the phone system.

As the number of administrators integrating Asterisk with their IMAP servers has increased, the number of bugs filed and fixed has first increased and then decreased, to the point where IMAP integration can be considered stable enough for production use. In this section we’ll discuss how to compile in IMAP voicemail support and connect your voicemail system to your IMAP server.

Compiling IMAP VoiceMail Support into Asterisk

To get IMAP voicemail support into Asterisk, we need to compile the University of Washington’s IMAP library. The UW IMAP toolkit will give us the functionality in Asterisk to connect to our IMAP server. Before compiling the software, though, we need to install some dependencies.

The dependencies for building the IMAP library include the tools required to build Asterisk, but the way we’re building it also requires the development libraries for OpenSSL and Pluggable Authentication Modules (PAM). We’ve included instructions for both CentOS and Ubuntu.

CentOS dependencies

Installing both the OpenSSL and PAM development libraries on CentOS can be done with the following command:

$ sudo yum install openssl-devel pam-devel

Note

Remember to add .x86_64 to the name of each package if installing on a 64-bit machine.

Ubuntu dependencies

Installing both the OpenSSL and PAM development libraries on Ubuntu can be done with the following command:

$ sudo apt-get install libssl-dev libpam0g-dev

Note

If you try to install libpam-dev on Ubuntu, it will warn you that libpam-dev is a virtual package and that you should explicitly install one of the packages in the list it presents you with (which in our case contained only a single package). If libpam0g-dev is not the correct package on your version of Ubuntu, try installing the virtual package. This should give you a list of valid packages for the PAM development library.

Compiling the IMAP library

Now that we have our dependencies satisfied, we can compile the IMAP library that Asterisk will use to connect to our IMAP server.

The first thing to do is change to the thirdparty directory located under the asterisk-complete directory. If you have not already created this directory, do so now:

$ cd ~/src/asterisk-complete
$ mkdir thirdparty
$ cd thirdparty

Next up is downloading the IMAP toolkit and compiling it. The next steps will get the latest version of the IMAP toolkit from the University of Washington’s server (more information about the toolkit is available at http://www.washington.edu/imap/):

$ wget ftp://ftp.cac.washington.edu/mail/imap.tar.Z
$ tar zxvf imap.tar.Z
$ cd imap-2007e

Note

The directory name imap-2007e may change as new versions of the toolkit become available.

There are a few options we need to pass to the make command when building the IMAP library, and the values you should pass will depend on what platform you’re building on (32-bit vs. 64-bit), if you need OpenSSL support, and whether you need IPv6 support or just IPv4. Table 18.1, “IMAP library compile time options” shows some of the various options you could pass on different platforms.

Table 18.1. IMAP library compile time options

OptionDescription
EXTRACFLAGS="-fPIC"Required when building on 64-bit platforms.
EXTRACFLAGS="-I/usr/include/openssl"Used for building in OpenSSL support.
IP6=4Many platforms that support IPv6 prefer that method of connection, which may not be desirable for all servers. If you would like to force IPv4 as the preferred connection method, set this option.

If you look in the Makefile shipped with the IMAP library, you will find a list of platforms for which the library can be compiled. In our case, we’ll be compiling for either CentOS or Ubuntu with PAM support. If you’re compiling on other systems, take a look in the Makefile for the three-letter code that tells the library how to compile for your platform.

To compile for a 64-bit platform with OpenSSL support and a preference for connecting via IPv4:

$ make lnp EXTRACFLAGS="-fPIC -I/usr/include/openssl" IP6=4

To compile for a 32-bit platform with OpenSSL support and a preference for connecting via IPv4:

$ make lnp EXTRACFLAGS="-I/usr/include/openssl" IP6=4

If you don’t wish to compile with OpenSSL support, simply remove the -I/usr/include/openssl from the EXTRACFLAGS option. If you would prefer connecting by IPv6 by default, simply don’t specify the IP6=4 option.

Note

When installing IMAP support, we have always compiled the c-client library from source. However, it may be available as a package for your distribution. For example, Ubuntu has a libc-client-dev package available. It may work and save you some trouble, but we have not tested it.

Compiling Asterisk

After compiling the IMAP library, we need to recompile the app_voicemail.so module with IMAP support. The first step is to run the configure script and pass it the --with-imap option to tell it where the IMAP library exists:

$ cd ~/src/asterisk-complete/asterisk/1.8.0
$ ./configure --with-imap=~/src/asterisk-complete/thirdparty/imap-2007e/

Once the configure script has finished executing, we need to enable IMAP voicemail support in menuselect:

$ make menuselect

From the menuselect interface, go to Voicemail Build Options. Within that menu, you should have the option to select IMAP_STORAGE.

Tip

If you don’t have the ability to select that option, check to make sure your IMAP library was built successfully (i.e., that you have all the required dependencies installed and that it didn’t error out when building) and that you correctly specified the path to the IMAP library when running the configure script. You can also verify that the IMAP library was found correctly by looking in the config.log file (located in your Asterisk build directory) for IMAP.

After selecting IMAP_STORAGE, save and exit from menuselect and run make install, which will recompile the app_voicemail.so module and install it to the appropriate location. The next step is to configure the voicemail.conf file located in /etc/asterisk/.

Configuring Asterisk

Now that we’ve compiled IMAP support into Asterisk, we need to enable it by connecting to an IMAP-enabled server. There are many IMAP servers that you could use, including those supplied with Microsoft servers, Dovecot, and Cyrus on Unix, or a web-based IMAP server such as that supplied by Google’s Gmail.[159] Our instructions are going to show how to connect Asterisk to a Gmail account with IMAP enabled, as it requires the least amount of effort to get up and running with IMAP voicemail, but these instructions can easily be adapted for use with any existing IMAP server.

Enabling IMAP on your Gmail account

Enabling IMAP support on your Gmail account is straightforward (see Figure 18.1, “Enabling Gmail IMAP”). Once logged into your account, select Settings from the upper-right corner. Then select Forwarding and POP/IMAP support from the task bar under the Settings header. In the IMAP Access section, enable IMAP support by selecting Enable IMAP. After enabling it, click the Save Changes button at the bottom of the screen.

Figure 18.1. Enabling Gmail IMAP

Enabling Gmail IMAP

Configuring voicemail.conf for IMAP

To enable our voicemail system to connect to an IMAP system, we need to make sure IMAP support has been built into the app_voicemail.so module per the instructions in the section called “Compiling Asterisk”. With IMAP support compiled into Asterisk, we just need to instruct the voicemail module how to connect to our IMAP server.

We’re going to demonstrate how to connect to an IMAP-enabled Gmail account and use that to store and retrieve our voicemail messages. If you haven’t already, read the section before proceeding. The final step is configuring voicemail.conf to connect to the server.

In voicemail.conf, add the following lines to the [general] section. Be sure you only specify a single format (we recommend wav49) for voicemail recordings, and remove any references to ODBC voicemail storage if you’ve enabled that previously:

[general]
format=wav49              ; format to store files
imapserver=imap.gmail.com ; IMAP server location
imapport=993              ; port IMAP server listens to
imapflags=ssl             ; flags required for connecting
expungeonhangup=yes       ; delete messages on hangup
pollmailboxes=yes         ; used for message waiting indication
pollfreq=30               ; how often to check for message changes

Before we configure our user for connecting to the Gmail IMAP server, let’s discuss the options we’ve just set in the [general] section. These are the basic options that will get us started; we’ll do some more customization shortly, but let’s see what we’ve done so far.

First, the format=wav49 option has declared that we’re going to save our files as GSM with a WAV header, which can be played on most desktops (including Microsoft Windows) while retaining a small file size.

Next, we’ve configured Asterisk to connect to an imapserver located at imap.gmail.com on imapport 993. We’ve also set imapflags to ssl, as Gmail requires a secure connection. Without the ssl IMAP flag being set, the server will reject our connection attempts (which is why it was important that we compiled our IMAP library with OpenSSL support). Another option that may be required on private IMAP servers such as Dovecot is to specify novalidate-cert for imapflags when an SSL connection is necessary, but the certificate is not generated by a certificate authority.

Next, we’ve set expungeonhangup=yes, which causes messages marked for deletion to be removed from the server upon hangup from the VoiceMail() application. Without this option, messages are simply marked as read and left on the server until they have been removed via an email application or web interface.

In order to get message waiting indication (MWI) updates correctly, we need to enable pollmailboxes=yes, which causes Asterisk to check with the server for any changes to the status of a message. For example, when someone leaves us a voicemail and we listen to it by opening the message via our email application, the message will be marked as read, but without polling the mailbox Asterisk will have no way of knowing this and will enable the MWI light on the associated device indefinitely. Finally, we’ve set the related option pollfreq to 30 seconds. This option controls how often Asterisk will ask the server for the status of messages: set it appropriately to control the amount of traffic going to the voicemail server.

Table 18.2, “Additional IMAP voicemail options” shows some of the other options available to us.

Table 18.2. Additional IMAP voicemail options

OptionDescription
imapfolderProvides the name of the folder in which to store voicemail messages on your IMAP server. By default they are stored in the INBOX.[a]
imapgreetingsDefines whether voicemail greetings are stored on the IMAP server or stored locally on the server. Valid values are yes or no.
imapparentfolderDefines the parent folder on the IMAP server. Usually this configured as INBOX on the server, but if it is called something else, you can specify it here.
greetingfolderSpecifies the folder in which to save the voicemail greetings, if you’ve enabled the imapgreetings option by setting it to yes. By default greetings are saved in the INBOX.
authuserSpecifies the master user to use for connecting to your IMAP server, if the server is configured with a single user that has access to all mailboxes.
authpasswordComplement to the authuser directive. See authuser for more information.
opentimeoutSpecifies the TCP open timeout (in seconds).
closetimeoutSpecifies the TCP close timeout (in seconds).
readtimeoutSpecifies the TCP read timeout (in seconds).
writetimeoutSpecifies the TCP write timeout (in seconds).

[a] It is important to store your voicemail messages in a folder other than the INBOX if the number of messages contained in the INBOX could be rather large. Asterisk will try to gather information about all the emails contained in the INBOX, and could either time out before retrieving all the information or just take a very long time to store or retrieve voicemail messages, which is not desirable.


With our [general] section configured, let’s define a mailbox for connecting to the IMAP server.

In Chapter 8, Voicemail we defined some users in the [shifteight] voicemail context. Here is the original configuration as defined in that chapter:

[shifteight]
100 => 0107,Leif Madsen,leif@shifteight.org
101 => 0523,Jim VanMeggelen,jim@shifteight.org,,attach=no|maxmsg=100
102 => 11042,Tilghman Lesher,,,attach=no|tz=central

We’re going to modify mailbox 100 in such a way that it connects to the Gmail IMAP server to store and retrieve voicemail messages:

[shifteight]
100 => 0107,Leif Madsen,,,|imapuser=leif@shifteight.org|imappassword=secret

Note

The voicemail.conf file uses both commas and pipes as separators, depending on which field is being used. The first few fields have specific settings in them, and the last field can contain extra information about the mailbox, which is separated by the pipe character (|).

We’ve removed the email address from the third field because we’re not going to use sendmail to email us voicemails anymore: they are just going to be stored on the email server directly now. We’ve configured the mailbox to connect with the IMAP username of leif@shifteight.org (because we’ve enabled Google Apps for the domain that hosts our email) and are connecting using the IMAP password secret.

After configuring Asterisk, we need to reload the app_voicemail.so module. If you enable console debugging, you should see output similar to the following upon connection to the voicemail server:

*CLI> core set debug 10
*CLI> module reload app_voicemail.so
DEBUG[3293]: app_voicemail.c:2734 mm_log: IMAP Info: Trying IP address [74.125.53.109]
DEBUG[3293]: app_voicemail.c:2734 mm_log: IMAP Info: Gimap ready for requests 
             from 99.228.XXX.XXX 13if2973206wfc.0
DEBUG[3293]: app_voicemail.c:2757 mm_login: Entering callback mm_login
DEBUG[3293]: app_voicemail.c:2650 mm_exists: Entering EXISTS callback for message 7
DEBUG[3293]: app_voicemail.c:3074 set_update: User leif@shifteight.org mailbox set for 
update.
DEBUG[3293]: app_voicemail.c:2510 init_mailstream: Before mail_open, server: 
             {imap.gmail.com:993/imap/ssl/user=leif@shifteight.org}INBOX, box:0
DEBUG[3293]: app_voicemail.c:2734 mm_log: IMAP Info: Reusing connection to 
             gmail-imap.l.google.com/user="leif@shifteight.org"

If you get any ERRORs, check your configuration and verify that the IMAP library is compiled with SSL support. Once app_voicemail.so is connected, try leaving yourself a voicemail; then check your voicemail via the Gmail web interface and verify that your message is stored correctly. You should also have an MWI light on your device if it supports it, and if you’ve configured mailbox=100@shifteight for the device in sip.conf. If you load the voicemail message envelope and mark it as read, the MWI light should turn off within 30 seconds (or whatever value you set pollfreq to in voicemail.conf).



[158] When we say “it works,” what we mean is that Asterisk will compose the email and submit it to the MTA, and the email will successfully be passed out of the system. What happens to it after it leaves the system is a bit more complicated, and will often involve spam filters treating the mail as suspect and not actually delivering it. This is not really Asterisk’s fault, but it’s something you’ll have to deal with.

[159] We recently checked out the open source webmail project roundcube project as well, and we were quite impressed.