Network Working Group Greg Vaudreuil Internet Draft Lucent Technologies Expires in six months February 21, 2002 Messaging profile for telephone-based Messaging clients Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. This document is an Internet Draft. Internet Drafts are working documents of the Internet Engineering Task Force (IETF), its Areas, and its Working Groups. Note that other groups may also distribute working documents as Internet Drafts. Internet Drafts are valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet Drafts as reference material or to cite them other than as a "work in progress". To learn the current status of any Internet-Draft, please check the "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or ftp.isi.edu (US West Coast). Abstract This document discussed issues and requirements for a profile of the IMAP 4, message submission, and notification protocols for use by telephone user interfaces delivering the traditional voice mail user experience. Experience with IMAP 4 and voice mail systems has shown quite a few limitations that may be addressed by protocol extensions and standardized conventions between clients and servers. Copyright Notice Copyright (C) The Internet Society (2002). All Rights Reserved. Internet Draft IMAP Profile for Voice February 20, 2002 Table of Contents 1. OVERVIEW..........................................................3 2. IMAP ISSUES.......................................................4 2.1 Performance Issues ..............................................4 2.2 Functional Limitations ..........................................5 3. MESSAGE SUBMISSION AND MESSAGE DELIVERY ISSUES....................8 3.1 Server-assisted user message forwarding .........................8 3.2 Quota-by-context ................................................9 3.3 Future Delivery .................................................9 4. NOTIFICATION......................................................9 4.1 Notification Support ............................................9 5. ACKNOWLEDGMENTS..................................................12 6. AUTHORS' ADDRESSES...............................................12 7. FULL COPYRIGHT NOTICE............................................12 Vaudreuil Expires August 20, 2002 [Page 2] Internet Draft IMAP Profile for Voice February 20, 2002 1. Overview In the open standards arena, IMAP 4 is the protocol most commonly identified for use by a TUI for access to messages in a voice mailbox. IMAP4 was developed primarily as an access mechanism to text-based email and has evolved in many ways to support multi-media content. The protocol is extensible and a large number of extensions have been made or are needed to full support the TUI experience of voice messaging. SMTP used for the transmission of voice messages from a telephone answering application needs to be optimized through use of various extensions and new conventions to perform at a level required for a large distributed system. Greeting and spoken name play needs to be provided, either through LDAP, LDAP extensions, or pointers to alternate streaming play mechanisms. IMAP4 limitations fall into two broad categories, performance and functionality. Most of these can be addressed by specific implementation choices, standards extensions, and standardized conventions shared between the message store and the client. Many limitations of commercial message store solutions are not IMAP protocol issues, but rather underlying logic necessary to support the voice mail user experience. Lucent's proprietary message server supports these functions today and will continue to support these mechanisms when delivered over standard protocols. Vaudreuil Expires August 20, 2002 [Page 3] Internet Draft IMAP Profile for Voice February 20, 2002 2. IMAP Issues 2.1 Performance Issues 2.1.1 Real-time playback The IMAP protocol itself does not provide streaming in the strict definition of the term. It does provide for the incremental download of content in blocks. Most IMAP clients do not support this behavior and instead download the entire contents into a temporary file to be passed to the application. There are several approaches to achieve real-time playback. The first approach is to implement an IMAP client that can pass data incrementally to the application as it is received from the network. The application can then read bytes from the network as needed to maintain a play buffer and not require the full download of contents. This approach may require server-side development to efficiently support partial download. (Avoid re-opening file and seeking to requested pointer) Alternately, the proposed IMAP channel extension can be used by the client to request that the servers make the selected content available via an alternate transport mechanism. In this way a client can ask the server to make the voice data available to the client via a streaming media protocol such as RTSP. This requires support on the client and server of a common streaming protocol. Third, given sufficient bandwidth between client and back-end data store, an implementation may download the entire contents before playing without noticeable latency. Combined with client-side caching to avoid re-fetches, this strategy may work with existing message servers. It is clear that a choice be made common to the server and the client to provide this functionality. 2.1.2 Data Size Inflation due to Text Based Transport Standard IMAP4 uses a text-based data representation scheme where all data is represented in a form that looks like text, that is, voice data must be encoded using "base 64" into a transport encoding that adds 30% to the size of a message. When downloading or appending messages to the server, substantial additional bandwidth is utilized. Proposed Solutions Vaudreuil Expires August 20, 2002 [Page 4] Internet Draft IMAP Profile for Voice February 20, 2002 Where IMAP channel is appropriate, the external channel may be binary capable; that is; the external access may not require re-encoding. Such mechanisms as HTTP, FTP, or RTSP are available for this download. The IMAP binary extension standards proposal extends the IMAP fetch command to retrieve data in the binary form. This is especially useful for large attachments and other binary components. Binary in conjunction with a streaming client implementation may be an attractive alternative to the Channel extension. 2.2 Functional Limitations In addition to performance challenges, there are a number of functional deficiencies in the IMAP protocol. 2.2.1 Mailbox Summary (Per-context counters in mailbox status command) The common TUI prompt "you have two new voice messages, six unheard messages, and one new fax message" requires more information than is made available by IMAP. The IMAP status command provides a count of new and total messages with standardized attributes extracted from the message headers. This pre-determined information does not include information about the message type. Without defined conventions to the status command, a client would have to download the header for each message to determine its type. (Are flags made available through the list command? Or do they need to be queried independently?) In standards-land, there is an effort do define an extensible mechanism for sending arbitrary message summary data in the LIST command. With proper MTA support for the necessary message-context attribute and support for reading the flags, a single command can extract enough data to count the messages in various categories. For adequate performance, the MTA must pre-parse the messages to extract the necessary information into an index for this request as messages are deposited. Without the standards, it is possible to use multiple virtual folders to achieve similar functionality. The "inbox" can be sub-divided into virtual unheard, new, unheard fax, and new fax message sub-folders. A folder status command issued against each sub-folder would yield the appropriate count. An MTA may implement these as truly separate folders or may present these as virtual folders to the client while storing the messages in a single inbox. 2.2.2 Sort by message context Voice mailboxes commonly present new voice messages first and then new fax messages within a single logical queue. This requires the ability to sort the messages by message context. Vaudreuil Expires August 20, 2002 [Page 5] Internet Draft IMAP Profile for Voice February 20, 2002 In standards-land, there is an effort do define an extensible sort mechanism for sorting on arbitrary header contents. An alternative is for the client to download the headers of all messages and perform a local sort. This works where mailboxes have fewer than a couple-dozen messages. A further alternative uses separate virtual folders to hold messages of different types and status, using the client to construct the expected user experience. 2.2.3 QUOTA by message context Voice mail systems' mailboxes commonly contain voice and fax messages. Sometimes, such systems also support E-mail messages (text, text with attachments) in addition to voice messages. Similarly to the requirement for sort by message context - - quota management is also required per message context. One possible use case is to prevent multiple (large) messages of one type (e.g. E-mail messages) to consume all available quota so that messages of other type (e.g. voice or fax messages) cannot be further deposited to the mailbox. In standards-land, there is an effort to define an extension to the QUOTA IMAP command to support multiple message contexts. (In addition, there is a parallel effort to enhance the SMTP SIZE extension for the same purpose). 2.2.4 Status of multiple mailboxes Extension mailbox support requires the ability to efficiently status a mailbox other than the one currently logged into. This facility is required to support submailboxes, where a common feature is to check whether other submailboxes in the same family group have new messages. Current mechanisms are limited to logging into each of set of mailboxes, checking status, logging out, and repeating until all submailboxes are statused. 2.2.5 Outbox, Sent Items, Delete Items, Expired items, Drafts IMAP provides only a single standardized folder inbox. Applications that provide features such as check receipt, deleted message recovery, resave, and others require the ability to access messages in pre- determined mailboxes with specific behaviors. This functionality does not require new protocol per-se, but standardized usage and naming conventions necessary for interoperability. It required that the server Vaudreuil Expires August 20, 2002 [Page 6] Internet Draft IMAP Profile for Voice February 20, 2002 provide the underlying logic to support these special folders including automatic insertion, scheduled copying, and periodic deletion. 2.2.6 CLID Restriction / Disclosure authorization Many calling features are dependent upon collected caller-ID information. Trusted clients such as the TUI, WUI, and WAP may have access to restricted caller-ID information for such purposes as callback. Untrusted clients must not receive this information. A mechanism for communicating "trust" between the client and the server is required to deliver this information to the end-user when appropriate. Some IMAP 4 servers can be configured to recognize certain clients by IP address and apply necessary CLID restriction treatment such as hiding addresses where CLI restriction has been indicated in the message. For systems operating in a closed network, the system may rely upon serving only trusted clients that protect the identity of the sender based on per-message markings. This usage requires custom proxies to use for Internet-facing services such as downloads by PC thick-clients. 2.2.7 Greeting Access and Play Voice messaging systems store multiple audio greetings files per user to play upon answering the phone. This information is logically directory information, but the size, access patterns, and streaming requirements exceed the capabilities of more directory access protocols and underlying servers. Rather than create a new specialized store, it is common to store greetings as messages in a hidden or special folder. As such, it is natural to use the IMAP protocol for access and update of these greetings. This provides the ability to update the greeting easily using the IMAP command. Access to the greetings requires a form of super-user access to log into the called party mailbox and extract the greeting. Through conventions, a given server or set of servers identified by IP address or login password may be granted privileged access to the mailbox contents. 2.2.8 Atomic Commit of Telephone Answering Messages into Inbox Voice messaging service has provided a high degree of reliability and performance for telephone answering messages. The expectation is that once the caller has hung-up, the messages is in the mailbox and available for review. Traditional Internet mail architecture suggests these messages should be sent to the mailbox via SMTP. This approach has two limitations. The first and most manageable is that the message forwarding may take more time than is tolerable by the subscriber. The second is that the message may fail to be delivered to the mailbox, and Vaudreuil Expires August 20, 2002 [Page 7] Internet Draft IMAP Profile for Voice February 20, 2002 because there is no way to return notice to the caller, the message is "lost". The standards community is working on an alliterative to SMTP called Local Message Transport Protocol. This protocol addresses a number of limitations in SMTP when used to provide atomic delivery to a mailbox. The failure modes in this proposal are carefully controlled, as are issues of per-message quota enforcement and message storage quota- override for designated administrative messages. An alternative approach is to mis-use the IMAP protocol slightly and use an IMAP append command to deposit a message directly into the subscriber's inbox. This append must be done by a special super-user with write permissions into the subscribers mailbox. Further, the message store must be able to trigger notification events upon insertion of a message into the mailbox via the Append command. The historic limitation on using IMAP4 for message sending involves the inability of IMAP to communicate a full SMTP envelope. For telephone answering, these limitations are not significant. 2.2.9 Multiple Access to Mailbox If the telephone answering application client uses IMAP4 for greeting access and message deposit, it is essential that the server provide support for simultaneous login. It is common in VoiceMail for an incoming call to be serviced by the telephone answering application client at the same time the subscribers is logged into their mailbox. Further, new applications such as WEB and WAP access to voicemail may entail simultaneous login sessions, one from the TUI client and one from the visual client. The standard does not preclude multiple accesses to a mailbox, but it does not explicitly support this practice. The lack of explicit support requires the server and client to adhere to a common set of practices and behaviors to avoid undesirable and unpredictable behaviors. RFC 2180 describes a candidate set of conventions necessary to support this multiple-access technique. It is not a standard. 3. Message Submission and Message Delivery Issues 3.1 Server-assisted user message forwarding It is common to forward messages, or to reply to messages with a copy of the attached content. Today such forwarding requires the sender to download a complete copy of the original message, attach it to the reply or forward message, and resubmit the result. For large messages, this represents a substantial amount of bandwidth and processing. For clients connected via long-thin pipes, alternatives are required. Vaudreuil Expires August 20, 2002 [Page 8] Internet Draft IMAP Profile for Voice February 20, 2002 One approach is to define an extension to message submission to request the submission server to resolve embedded URL's within a message before relaying the message to the final destination. 3.2 Quota-by-context It is common in a unified messaging system to offer separate quota's for each of several message contexts to avoid the condition where a flood of email fills the mailbox and prevents the subscriber from receiving voice messages via the telephone. It is necessary to extend the protocols to support the reporting of the "mailbox full" status based on the context of the submitted message. Clear security issues are involved to prevent the mis-identification of a message context for the purpose of intentionally filling a subscribers mailbox. It is envisioned that the message submission protocol will support authentication of trusted submission agents authorized to submit distinguished messages. 3.3 Future Delivery Traditionally messages sent with "future delivery" are held in the recipients client "outbox" or equivalent until the appointed submission time. Think clients used for WEB or TUI do not have such persistent storage and must rely upon server-based outbox queues. Such support requires extensions to message submission protocols to identify a message as requiring queuing for future delivery. Extensions to IMAP4 or conventions are required to view and manipulate the outbound queue, for such purposes as cancelling a future message. Server support for managing such a queue is required such that message are sent when they are intended. 4. Notification 4.1 Notification Support Voicemail systems traditionally notify subscribers on certain events happening in their mailbox. For example, it is common to send an SMS, or a pager notification for new message arrival, when messages have been read (and are not considered "new" anymore), mailbox full etc. When implemented over IMAP-based message stores, voice mail system need to be notified about these events. Furthermore, when other applications are accessing/manipulating the mailbox, it is desirable that a notification component (which is sometimes part of the voicemail Vaudreuil Expires August 20, 2002 [Page 9] Internet Draft IMAP Profile for Voice February 20, 2002 application) gets notifications from the message store about these events, so that it can produce the desired user notification. The standards community is working on a standard for "Simple Notification and Alarm Protocol (SNAP)" that defines the expected behavior of the message store for various events, much of them triggered by IMAP commands. Vaudreuil Expires August 20, 2002 [Page 10] Internet Draft IMAP Profile for Voice February 20, 2002 4.1.1 References [VPIM2] Vaudreuil, Greg, Parsons, Glenn, "Voice Profile for Internet Mail, Version 2", RFC 2421, September 1998. "IMAP4 Binary Content Extension", 01/18/2002, "IMAP4 Channel Transport Mechanism", 11/27/2001, "LMTP Service Extension for Ignoring Recipient Quotas", 08/30/2001, "Message Context for Internet Mail", 06/05/2001, "Simple Notification and Alarm Protocol (SNAP)", 12/20/2001 RFC 2476, Message Submission. R. Gellens, J. Klensin. December 1998. (Status: PROPOSED STANDARD) RFC 2033, Local Mail Transfer Protocol. J. Myers. October 1996. (Status: INFORMATIONAL) RFC 2086, IMAP4 ACL extension. J. Myers. January 1997. (Status: PROPOSED STANDARD) RFC 2087 IMAP4 QUOTA extension. J. Myers. January 1997. (Status: PROPOSED STANDARD) RFC 2180, IMAP4 Multi-Accessed Mailbox Practice. M. Gahrns. July 1997. (Status: INFORMATIONAL) RFC 2221, IMAP4 Login Referrals. M. Gahrns. October 1997. (Status: PROPOSED STANDARD) Vaudreuil Expires August 20, 2002 [Page 11] Internet Draft IMAP Profile for Voice February 20, 2002 5. Acknowledgments Ari Erev and Naom Shapira have contributed substantial requirements to this document. 6. Authors' Addresses Gregory M. Vaudreuil Lucent Technologies 7291 Williamson Rd Dallas, TX 75214 United States Phone/Fax: +1-972-733-2722 Email: GregV@ieee.org 7. Full Copyright Notice "Copyright (C) The Internet Society (2002). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE." Vaudreuil Expires August 20, 2002 [Page 12]