Email Newsletter Usability
Executive Summary

By Jakob Nielsen's Nielsen Norman Group

Summary

The most significant finding from our usability research on email newsletters is that users have highly emotional reactions to them. This is in strong contrast to research on website usability, where users are usually much more oriented toward functionality. Even a website that users visit daily seems to feel like a tool: users want to get in and get out as quickly as possible rather than “connect” with the site. Newsletters feel personal because they arrive in users’ inbox, and users have an ongoing relationship with them. In contrast, users tend to glance at websites when they need to accomplish something or to find the answer to a specific question. Newsletters also have a social aspect, as users often forward them to colleagues and friends. The positive emotional aspect of newsletters is that they can create much more of a bond between users and company than a website can. The negative aspect is that usability problems with newsletters have a much stronger impact on the customer relationship than website usability problems. For example, one user received an error message that read “Email address is not valid.” Even though this would be a poorly worded error message in any user interface, the emotional aspect to newsletters increased the user's anger: “Mine’s as valid as the next person’s! ... It's questioning my validity as an entity in cyberspace.”

Spam Is a Fact of Life

There’s a little good news, but mostly bad news about the impact of spam on email newsletters. The good news is that users in our most recent study were better able to differentiate legitimate opt-in newsletters from unsolicited messages than they could in the past. In our earlier newsletter usability studies, users sometimes confused the two. Now, spam has a very prominent profile in terms of popular awareness, press coverage, and the sheer amount of it hitting inboxes. Users have thus developed a reasonable understanding of the spam phenomenon as opposed to simply being baffled about unexpected messages. The bad news is that the increased burden on email users has caused people to become even more stressed and impatient when processing their inbox. Users have less tolerance for newsletters that waste their time. We have also found that users often use their spam filters as a shortcut to eliminating newsletters that they don’t want any more. Instead of unsubscribing, which users often view as too cumbersome, they simply tell their spam-blocker that the newsletter is spam. Voila, that newsletter no longer shows up in the inbox. The fact that many users will declare a newsletter to be spam when they tire of it has terrifying implications: legitimate newsletters might get blacklisted and thus be undeliverable to other subscribers who still welcome new issues. This is a compelling reason to increase the usability of the unsubscribe process: better to lose a subscriber than to be listed as spam.

User Research

To assess how people use email newsletters, we conducted two rounds of user studies, as well as a pilot study to refine the test methodology. In total, forty-five users participated in our testing. Most participants were in the United States (in twelve states across the country), but we also studied users in Australia, Hong Kong, Japan, Sweden, and the United Kingdom. The first study focused on testing newsletter usability in terms of subscribing, unsubscribing, and maintaining the user’s account. For this study, we systematically tested ten different newsletters, which we assigned to users to ensure that they hadn’t previously used a newsletter’s subscription interface. Most of the study was conducted as a traditional laboratory test, where users were observed individually as they read newsletters and attempted to subscribe and unsubscribe. Additional parts of the study were conducted remotely, through telephone calls. The second study was conducted remotely, using a diary methodology, which allowed us a much wider geographical distribution of participants. All of the international participants and all the U.S. users who were not on the east coast participated in the second study. The first study had the benefit of systematically testing a set of design variations, with multiple users for each design. However, it also had the distinct disadvantage of people getting newsletters that they had not selected themselves. For the second study, we looked in detail at users’ experience receiving and reading newsletters that they’d already subscribed to on their own initiative. In total, the participants subscribed to 345 different newsletters, but we studied only 101. Users’ newsletter experience was studied over a four-week period for most participants, and over two weeks in a few cases. This longitudinal approach allowed more emphasis on how people deal with incoming newsletters during their workday. We were also able to test many more B2B and intranet newsletters than could be covered in the first study, which mainly tested B2C newsletters. Of the newsletters received by the users in our second study, 65% were for personal purposes and 40% were for business purposes (5% of newsletters were seen as both personal and business, and so were counted twice).

High Nominal Usability

Our test users experienced unprecedentedly high levels of task completion in their attempts to subscribe and unsubscribe to the newsletters in the study: 78% for subscribing and 92% for unsubscribing. When you consider that most usability studies find success rates around 65% for other areas of Web design, the success rates for newsletter usability are incredibly high, even though they are still lower than anything we would deem to be a truly great user experience. Given these success rates, a newsletter with 50,000 subscribers could add an estimated 14,000 subscribers on average if everybody could operate its subscription interface correctly. There are probably two reasons for the fairly high success rates we found in the newsletter study. First, the functionality is very simple: Get on or off a mailing list. In fact, the main failures came on websites that had more complex functionality, such as combining newsletter subscriptions with site registration. In general, it is easier to design a simple user interface when the underlying functionality is simple. The second reason that the subscription process had much better usability than other Web designs is that newsletter designs are highly accountable. In many other areas of Web designs, project managers can delude themselves and their bosses that there are some benefits to be achieved from user-hostile designs such as splash pages. Create a design where people can’t find what they want, and page views may even go up as users wander aimlessly before they leave and give up doing business with the company. With a newsletter subscription design, users either subscribe or they don’t. In the latter case, websites will eventually tone down their design excesses and focus on simplicity, and subscriptions will increase accordingly. If a site were to replace a simple design with a complex one, it would soon notice a lack of new subscriptions and revert to the previous design, writing off the bad design as an expensive lesson in usability.

Low Perceived Usability

Even though users were able to successfully unsubscribe 92% of the time during the test sessions, they often refrained from even trying to get off mailing lists that they didn’t want any more. The four main reasons people didn’t attempt to unsubscribe were:

• Emotional attachment to the newsletter: Users said that it didn’t feel good to sever the relationship, even when they no longer read the mailings.
• Low expectations for the website’s usability: People assumed that it would be difficult and time-consuming to unsubscribe, so they postponed the job for another day and simply deleted the newsletter’s current issue.
• Fear that unsubscribing would not work and would subject the user to even more mail: Many people have heard that asking to get off spam lists only confirms the validity of their email address to the spammers; this notion has become an urban legend that contaminates users’ mental model of legitimate newsletter publishers as well.
• It’s often easier to simply use a spam-blocking feature to stop future issues than it is to unsubscribe. Whatever the reasons, it is clear that mailing list owners should not assume that all subscribers actually want to receive the newsletter. Many users may simply have neglected to unsubscribe for one of the above reasons. Some newsletters deliberately make it difficult to unsubscribe by hiding the instructions or making them overly complex. The motive is probably to retain as many subscribers as possible to maximize the reach of permission marketing programs. But, in reality, you don’t have users’ “permission” once they stop wanting the newsletter, regardless of whether they jump through the hoops required to get off the list. If users keep getting unwanted newsletters, the messages will start to backfire and become regular reminders that they’re annoyed with your company. Better to let them go.
  
  Speed Matters
  
  On average across the newsletters we studied, the subscribe process took five minutes, and the unsubscribe process took three minutes. Even though these task times are not prohibitive, they’re much too long for the simple functionality involved. We recommend setting a usability goal of allowing an existing user to unsubscribe in less than one minute, assuming that the user has a recent copy of the newsletter at hand. New subscriptions should also take less than a minute for newsletters that don’t require any information beyond the user's email address. Even if additional information is required, users should be able to subscribe to free newsletters in less than two minutes. Only newsletters that involve a subscription fee should be allowed so many steps to their subscription process that the average user can’t subscribe in two minutes. Users are very demanding with respect to the efficiency of operations like subscribing or unsubscribing. For both tasks, we found extremely strong correlations between the task time and the users’ subjective satisfaction: r = -.71 and -.96, respectively. These correlations basically say that the slower the subscribe or unsubscribe process, the less people will like the site. For each additional minute it takes to subscribe, you will lose 0.3 satisfaction points on a 1 to 7 scale, and for each additional minute it takes to unsubscribe, you will lose 0.6 satisfaction points. As indicated by the numbers, users are substantially more critical of a slow unsubscribe process. Once they want out, they want out quickly. A perfect satisfaction rating of 7 would require instantaneous task performance according to the regression estimates. It seems impossible to create a design that allows users to subscribe and unsubscribe in zero seconds, but that’s ultimately what users want. It's nobody’s goal in life to “manage your subscriptions,” so any overhead becomes an annoyance. Extreme simplicity and ease of use are necessary to make a positive impact on customers.
  
  Significant Platform Diversity
  
  The Web is getting to feel like a science-fictional World of Clones with virtually no biodiversity, as almost all users have the same browser these days. It’s still good practice to check that Web designs work on additional platforms beyond Internet Explorer and Windows, but the vast majority of users do use this combination. In any case, from an interaction design perspective, Netscape is a pale imitation of IE, and Macintosh is a higher-priced dolled-up variant of Windows. The differences between Web browsing platforms are like those between Indian and African elephants, not like the differences between crabs and eagles. In contrast, email newsletters have to contend with platform diversity that is much more like the biodiversity of the Cretaceous Period (before the comet hit). Microsoft Outlook was the most commonly used email reader in our second study, accounting for 42% of the users, but seven additional platforms were represented, with Yahoo! Mail, Hotmail, AOL, and Outlook Express all having more than 10% of the user base. It is also common to find people using Eudora, Lotus Notes, and a broad variety of mainframe systems and variants of Unix mail. Each email platform has a different way of displaying the From line, the Subject line, and the newsletter content, as well as different approaches to spam filtering and other ways of influencing the newsletter subscriber’s user experience. This diversity makes it very important for newsletter designers to consider the many different platforms and test their subscribe and unsubscribe processes as well as the delivery and display of the actual newsletter on all major email platforms.
  
  Scannability and Immediate Utility
  
  People get a lot of email. They don't have time to read a lot of text. The most frequent complaint in our study was about newsletters that arrived too often. And, when we let them vent, the most frequent advice our study participants had for newsletter creators was to “keep it brief.” Newsletters must be designed to facilitate scanning. In our first study, 23% of the newsletters were read thoroughly. In our second study, two years later, only 11% of the newsletters were read thoroughly. The drop in percentage of thoroughly read newsletters is a good indication of the increased volume of email users have to process. The dominant mode of dealing with email newsletters is to skim them: that’s what happened to 57% of the newsletters in our second study. Remaining newsletters were either never read (22%) or saved for possible later reading (10%). Of course, many of the newsletters that users “save” will never actually be read: once they scroll below the visible area of the inbox, they may never be seen again. Sometimes users will simply skim the headlines to get an update or overview of what’s going on in the field covered by the newsletter. As one user said, “I like to keep up to date in the industry, but rarely delve deeper than the cover page.” Other times, users deliberately pick out those few elements that are most important to them and ignore the rest. As another user said, “I review the contents by company and only read the companies of interest to me.” Designing for users who scan rather than read is essential for a newsletter’s survival. Scannability is important for websites as well, but it’s about 50% more important for newsletters. This implies the need for layouts that let users quickly grasp each issue’s content and zero in on specifics. Content and writing styles must support users who read only part of the material. Newsletters must be current and timely, as indicated by three of the four main reasons that users listed for why a certain newsletter was the most valuable they received. All of the following four reasons were given by more than 40% of users:
• Informs of work-related news and/or my own company’s or other companies’ actions (mentioned by two-thirds of users)
• Reports prices/sales
• Informs about personal interests/hobbies
• Informs about events/deadlines/important dates There is pretty much a “what have you done for me lately” phenomenon at play, where newsletters have to justify their space in the inbox on a daily basis. Having been relevant in the past is not enough. Because of the immediacy of the medium, newsletters must be relevant today and address the user’s specific needs in the moment. Because newsletters build relationships with readers and because it’s so easy to ignore individual issues, newsletters do get some leeway if they are predictably relevant at certain times. During those periods where a newsletter isn’t relevant to the user’s immediate needs, the user might simply ignore it for a while instead of unsubscribing. For example, a speech pathologist at an elementary school said that she could only purchase new products at the end of the school year, and so ignored product-related newsletters most of the year. Still, she didn’t unsubscribe, and simply receiving the sales newsletters reminded her of the brand when she received her budget. Users will often avoid signing up for newsletters because they feel crushed by information overload. It is the job of the newsletter publisher to convince users that the newsletter will be simple, useful, and easy to deal with. A predictable publication frequency that is not too aggressive is usually best, except for newsletters that report breaking news. Not only are users more likely to sign up for newsletters that feel less intimidating, but a regular publication schedule lets users know when to look for the newsletter and reduces the probability that it will be deleted because it is confused with spam. Writing good subject lines is especially important, both to encourage users to open the newsletter and to distinguish the newsletter from spam. We recommend including actual content from the issue in each subject line, even though it's a difficult job to write good microcontent within the 50 to 60 character limit that many email services impose.
  
  Future of Email Newsletters
  
  Two years ago, in our first report about newsletter usability, we said about the future of email newsletters: “There may be none. Legitimate use of email is at war with spam, and spam may be winning.”Although two years is a very short period in which to assess big trends, we now believe that this assessment was too negative. Email newsletters are so powerful that the best of them do have a future, despite ever-more adverse conditions. Ever-increasing information overload is definitely making users reluctant to sign up for more email. And once newsletters arrive in the user’s inbox, they might simply be deleted as part of a ruthless mass deletion procedure aimed at the morning’s spam. Finally, as discussed above, fear of spam and other email abuse is keeping users from dealing rationally with newsletter subscriptions. When we asked users about the benefits of email newsletters, more than one-third of them highlighted the following three reasons:
• Email newsletters are informative and keep users up to date (mentioned by two-thirds of the users).
• Email newsletters are convenient and are delivered straight to the user’s information central; they then require no further action beyond a simple click.
• Email newsletters have timely information and real-time delivery. Newsletters that leverage these advantages (along with other points that users mentioned) have a stable future. They need to give users specific benefits that help them with life or work issues in the here and now. Comparing email newsletters with other media, one user said: “Bottom line, I’d rather have it in an email newsletter than in the regular mail. I can click Delete if I don’t want it; I don’t have to throw anything away; and it is usually easier to unsubscribe if you don’t want to get anymore.” Convenience rules.
  
  

Return to main page of:
Email Newsletter Usability:127 design guidelines for subscription interfaces & newsletter content based on user research