Who's the most novel? - A quantitative comparison of the user experience of Netflix, Amazon Prime Video and Disney+

Sandra Roth

1.7.2021

If you've already read one of our other articles comparing Netflix, Amazon Prime Video, and Disney+, feel free to skip the following introduction and read straight on to the "Novelty" section.

The three established video-on-demand (VoD) providers Netflix, Amazon Prime Video and Disney+ all basically promise the same thing: watch whatever you like best at any time. And yet the implementation in the details and thus the success of the providers are different. Netflix currently leads the way with around 208 million subscribers worldwide, followed by Amazon Prime Video with strong growth in 2020 and currently around 150 million subscribers. Still quite new to the market, the provider Disney+ can count 100 million subscribers worldwide (as of 24.06.2021).

Do the numbers of subscribers also reflect the user experience with the product? Why do we like Netflix better than Amazon Prime Video or the other way around? Are there specific things we can name or is it simply the big picture that convinces us more with one provider than the other? Most of the time, we can't put it exactly into words, or it's just a couple of good experiences with the one product that inspires a positive feeling in us. That's why we tried to put into numbers what differentiates Netflix, Amazon Prime Video and Disney+ from each other and present our findings here.

To measure quantitatively, we used the User Experience Questionnaire + (UEQ+). The UEQ measures user experience based on 6 scales. Each scale has opposing word pairs that describe the experience (eg. complicated - simple). The UEQ+ is the modular version of the UEQ questionnaire and allows to choose from multiple scales. In this way, the survey can be better adapted to the respective product, since not all scales are suitable for all products.

The respondents were asked exclusively about their experience with one of the three providers in relation to TV usage. Thus, the results only reflect the experience of TV usage; mobile and PC usage are not included. We asked more than 60 respondents to rate the following five scales, 20 respondents per provider:

‍

Infographic with the 5 measured values: Aesthetics, Novelty, Trust, Dependability, Perspicuity

‍

The range of the scales is between -3 (terribly bad) and +3 (extremely good). Due to the calculation of mean values across different subjects with different opinions, it is extremely unlikely to observe values above +2 or below -2. This article describes the evaluation of the novelty scale. The provider Netflix scored best. Why is that? Feel free to read on.

‍

Novelty

"Users have the impression that the design of the product looks new, fresh and original, and thus attracts their attention."

In the category of novelty, Netflix consistently receives the highest scores (mean value: 1.24). Followed by Amazon Prime Video with a mean value of 0.74 and in last place Disney+ with a negative mean value of -0.29. The scale of the mean values ranges from -3 to 3.

‍

‍

A few words in advance to narrow down the topics of the novelty scale. Based on the statement "I find the product idea or the design of the product", the respondents were able to weigh up the following opposing word pairs: unimaginative - creative; conventional - original; conventional - novel; conservative - innovative. The positive expressions of the scale, creative, original, novel and innovative mean something new is being formed. An idea is developed, someone in the process of designing something, is creatively active. But it also means, something unknown occurs, the routine changes. Attention, there is a 'danger' of being surprised. And a surprise can produce both a positive and a negative emotional effect. For evolutionary reasons, humans are creatures of habit and look skeptically at the unknown. In this way, we make our everyday life easier by minimizing the cognitive effort required to accomplish daily tasks. In addition, skepticism and caution about new things helps us recognize danger. Nevertheless, it is equally evolutionarily important for our mind and our progress to break out of routine and engage with the unknown. This creates a tension between routine and variety. And this is also visible in the results of the dependability and novelty scale. Dependability, i.e. the feeling that users are always in control of the application, and novelty, i.e. the feeling that users experience something fresh and original that attracts their attention, are in contrast to each other. Returning to the topic of video-on-demand providers, new features and the content presentation of movies and series can have a positive effect on novelty if they surprise and inspire, and at the same time have a negative effect on dependability, since they can also create a feeling of loss of control. Where frequent variety and deviation from routine have a negative effect on the dependability scale, this is precisely what makes Netflix stand out so much on the novelty scale. Finding a balance here that inspires users with creative features and content without "scaring" them by deviating too much from routine is something Netflix manages to do quite convincingly.

When looking at Netflix, the creative preparation of the content is particularly striking. Varied images, film sequences as a glimpse into a movie or series instead of polished cover images, and the matching typography give the winner of this category a convincing and inspiring overall impression. Netflix frequently breaks out of the grid, brings movement into the movie and series offerings by rearranging the lanes or changing covers. When it comes to the content of the offerings, Netflix is oriented toward the current spirit of the times as well as the individual user. For example, there are changing categories according to which movies and series are sorted: "Movies with strong female leads," "Because you've seen 'Stromberg,' you might also like," or "Today's TopTen in Germany." In addition, Netflix regularly makes feature adjustments and introduces new functions. For example, a new arrangement of the selection of language and subtitles, which can be accessed without clicking in a submenu while watching. Or the addition of the movie collection in the submenu of a movie, with all the movies in a movie series. The impression rises that Netflix is really concerned with its content and its use, and shows attention to detail. For example, editorially created insights about successes, directorial work, awards or nominations can be found for films and series. If the open Netflix is not accessed for a longer period of time, the view switches to a screensaver mode with suggestions for various series and films that suit the user. Simple methods, but with the inclusion of personal preferences, convincing and inspiring.

Amazon Prime Video, on the other hand, makes an overall dry and not very emotional impression during use. Visuals and the use of appropriate typography and color are used sparingly. The usage feels somewhat static and ticked off. In order to learn more about a film or series users are redirected to a new page with a feeling of being interrupted, and there, with luck, they receive the information that helps them decide whether or not they want to see this movie. The content preparation of the offers has, instead of the users, mainly the subscription model in focus. The lanes sometimes have this in highlighted color in their headlines: "Included in Prime" or "Buy and Rent." That's informative, but not exactly creative. At this point, it's a good idea to bring up the topic of surprises again. Any planned positive surprise also requires knowledge about personal interests or tastes. In terms of personalization, however, you won't find many tangible moments on Amazon Prime Video. Surprising situations that capture the attention of users thus only arise by chance. The entire menu structure is reminiscent of a folder structure on the PC and therefore unfortunately looks a bit convoluted compared to Netflix. Overall, a monotonous, conventional, but also reliable image of the provider emerges, which the results of the survey reflect well.

And which other provider was included in the study? Oh yes, not even mentioned yet, because perhaps not worth mentioning in this scale? Disney+ unfortunately lands in the negative range with its values, and is therefore seen as conventional, traditional and conservative. Yet the Disney brand stands for exactly the opposite, is linked to experience, magic, creativity and inspiration. Imaginative worlds in which children never grow up, stories full of adventure and lovable characters, superheroes who save the universe from real villains. However, the last on the novelty scale prepares its content primarily in a clear, classic manner and, compared to Netflix, in a somewhat more neutral and cautious way. Compared to Amazon Prime Video, Disney+ uses typography, color and effective pictures, but scores worse. Disney+ also offers the same standards as the competition in terms of depth of information, control of content, and common VOD features. And yet it looks like the users are disappointed in terms of novelty. It seems that Disney+ does not manage to make users feel like 'Alice in Wonderland', but rather like 'Stephen in the living room'. And this disappointment due to unfulfilled expectations could be the reason for the poor performance. However, since Disney+ is still quite new on the market, hopes are high that the provider will continue to develop focused on a ''magical'' user experience.

‍

This blog article is part of the series "A quantitative comparison of the user experience of Netflix, Amazon Prime Video and Disney +". If you're interested in how the three providers scored on the other four scales, we'll link to the articles here.

Is dependability the determining factor?

The importance of aesthetics for UX

How perspicuous does a VOD provider have to be?

Who inspires the most trust?

‍