Competition among TV broadcasters and streaming services continues to grow. Viewers have more choice and are becoming more demanding. For providers, this means world-class competition in terms of content and the need for a customer experience on a par with the top media brands. The most efficient way to achieve a sustainable customer experience is through detailed testing during the development process as well as a well-thought-out Quality Assurance (QA). I wrote about the reasons within a byline for FKT Magazine, which I would also like to share with you.

When a football match in the German Bundesliga was shown exclusively on Eurosport for the first time in August 2017, subscribers were completely annoyed. However, the trigger was not the match on offer, but the fact that many of them were literally seeing black. The broadcaster had to contend with massive technical problems due to the high volume of viewers. It was not a good start for Eurosport as the rights holder for the live broadcast of a total of 45 Bundesliga matches per season at the time. And it's no wonder that the commitment has since ended again.

Quality assurance is the key to success

This case is an object lesson in how important a good customer experience (CX) is in video streaming. On the other hand, poor CX can hardly be compensated for by even the best content. The fact that Amazon, Apple, Disney, and even more players are actively participating in the "streaming wars," and they also have gigantic resources at their disposal, only makes a good CX even more important, especially for smaller players, in order to hold their own against the competition. There's a lot to gain here – but also a lot to lose.

Content playout via app is becoming increasingly complex across the entire value chain due to competition for content, greater diversity of technologies, and platform effects. The success of D2C offerings in particular, which distribute their content independently rather than via the major platforms, depends crucially on good QA and smooth CX.

Release testing is part of the development process

For viewers, a good customer experience is an absolutely key criterion. That's why it's important for providers to put their own service through its paces with a lot of love, to put it through its paces well before launch, and to eliminate errors. This starts as early as the development stage. Professional test teams should be involved in the entire software development cycle and test it using carefully selected test methods (static, dynamic, black box, white box, etc.) and types (e.g., functional, structure-related, requirements-related) at all stages (component tests, integration tests, system tests, and acceptance tests). In this way, they can provide early support in eliminating errors and resolving ambiguities. After all, one of the principles of testing is: the earlier the better! A particularly promising approach is to set up such a testing process for the long term, right up to the final release.

For the customer of a large OTT offering, TeraVolt was able to identify an average of 6.7 bugs per regression test in the digital products using pre-release acceptance and release testing. With thorough testing, efficiency and reliability (retention) can be ensured and bugs fixed before an application is launched. TeraVolt was able to identify almost 3,800 bugs in the last four years with the help of the development-accompanying testing at the aforementioned customer and thus drastically reduce the error rate before the customer. In addition, approximately 6,200 feature tickets were tested in detail during the project in acceptance testing before the release. This ensured that new modules of the application met the functional requirements of the product team as well as the expectations of the users. The successful combination of release and acceptance testing is also reflected in the app store ratings, which are now at a consistently high level for Android (4.3) and iOS (4.5), for example.

Quality control in the further course

But quality assurance before launch is not the end of the story. Even after launch, providers must stay on the ball. After all, errors can occur again and again during live operations and have a variety of causes: Platforms can implement new software, change backend structures and provider APIs, or adapt payment processes. Meanwhile, advertising providers optimize their advertising integrations and change their delivery processes. In addition, app releases bring additional features and new content, but also new sources of errors that can impair the customer experience – sometimes even in the form of total failures. To prevent this, it doesn't even take too much effort, but it does take a well-structured approach to software maintenance and support, as well as careful testing after further developments.

Live monitoring for short-term solutions

The final pillar of a promising QA portfolio is live monitoring. This gives providers the opportunity to have their products monitored live while the respective content is being broadcast and to be informed immediately of any faults or restrictions to the customer experience. This not only enables timely troubleshooting in the event of errors, but also provides a unique view of the product's performance.

The Arte Media Library case study.

TeraVolt's work for the German-French TV channel Arte began with an initial inventory of all devices (TVs, OTT, and satellite boxes); this involved testing basic functions and randomly checking content playback on around 60 devices. In the next step, a report recorded the errors and faults found. Based on the inventory, the current version of the app was blacklisted on devices on which it did not run at all or ran incorrectly. Instead, a legacy version was played out there, i.e., a version of the respective app for older devices. This allowed the developers to isolate the problem and fix it (partly via group conferencing and pair programming), so that the blacklisting could be lifted device by device and the app could go live again.

Kemal Görgülü, Chief Technology Officer at Arte, confirmed that the media library has improved enormously in terms of availability and user experience as a result of quality control, because the app is now continuously tested on almost all common devices, including older devices. Debugging, where appropriate, is done in direct collaboration between testers and developers, involving the affected devices. The corrections have led to a sharp decrease in reported errors.

Conclusion: Quality assurance with release testing and quality control are the be-all and end-all – before and after the launch.

Summarizing all these findings, it becomes clear that quality assurance is usually of very underestimated value when it comes to building and maintaining a TV app. This is the only way to ensure a lasting, high-quality customer experience - which in turn turns viewers into satisfied users. This makes it clear that the customer experience of streaming media services and TV media libraries is a key success factor alongside content. Users switch off if the content is not relevant and stop using apps that don't work. Providers who don't take this seriously enough run the risk of losing their viewers to the competition, which is getting bigger rather than smaller in the face of the “streaming wars”. This is exactly why quality assurance has never been more important in the TV segment.