Adaptive bitrate streaming

Gepubliceerd op Auteur: koenarchief
Bron: Wikipedia

Adaptive streaming overview

Adaptive streaming in action

Adaptive bitrate streaming is a technique used in streaming multimedia over computer networks. While in the past most video streaming technologies utilized streaming protocols such as RTP with RTSP, today’s adaptive streaming technologies are almost exclusively based on HTTP[1] and designed to work efficiently over large distributed HTTP networks such as the Internet.

It works by detecting a user’s bandwidth and CPU capacity in real time and adjusting the quality of a video stream accordingly. It requires the use of an encoder which can encode a single source video at multiple bit rates. The player client[2] switches between streaming the different encodings depending on available resources.[3] “The result: very little buffering, fast start time and a good experience for both high-end and low-end connections.”[4]

More specifically, and as the implementations in use today are, adaptive bitrate streaming is a method of video streaming over HTTP where the source content is encoded at multiple bit rates, then each of the different bit rate streams are segmented into small multi-second parts.[5] The streaming client is made aware of the available streams at differing bit rates, and segments of the streams by a manifest file. When starting, the client requests the segments from the lowest bit rate stream. If the client finds the download speed is greater than the bit rate of the segment downloaded, then it will request the next higher bit rate segments. Later, if the client finds the download speed for a segment is lower than the bit rate for the segment, and therefore the network throughput has deteriorated, then it will request a lower bit rate segment. The segment size can vary depending on the particular implementation, but they are typically between two (2) and ten (10) seconds.[3][5]

Current uses

Post-production houses, content delivery networks and studios use adaptive bit rate technology in order to provide consumers with higher quality video using less manpower and fewer resources. The creation of multiple video outputs, particularly for adaptive bit rate streaming, adds great value to consumers.[6] If the technology is working properly, the end user or consumer’s content should play back without interruption and potentially go unnoticed. Media companies have been actively using adaptive bit rate technology for many years now and it has essentially become standard practice for high-end streaming providers; permitting little buffering when streaming high-resolution feeds (begins with low-resolution and climbs).

Benefits of adaptive bitrate streaming

Traditional server-driven adaptive bitrate streaming provides consumers of streaming media with the best-possible experience, since the media server automatically adapts to any changes in each user’s network and playback conditions. The media and entertainment industry also benefit from adaptive bitrate streaming. As the video space grows, content delivery networks and video providers can provide customers with a superior viewing experience. Adaptive bitrate technology requires additional encoding, but simplifies the overall workflow and creates better results.

HTTP-based adaptive bitrate streaming technologies yield additional benefits over traditional server-driven adaptive bitrate streaming. First, since the streaming technology is built on top of HTTP, contrary to RTP-based adaptive streaming, the packets have no difficulties traversing firewall and NAT devices. Second, since HTTP streaming is purely client-driven, all adaptation logic resides at the client. This reduces the requirement of persistent connections between server and client application. Furthermore, the server is not required to maintain session state information on each client, increasing scalability. Finally, existing HTTP delivery infrastructure, such as HTTP caches and servers can be seamlessly adopted.[7][8][9][10]

A scalable CDN is used to deliver media streaming to an Internet audience. The CDN receives the stream from the source at its Origin server, then replicates it to many or all of its Edge cache servers. The end-user requests the stream and is redirected to the “closest” Edge server. This can be tested using libdash[11] and the Distributed DASH (D-DASH) dataset,[12] which has several mirrors across Europe, Asia and the US. The use of HTTP-based adaptive streaming allows the Edge server to run a simple HTTP server software, whose licence cost is cheap or free, reducing software licensing cost, compared to costly media server licences (e.g. Adobe Flash Media Streaming Server). The CDN cost for HTTP streaming media is then similar to HTTP web caching CDN cost.

History

Adaptive bit rate over HTTP was created by the DVD Forum at the WG1 Special Streaming group in October 2002. The group was co-chaired by Toshiba and Phoenix Technologies, The expert group count with the collaboration of Microsoft, Apple Computer, DTS Inc., Warner Brothers, 20th Century Fox, Digital Deluxe, Disney, Macromedia and Akamai.[dubious – discuss][citation needed] The technology was originally called DVDoverIP and was an integral effort of the DVD ENAV book.[13] The concept came from storing MPEG-1 and MPEG-2 DVD TS Sectors into small 2KB files, which will be served using an HTTP server to the player. The MPEG-1 segments provided the lower bandwidth stream, while the MPEG-2 provided a higher bit rate stream. The original XML schema provided a simple playlist of bit rates, languages and url servers. The first working prototype was presented to the DVD Forum by Phoenix Technologies at the Harman Kardon Lab in Villingen Germany.[citation needed]

Implementations

Adaptive bit rate streaming was introduced by Move Networks and is now being developed and utilized by Adobe Systems, Apple, Microsoft and Octoshape.[14] In October 2010, Move Networks was awarded a patent for their adaptive bit rate streaming (US patent number 7818444).[15]

MPEG-DASH

MPEG-DASH is the only adaptive bit-rate HTTP-based streaming solution that is an international standard[16] MPEG-DASH technology was developed under MPEG. Work on DASH started in 2010; it became a Draft International Standard in January 2011, and an International Standard in November 2011.[16][17][18] The MPEG-DASH standard was published as ISO/IEC 23009-1:2012 in April, 2012.

MPEG-DASH is a technology related to Adobe Systems HTTP Dynamic Streaming, Apple Inc. HTTP Live Streaming (HLS) and Microsoft Smooth Streaming.[19] DASH is based on Adaptive HTTP streaming (AHS) in 3GPP Release 9 and on HTTP Adaptive Streaming (HAS) in Open IPTV Forum Release 2.[20] As part of their collaboration with MPEG, 3GPP Release 10 has adopted DASH (with specific codecs and operating modes) for use over wireless networks.[20]

Standardizing an adaptive streaming solution is meant to provide confidence to the market that the solution can be adopted for universal deployment, compared to similar but more vendor-centric solutions such as HLS by Apple, Smooth Streaming by Microsoft, or HDS by Adobe.

Available implementations are the HTML5-based bitdash MPEG-DASH player[21] as well as the open source C++-based DASH client access library libdash of bitmovin GmbH,[11] the DASH tools of the Institute of Information Technology (ITEC) at Alpen-Adria University Klagenfurt,[2][22] the multimedia framework of the GPAC group at Telecom ParisTech,[23] and the dash.js[24] player of the DASH-IF.

Adobe HTTP Dynamic Streaming

“HTTP Dynamic streaming is the process of efficiently delivering streaming video to users by dynamically switching among different streams of varying quality and size during playback. This provides users with the best possible viewing experience their bandwidth and local computer hardware (CPU) can support. Another major goal of dynamic streaming is to make this process smooth and seamless to users, so that if up-scaling or down-scaling the quality of the stream is necessary, it is a smooth and nearly unnoticeable switch without disrupting the continuous playback.”[25]

The latest versions of Flash Player and Flash Media Server support adaptive bit-rate streaming over the traditional RTMP protocol, as well as HTTP, similar to the HTTP-based solutions from Apple and Microsoft,[26] HTTP dynamic streaming being supported in Flash Player 10.1 and later.[27] HTTP-based streaming has the advantage of not requiring any firewall ports being opened outside of the normal ports used by web browsers. HTTP-based streaming also allows video fragments to be cached by browsers, proxies, and CDNs, drastically reducing the load on the source server.

Apple HTTP Live Streaming

Main article: HTTP Live Streaming

HTTP Live Streaming (HLS) is an HTTP-based media streaming communications protocol implemented by Apple Inc. as part of QuickTime X and iOS. HLS supports both live and Video on demand content. It works by breaking down streams or video assets into several small MPEG2-TS files (video chunks) of varying bit rates and set duration using a stream or file segmenter. One such segmenter implementation is provided by Apple.[28] The segmenter is also responsible for producing a set of index files in the M3U8 format which acts as a playlist file for the video chunks. Each playlist pertains to a given bitrate level, and contains the relative or absolute URLs to the chunks with the relevant bitrate. The client is then responsible for requesting the appropriate playlist depending on the available bandwidth.

HTTP Live Streaming is a standard feature in the iPhone 3.0 and newer versions.[29]

Apple has submitted its solution to the IETF for consideration as an Informational Request For Comments.[30] A number of proprietary and open source solutions exist for both the server implementation (segmenter) and the client player.

HLS streams can be identified by the playlist URL format extension of .m3u8. These adaptive streams can be made available in many different bitrates and the client device interacts with the server to obtain the best available bitrate which can reliably be delivered. The client devices range from iPad, iPhones, Set Top Boxes (STBs) and other suitable client devices.

Playback of HLS is only natively supported in Safari on iOS and Mac and Microsoft Edge on Windows 10. Solutions for playback of HLS on other platforms mostly rely on third-party plug-ins such as Flash or QuickTime.

Microsoft Smooth Streaming

Smooth Streaming is an IIS Media Services extension that enables adaptive streaming of media to clients over HTTP.[31] The format specification is based on the ISO base media file format and standardized by Microsoft as the Protected Interoperable File Format.[32] Microsoft is actively involved with 3GPP, MPEG and DECE organizations’ efforts to standardize adaptive bit-rate HTTP streaming. Microsoft provides Smooth Streaming Client software development kits for Silverlight and Windows Phone 7, as well as a Smooth Streaming Porting Kit that can be used for other client operating systems, such as Apple iOS, Android, and Linux.[33] IIS Media Services 4.0, released in November 2010, introduced a feature which enables Live Smooth Streaming H.264/AAC videos to be dynamically repackaged into the Apple HTTP Adaptive Streaming format and delivered to iOS devices without the need for re-encoding. Microsoft has successfully demonstrated delivery of both live and on-demand 1080p HD video with Smooth Streaming to Silverlight clients. In 2010, Microsoft also partnered with NVIDIA to demonstrate live streaming of 1080p stereoscopic 3D video to PCs equipped with NVIDIA 3D Vision technology.[34]

QuavStreams Adaptive Streaming over HTTP

QuavStreams Adaptive Streaming is a multimedia streaming technology developed by Quavlive. The streaming server is an HTTP server that has multiple versions of each video, encoded at different bitrates and resolutions. The server delivers the encoded video/audio frames switching from one level to another, according to the current available bandwidth. The control is entirely server-based, so the client does not need special additional features. The streaming control employs feedback control theory.[35] Currently, QuavStreams supports H.264/MP3 codecs muxed into the FLV container and VP8/Vorbis codecs muxed into the WEBM container.

upLynk

upLynk delivers HD Adaptive Streaming to multiple platforms, including: iOS, Android, Windows 8/10/Mobile, Roku and all PC/Mac/Linux browser combinations by encoding video in the cloud using a single non-proprietary adaptive streaming format. Rather than streaming and storing multiple formats for different platforms and devices, upLynk stores and streams only one. The first studio to use this technology for delivery was Disney ABC Television, using it for video encoding for web, mobile and tablet streaming apps on the ABC Player, ABC Family and Watch Disney apps, as well as the live Watch Disney Channel, Watch Disney Junior, and Watch Disney XD.[36][37]

Self-learning clients

In recent years, the benefits of self-learning algorithms in adaptive bitrate streaming have been investigated in academia. While most of the initial self-learning approaches are implemented at the server-side[38][39][40] (e.g. performing admission control using reinforcement learning or artificial neural networks), more recent research is focusing on the development of self-learning HTTP Adaptive Streaming clients. Multiple approaches have been presented in literature using the SARSA[41] or Q-learning[42] algorithm. In all of these approaches, the client state is modeled using, among others, information about the current perceived network throughput and buffer filling level. Based on this information, the self-learning client autonomously decides which quality level to select for the next video segment. The learning process is steered using feedback information, representing the Quality of Experience (QoE) (e.g. based on the quality level, the number of switches and the number of video freezes). Furthermore, it was shown that multi-agent Q-learning can be applied to improve QoE fairness among multiple adaptive streaming clients.[43]

Criticisms

HTTP-based adaptive bit rate technologies are significantly more operationally complex than traditional streaming technologies. Some of the documented considerations are things such as additional storage and encoding costs, and challenges with maintaining quality globally. There have also been some interesting dynamics found around the interactions between complex adaptive bit rate logic competing with complex TCP flow control logic.[7][44] [45] [46][47]

However, these criticisms have been outweighed in practice by the economics and scalability of HTTP delivery: whereas non-HTTP streaming solutions require massive deployment of specialized streaming server infrastructure, HTTP-based adaptive bit-rate streaming can leverage the same HTTP web servers used to deliver all other content over the Internet.

With no single clearly defined or open standard for the digital rights management used in the above methods, there is no 100% compatible way of delivering restricted or time-sensitive content to any device or player. This also proves to be a problem with digital rights management being employed by any streaming protocol.

The method of segmenting files into smaller files used by some implementations (as used by HTTP Live Streaming) could be deemed unnecessary due to the ability of HTTP clients to request byte ranges from a single video asset file that could have multiple video tracks at differing bit rates with the manifest file only indicating track number and bit rate. However, this approach allows for serving of chunks by any simple HTTP server and so therefore guarantees CDN compatibility. Implementations using byte ranges such as Microsoft Smooth Streaming require a dedicated HTTP server such as IIS to respond to the requests for video asset chunks.

See also

Multiple description coding

References

  1. ^ Saamer Akhshabi; Ali C. Begen; Constantine Dovrolis (2011). An Experimental Evaluation of Rate-Adaptation Algorithms in Adaptive Streaming over HTTP. In Proceedings of the second annual ACM conference on Multimedia systems (MMSys ’11). New York, NY, USA: ACM.
  2. ^ a b DASH at ITEC, VLC Plugin, DASHEncoder and Dataset by C. Mueller, S. Lederer, C. Timmerer
  3. ^ a b “Proceedings Template – WORD” (PDF). Retrieved 2017-12-16.
  4. ^ Gannes, Liz (10 June 2009). “The Next Big Thing in Video: Adaptive Bitrate Streaming”. Archived from the original on 19 June 2010. Retrieved 1 June 2010.
  5. ^ a b “mmsys2012-final36.pdf” (PDF). Retrieved 2017-12-16.
  6. ^ Marshall, Daniel (18 February 2010). “Show Report: Video Processing Critical to Digital Asset Management”. Elemental Technologies. Retrieved 15 October 2011.
  7. ^ a b Saamer Akhshabi; Ali C. Begen; Constantine Dovrolis. “An Experimental Evaluation of Rate-Adaptation Algorithms in Adaptive Streaming over HTTP” (PDF). Retrieved 15 October 2011.
  8. ^ Anthony Vetro. “The MPEG-DASH Standard for Multimedia Streaming Over the Internet” (PDF). Retrieved 10 July 2015.
  9. ^ Jan Ozer. “What Is Adaptive Streaming?”. Retrieved 10 July 2015.
  10. ^ Jeroen Famaey; Steven Latré; Niels Bouten; Wim Van de Meerssche; Bart de Vleeschauwer; Werner Van Leekwijck; Filip De Turck. “On the merits of SVC-based HTTP Adaptive Streaming”. Retrieved 10 July 2015.
  11. ^ a b libdash: Open-source DASH client library by bitmovin
  12. ^ “Distributed DASH Datset | ITEC – Dynamic Adaptive Streaming over HTTP”. Itec.uni-klu.ac.at. Retrieved 2017-12-16.
  13. ^ DVD Book Construction, DVD Forum, May 2005
  14. ^ Gannes, Liz (10 June 2009). “The Lowdown on Apple’s HTTP Adaptive Bitrate Streaming”. Archived from the original on 19 June 2010. Retrieved 24 June 2010.
  15. ^ “Move Gets Streaming Patent; Are Adobe & Apple Hosed? — Online Video News”. Gigaom.com. 15 September 2010. Retrieved 15 October 2011.
  16. ^ a b “MPEG ratifies its draft standard for DASH”. MPEG. 2 December 2011. Archived from the original on 20 August 2012. Retrieved 26 August 2012.
  17. ^ Timmerer, Christian (2012-04-26). “HTTP streaming of MPEG media – blog entry”. Multimediacommunication.blogspot.com. Retrieved 2017-12-16.
  18. ^ “ISO/IEC DIS 23009-1.2 Dynamic adaptive streaming over HTTP (DASH)”. Iso.org. Retrieved 2017-12-16.
  19. ^ Updates on DASH – blog entry
  20. ^ a b ETSI 3GPP 3GPP TS 26.247; Transparent end-to-end packet-switched streaming service (PSS); Progressive Download and Dynamic Adaptive Streaming over HTTP (3GP-DASH)
  21. ^ “bitdash HTML5 MPEG-DASH player”. Dash-player.com. Retrieved 2017-12-16.
  22. ^ “A VLC media player plugin enabling dynamic adaptive streaming over HTTP” (PDF). Retrieved 2017-12-16.
  23. ^ GPAC Telecom ParisTech
  24. ^ “dash.js”. Github.com. Retrieved 2017-12-16.
  25. ^ Hassoun, David. Adobe Developer Connection. Adobe Systems. http://www.adobe.com/devnet/flashmediaserver/articles/dynstream_advanced_pt1.html. Blog. [Accessed 24 June 2010].
  26. ^ “HTTP Dynamic Streaming”. Adobe Systems. Retrieved 13 Oct 2010.
  27. ^ “FAQ HTTP Dynamic Streaming”. Adobe Systems. Retrieved 12 Jan 2015.
  28. ^ Mac Developer Library. Apple. https://developer.apple.com/library/mac/documentation/networkinginternet/conceptual/streamingmediaguide/UsingHTTPLiveStreaming/UsingHTTPLiveStreaming.html. [Accessed 02 June 2014].
  29. ^ Prince McLean (9 July 2009). “Apple launches HTTP Live Streaming standard in iPhone 3.0”. AppleInsider. Retrieved 15 October 2011.
  30. ^ R. Pantos, HTTP Live Streaming, IETF, retrieved 11 Oct 2011
  31. ^ “Smooth Streaming”. IIS.net. Archived from the original on 15 June 2010. Retrieved 24 June 2010.
  32. ^ Chris Knowlton (8 September 2009), Protected Interoperable File Format, Microsoft, retrieved 15 October 2011
  33. ^ “Microsoft End-to-End Platform Powers Next-Generation Silverlight and IIS Media Experiences Across Multiple Screens”. Microsoft. 8 April 2010. Retrieved 30 July 2011.
  34. ^ “First Day of IBC”. Microsoft. Retrieved 22 Jan 2011.
  35. ^ Luca De Cicco; Saverio Mascolo; Vittorio Palmisano. “Feedback Control for Adaptive Live Video Streaming” (PDF). MMSYS2011. Retrieved 9 September 2012.
  36. ^ Dean Takahashi (2013-01-16). “Uplynk creates a cheap and efficient way for Disney to stream videos”. VentureBeat. Retrieved 2017-12-16.
  37. ^ Dreier, Troy (2013-01-16). “UpLynk Emerges from Stealth Mode; DisneyABC Is First Customer – Streaming Media Magazine”. Streamingmedia.com. Retrieved 2017-12-16.
  38. ^ Y. Fei; V. W. S. Wong; V. C. M. Leung (2006). “Efficient QoS provisioning for adaptive multimedia in mobile communication networks by reinforcement learning”. Mobile Networks and Applications. 11 (1): 101–110. doi:10.1007/s11036-005-4464-2.
  39. ^ V. Charvillat; R. Grigoras (2007). “Reinforcement learning for dynamic multimedia adaptation”. Journal of Network and Computer Applications. 30 (3): 1034–1058. doi:10.1016/j.jnca.2005.12.010.
  40. ^ D. W. McClary; V. R. Syrotiuk; V. Lecuire (2008). “Adaptive audio streaming in mobile ad hoc networks using neural networks”. Ad Hoc Networks. 6 (4): 524–538. doi:10.1016/j.adhoc.2007.04.005.
  41. ^ V. Menkovski; A. Liotta (2013). “Intelligent control for adaptive video streaming”. IEEE International Conference on Consumer Electronics (ICCE). Washington, DC. pp. 127–128.
  42. ^ M. Claeys; S. Latré; J. Famaey; F. De Turck (2014). “Design and evaluation of a self-learning HTTP adaptive video streaming client”. IEEE Communications Letters. 18 (4): 716–719. doi:10.1109/lcomm.2014.020414.132649.
  43. ^ S. Petrangeli; M. Claeys; S. Latré; J. Famaey; F. De Turck (2014). “A multi-agent Q-Learning-based framework for achieving fairness in HTTP Adaptive Streaming”. IEEE Network Operations and Management Symposium (NOMS). Krakow. pp. 1–9.
  44. ^ Pete Mastin (28 January 2011). “Is adaptive bit rate the yellow brick road, or fool’s gold for HD streaming?”. Retrieved 15 October 2011.
  45. ^ Luca De Cicco; Saverio Mascolo. “An Experimental Investigation of the Akamai Adaptive Video Streaming” (PDF). Retrieved 29 November 2011.
  46. ^ “Adaptive streaming: a comparison”.
  47. ^ Chris Knowlton (28 January 2010). “Adaptive Streaming Comparison”.

Further reading