GStreamer consultancy

GStreamer is the framework of choice for Linux-based multimedia. It is deployed on a wide range of hardware products and environments. Our multimedia team has accumulated years of experience in the development of the GStreamer core framework and its plugins. Thanks to its flexible architecture design, the framework can be used to implement software applications targeting many use-cases such as media players, multimedia-aware web engines, video editors, transcoders and streaming servers.

Web engines integration

Web engines provide an attractive approach to designing user-interface in an increasing number of products. Moreover the Web platform provides multiple specifications related to Multimedia support, such as HTML5 video playback, WebAudio, WebCodecs, MediaRecorder, WebRTC, adaptive streaming (Media Source Extensions) and DRM (Encrypted Media Extensions).

We are the main developers and maintainers of the GStreamer-based multimedia back-ends of WebKit that are used by several of its ports, including WebKitGTK and WPE. Since 2009 we have led the development of the GStreamer WebKit back-ends, always making sure they integrate and perform well in the web.

Leveraging our deep knowledge of the GStreamer WebKit back-ends, we also worked on a GStreamer backend for hardware-accelerated multimedia playback for the Servo web engine.

Web augmented video production

Live video events commonly display infographic overlays on top of the video content. For instance a live sports broadcast event would often display a score-board overlay in a corner of the screen. Another example are news tickers, auxiliary infographic scrolling over news broadcasts.

Nowadays the Web platform allows for rich and visually compelling content animations, relying on CSS animations, styling and optionally JavaScript.

By combining the rendering potential of WPE WebKit with the GStreamer multimedia framework, HTML-based graphics overlays can be mixed and composed with any live video source. We are the maintainers of the GstWPE GStreamer plugin, which was designed exactly for this use-case. Any GStreamer-based application can now leverage the full potential of Web-based graphics overlays in the context of live video production.

Our multimedia engineers can offer advise on the design and implementation of GStreamer applications covering live video mixing use-cases, with a dedicated focus on embedded graphics overlays.

Hardware accelerated multimedia processing

Hardware accelerated multimedia is a popular strategy to offload media processing (decoding, encoding, post-processing), which are compute-intensive, from the general purpose processor to custom hardware such as GPUs, DSPs, and so on. Platform vendors usually offer kernel or user-space interfaces to access that external hardware, and interchange encoded/decoded video frames, for example.

We have experience bringing hardware accelerated multimedia to GStreamer, writing plugins and the core functionality required to efficiently use these resources. We are the maintainers of GStreamer-VA, a plugin which provides hardware-accelerated video encoding, decoding, post-processing and rendering for most Intel-based platforms, AMD-based GPUs and others, through VA-API, a widely adopted user-space interface for hardware accelerated multimedia. We also have experience and contributions in GStreamer-Video4Linux plugin, which uses the kernel V4L2 interface, offered for most ARM SoCs vendors. We have contributed to OpenMAX GStreamer plugins too, which used to support platforms such as the Raspberry Pi, and a few other custom and vendor-specific interfaces. We are currently working with the new Vulkan extension for video processing to use it within the GStreamer-Vulkan plugin.

Post production

Non-linear video editing tools are a key component of the post-production pipeline and require a solid and mature multimedia framework. Igalia’s contributions to GStreamer aim to position the project as the sustainable alternative to proprietary and monolithic tools in this particular field of the multimedia industry.

For example, as part of the movie production process, editors often need to script their project timeline creation in order to automate otherwise repetitive tasks. Unfortunately the existing tools don’t naturally fit these new workflows.

GStreamer Editing Services (GES) provides a high level API to build video editing tools in a simple and efficient way, making it possible to reuse the amazing set of GStreamer elements out there. By leveraging its API, new tools can be developed in order support a new generation of editing workflows. For instance, GES allows easy scripting of the timeline editing process from the Python programming language. GES is maintained as a Free Software (FOSS) project, its developers are constantly adding new features in order to make it a good alternative to proprietary frameworks and tools.

Building upon the GES foundations, our Multimedia team members are core contributors and maintainers of the Open-Source Pitivi video editing application. Pitivi is a simple, flexible and powerful application with an easy-to-use user interface. By relying on the GES API, Pitivi provides all the features commonly needed for video post-production editing. Additionally, it can easily be adapted for custom editing workflows, our engineers can provide guidance towards achieving that goal.

Igalia has years of experience developing key technologies for video content creation. We lead the development of the GStreamer Editing Services framework. As maintainers and core developers of the library we are able to guide and advise our customers in the development of custom editing software tools aimed to optimize their workflow.

WebRTC for native applications

WebRTC was initially aimed for peer-to-peer multimedia streaming on the web, but nowadays it can also be used for native GStreamer applications. We maintain the GStreamer webrtcsrc plugin and we are also co-maintainers of the webrtcsink plugin. By using those GStreamer plugins, a GStreamer application can now interact with Web-based WebRTC applications. Thus, an audio/video stream encoded by GStreamer can be sent to WebRTC Selective Forward Units (SFUs) using webrtcsink. In the other direction, GStreamer can receive audio/video streams from WebRTC applications using webrtcsrc. These plugins are particularly important for any use-case where a WebRTC application requires features that are not supported on the Web Platform. For instance, if an application needs to record WebRTC streams, the webrtcsrc plugin can be combined with the GStreamer muxers and encoders. Another example is seamless WebRTC to DASH streaming.

Our Multimedia team has years of experience in WebRTC and has successfully contributed to several products leveraging the GStreamer webrtcsrc and webrtcsink elements.

Other frameworks

GStreamer integrates with third-party multimedia frameworks, such as FFmpeg, and as example, our multimedia engineers have developed GStreamer plugins using Intel’s VA-API and MediaSDK or Google’s libwebrtc.

Thus, our multimedia engineers also have deep expertise in third-party frameworks such as the before mentioned ones. Either by working on the core of these libraries or by making use of them in Multimedia applications, we can help our customers reach their technical milestones.