OBS Studio on Wayland

As of today, I’m happy to announce that all of the pull requests to make OBS Studio able to run as a native Wayland application, and capture monitors and windows on Wayland compositors, landed.

I’ve been blogging sparsely about my quest to make screencasting on Wayland a fluid and seamless experience for about a couple of years now. This required some work throughout the stack: from making Mutter able to hand DMA-BUF buffers to PipeWire; to improving the GTK desktop portal; to creating a plugin for OBS Studio; to fixing bugs in PipeWire; it was a considerable amount of work.

But I think none of it would matter if this feature is not easily accessible to everyone. The built-in screen recorder of GNOME Shell already works, but most importantly, we need to make sure applications are able to capture the screen properly. Sadly our hands are tied when it comes to proprietary apps, there’s just no way to contribute. But free and open source software allows us to do that! Fortunately, not only OBS Studio is distributed under GPL, it also is a pretty popular app with an active community. That’s why, instead of creating a fork or just maintaining a plugin, I decided to go the long hard route of proposing everything to OBS Studio itself.

The Road to Native Wayland

Making OBS Studio work on Wayland was a long road indeed, but fortunately other contributors attempted to do it before I did, and my pull requests were entirely based on their fantastic work. It took some time, but eventually the 3 big pull requests making OBS Studio able to run as a native Wayland application landed.

OBS Studio running as a native Wayland client. An important step, but not very useful without a way to capture monitors or windows.

After that, the next step was teaching OBS Studio how to create textures from DMA-BUF information. I wrote about this in the past, but the tl;dr is that implementing a monitor or window capture using DMA-BUFs means we avoid copying buffers from GPU memory to RAM, which is usually the biggest bottleneck when capturing anything. Exchanging DMA-BUFs is essentially about passing a few ids (integers) around, which is evidently much faster than copying dozens of megabytes of image data per second.

Fortunately for us, this particular feature also landed, introducing a new function gs_texture_create_from_dmabuf() which enables creating a gs_texture_t from DMA-BUF information. Like many other DMA-BUF APIs, it is pretty verbose since it needs to handle multiplanar buffers, but I believe this API is able to handle pretty much anything related to DMA-BUFs. This API is being documented and will freeze and become stable soon, with the release of OBS Studio 27, so make sure to check if and see if there’s anything wrong with it!

This was the last missing piece of the puzzle to implement a Wayland-compatible capture.

PipeWire & Portals to the Rescue!

An interesting byproduct of the development of apps sandboxing mechanisms are portals. Portals are D-Bus interfaces that provide various functionalities at runtime. For example, the Document portal allows applications with limited access to the filesystem to ask the user to select a file; the user is then presented with a file chooser dialog managed by the host system, and after selecting, the application will have access to that specific file only, and nothing else.

The portal that we’re interested in here is the Desktop portal, which provides, among others, a screencasting interface. With this interface, sandboxed applications can ask users to select either a window or a monitor, and a video stream is returned if the user selects something. The insecure nature of X11 allows applications to completely bypass this interface, but naturally it doesn’t work Wayland. At most, Xwayland will give you an incomplete capture of some (or no) applications running through it.

It is important to notice that despite born and hosted under the Flatpak umbrella, portals are mostly independent of Flatpak. It is perfectly possible to use portals outside of a Flatpak sandbox, and even when running it as a Snap or an AppImage. It’s merely a bunch of D-Bus calls after all. Portals are also implemented by important Wayland desktops, such as GNOME, KDE, and wlroots, which should cover the majority of Wayland desktops out there.

Remember that I vaguely mentioned above that the screencast interface returns a video stream? This video stream is actually a PipeWire stream. PipeWire here is responsible for negotiating and exchanging buffers between the video producer (GNOME Shell, Plasma, etc) and the consumer (here, OBS Studio).

These mechanisms (portals, and PipeWire) were the basis of my obs-xdg-portal plugin, which was recently merged into OBS Studio itself as part of the built-in capture plugin! Fortunately, it landed just in time for the release of OBS Studio 27, which means soon everyone will be able to use OBS Studio on Wayland.

And, finally, capturing on Wayland works!

Meanwhile at Flatpakland…

While contributing with these Wayland-related features, I sidetracked a bit and did some digging on a Flatpak manifest for OBS Studio.

Thanks to the fantastic work by Bilal Elmoussaoui, there is a GitHub action that allows creating CI workflows that build Flatpaks using flatpak-builder. This allowed proposing a new workflow to OBS Studio’s CI that generates a Flatpak bundle. It is experimental for now, but as we progress towards feature parity between Flatpak and non-Flatpak, it’ll eventually reach a point where we can propose it to be a regular non-experimental workflow.

In addition to that, Flatpak greatly helps me as a development tool, specially when used with GNOME Builder. The Flatpak manifest of OBS Studio is automatically detected and used to build and run it. Running OBS Studio is literally an one-click action:

Running OBS Studio with Flatpak on GNOME Builder


Next Steps

All these Wayland, PipeWire, and portals pull requests are only the first steps to make screencasting on Wayland better than on X11. There’s still a lot to do and fix, and contributions would be more than welcomed.

For a start, the way the screencast interface currently works doesn’t mix well with OBS Studio’s workflow. Each capture pops up a dialog to select a monitor or a window, and that’s not exactly a fantastic experience. If you have complex scenes with many window or screen captures, a swarm of dialogs will pop up. This is clearly not great UX, and improving this would be a good next step. Fortunately, portals are extensible enough to allow implementing a more suitable workflow that e.g. saves and restores the previous sessions.

Since this was tested in a relatively small number of hardware setups and environments, I’m sure we’ll need a round of bugfixes once people start using this code more heavily.

There’s also plenty of room for improvements on the Flatpak front. My long-term goal is to make OBS Studio’s CI publish stable releases to Flathub directly, and unstable releases to Flathub Beta, all automatically with flat-manager. This will require fixing some problems with the Flatpak package, such as the obs-browser plugin not working inside a sandbox (it uses CEF, Chromium Embedded Framework, which apparently doesn’t enjoy the sandbox PID remapping) nor on Wayland (Chromium barely supports Wayland for now).

Of course, I have no authority over what’s going to be accepted by the OBS Studio community, but these goals seem not to be controversial there, and might be great ways to further improve the state of screencasting on Wayland.

I’d like to thank everyone who has been involved in this effort; from the dozens of contributors that tested the Wayland PRs, to the OBS Studio community that’s been reviewing all this code and often helping me, to the rest of the Flatpak and GNOME communities that built the tooling that I’ve been using to improve OBS Studio.

17 thoughts on “OBS Studio on Wayland

  1. his will require fixing some problems with the Flatpak package, such as the obs-browser plugin not working inside a sandbox (it uses CEF, Chromium Embedded Framework, which apparently doesn’t enjoy the sandbox PID remapping)

    I think you can work around CEF not liking the sandbox with https://github.com/refi64/zypak at least that works with Electron.

    Like

  2. Thank you for all this work. As someone that likes to run a lot of bleeding-edge technology (Wayland, PipeWire, BTRFS etc) and livestreams regularly, your work here has been incredibly valuable. I hope for more neat things to come!

    Like

  3. Hey there, thanks for doing all the work for Wayland. Is there a particular reason upstream has its own manifest different from Flathub’s rather than having them be the same? I recently did some work on Flathub’s package for OBS Studio to enable extensions for plugins like obs-websocket, and got a package for that plugin submitted to Flathub, and would hate to see that work get lost.

    Like

    1. There’s no particular reason. But over time, I’d like to delete the Flahub manifest, and make OBS Studio’s CI push updates directly to Flathub repositories using flat-manager. We can even go crazy and push RCs and betas to Flathub Beta 🙂

      Like

  4. Cool! Have you looked at ffmpeg VAAPI? I get one frame every 5 seconds only. Tried 30 fps @ 720p, exactly the same. It looks in the result like it’s actually encoding a lot of frames, it’s just the same frame over and over mostly…

    It’s the same on X11 so not Wayland related at all 🙂

    Like

  5. Thank you very much for all of this work! I really want to use sophisticated tools like OBS for future live streams instead of ffmpeg with kmsgrab like I do right now. I will be trying out the latest git branch tomorrow on sway (wlroots based) to see if there are any major problems. I’m very excited to try this! ^^

    Like

  6. Thank you for your work. I have been following the PR’s on and off for quite a while. Even tried building them at some point to give feedback, but failed. So it’s great to see that it’s now going to be available in an easy to consume way.

    Like

Leave a Reply to schuhumi Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s