The last time I built an application with Flutter, I- okay to be honest I didn’t really write much Flutter stuff; as it was primarily low-level backend-y stuff with Dart.
This time it will be different! (ish..)
Whilst OBS is great and all, many agree that it is not very user-friendly..
To the newbie… it’s complicated.
To the UI folks out there… it’s ugly.
To the UX peeps… it’s convoluted.
To the programmers… it’s open source and that’s great news for us!
I wanted to address a long-standing issue that I had with OBS - that there was no inbuilt ‘control surface’ which I could use. So as the usual, let’s create one ourselves!
But but but Andrew, software XYZ exists already!
Let’s.
Create.
One.
Ourselves.
There already exists some popular solutions, however they fell short in satisfying my requirements
- OBS Remote Tablet
- No Preview / Program / Source feed
- No Preview / Program bank
- Can’t hide specific scenes
- OBS-web
- No Preview / Program bank
- Can’t hide specific scenes
- OBS Studio Vision mixer
- No Preview / Program / Source feed
- No Preview bank
- Can’t hide specific scenes
There probably is some solution that does exactly what I want, but whatever.
I was quite captivated when I first stumbled upon illusdidi’s OBS Studio Vision mixer. Compared to the other two mentioned projects, this app most closely emulated the ‘vision mixer’ vibe that I was looking for. It also had support for some auxiliary banks, to essentially switch sources in a scene separate to the main program feed.
📒 The Plan 📝
Requirements
Functionality
Source Feed
The ability to preview the display of a scene before it is displayed on the program feed.
Essentially something like a cut-down multiviewer. I know that it would be network heavy and computationally expensive to implement a smooth video feed of multiple sources, so I settled for a periodic update of certain scenes.
Customisable Preview and Program banks
When creating similar scenes in OBS, you would often create a “base scene” where common elements would be added. The “base scene” could then be composited on to the relevant scenes which share those elements. However this would create scenes that you would never really want to individually send to PGM.
Exeldro’s Downstream Keyer (DSK) somewhat addresses this issue with so-called ‘global sources’, but it wouldn’t fix the extraneous sources from appearing on other control surface solutions. I wanted the ability to select certain scenes and to display them (in any order) on a bank of buttons. This is akin to what you can do with a Streamdeck paired with Bitfocus Companion.. but of course a web/desktop/mobile application solution.
n-Box Management
When you see two people/sources at the same time, you see a 2-box
When you see three people/sources at the same time, you see a 3-box
When you seenpeople/sources at the same time, you see ann-box
(For the ATEM users out there, this is also known as a SuperSource)
On a hardware video mixer you would probably build up your n-box with an M/E bus and maybe a DSK too. To emulate that functionality in OBS, we could programatically create some layout scenes, as well as switcher scenes. To perform the switching we could then have additional control banks for the different box n-boxes.
Maybe in the future I could implement layout switching functionality, similar to to what the MixEffect app does (although I would have to see if obs-websocket can handle that). For now, all the layout stuff has to be set up manually in OBS.
Note to the future me if I come around to implementing this: Issue #6
Persistent Settings
A given really, but the ability to save settings and load them the next time the application is run.
Cross Compatibility
As much as I do love web applications, I wanted the final application to also be deployable as a native desktop application, as well as a mobile app! Flutter seemed to fit the bill (albeit Dart is still not my forte, and so I had had to invest time into learning syntax and best practices rather than implementing features)
Lightweight Footprint
As this client would be contacting a production machine (i.e. the computer actually running OBS, or perhaps the same machine itself) - this program should not consume excessive computer resources such as CPU time and memory (ahem looking at you Electron) that would otherwise degrade OBS’s operating performance.
As of 5/09/2021, Vix only consumes around 75MB of RAM - nice!
🚧 Building Vix 👷♂️
Frontend
Whilst I had some experience with the Dart language from my previous project, I still didn’t have experience with the Flutter UI framework itself, so there was definitely much learning that had to be done. I’ve outlined some of my experiences (both the positives and negatives) in a a different post.
Overall, the UI/UX development experience was quite pleasant as Google’s Material widgets in Flutter (FYI Flutter and Dart were created by Google) were well created.
I however wasn’t able to find any graphical designer tool to visually create interfaces. Whilst I did find Flutter Studio, it felt unpolished and not feature-rich enough (though I might have missed them). Furthermore, the process of creating “stateful widgets” was abit time-consuming, even with the templates that VS Code offered me.
Also, figuring out state management in Flutter was an interesting experience - I found out about flutter_hooks all too late :').
I most definitely have unoptimised and excessive state cycle routines. I even created a global variable somewhere in the code to pass some variable (although technically this is okay since it shouldn’t trigger any component re-render)
With all that said, it was straight-forwards to compose the different widgets together to create the different interface components. I was abit concerned with the levels of nesting that were needed to be done, but I guess it’s probably fine in this day age, added to the fact that components only rerender if needed (instead of rerendering on every cycle).
Main Interface
Source Feeds
The source feeds ran a function to fetch a screenshot of a given source every defined period of time, as well as when requested
Button Banks
I created reusable button banks components that would highlight their active button (i.e. red for PGM switching, green for PRVW)
n-box Source Switcher
Each box had its own preview to easily identify what the current source was for a given box
Settings
I created separate settings pages too, one for OBS settings, and the other for application settings
Backend
OBS WebSockets
Did there already exist a Dart package that would allow me to connect to OBS through WebSockets? Yes
Did I use it? Of course not!
It looked alright, however it also didn’t seem to be adopted well, with only two stars on the GitHub repository (more than I’ve ever had???) (also I didn’t want to read the documentation)
web_socket_channel
The official web_socket_channel WebSocket wrapper package seemed to have a bug (#38 #128) with handling failed WebSocket connections, preventing the programmatic handling of the exception, so I worked around that by creating the underlying WebSocket connection myself, then passing the connection object into the wrapper.
omg cross platform support
Vix
Anyway, so after about a week’s worth of sporadic development, we now have some sort of ‘alpha’ version! I created a GitHub Actions workflow to automatically build Vix for different platforms, and was moreorless effortless!
To anyone wanting to try it out, here’s the web version - http://featherbear.cc/obs-vix
Note - Make sure to visit the site as http:// as a HTTPS site cannot establish a connection with OBS (without extra setup)
For the native versions, you can find them on the GitHub repository - though they might expire so just build it yourself I guess 🤷♂️
alright bye
