›

Browser source layer - any timeline on this?

Is there a roadmap for providing a browser source for MimoLive? While I know there is a workaround using screencapture, it's not ideal. For example, streams that use HTTP assets like streamlabs or muxy should be used with a browser source. I've been trying the screencapture workaround and I have to set it up each time I restart my computer (since the browser window gets closed and opened as a result, and ML needs to be told which window to capture again).

There are many items in esports/gaming that I would use a browser source to integrate:
- alerts like streamlabs or muxy
- embedded brackets (see http://challonge.com/module/instructions for details)
- customized statistics displays

I know that these things are meant to be able to be achieved using Quartz but considering the amount of html assets already available (and the wide range of devs that can do html as opposed to Quartz) it would be ideal if a browser source was available in ML.

Comments

  • Here's another example of the use of a browser source - https://strexm.tv/

    Gaming streamers currently can't leverage this functionality with MimoLive (I don't think). Might be possible I suppose with the right chroma-keying set up using screengrab - but this is not ideal.

    It would be great if Boinx could provide a response to this question :)

  • I would also love to see this feature added as it would make it easier to use many streaming tools that are available. Including various chat browser sources for integrating chat into your stream and various overlay and alert functionality offered by numerous streaming and third party services.

  • I recently uncovered CoGeWebkit which is a Quartz Composer Patch that implements a WebKit browser instance within Quartz. I attempted to use this to build a custom mimoLive Browser Source Layer. The problem that I ran into was that CoGeWebKit has not been maintained in nearly 7 years and as such apparently some of the frameworks it was using at one point in time have changed and it no longer functions properly. Partial renders of web content into a Billboard patch were the most I could get.

    I later stumbled upon the source code for CoGeWebkit and started developing my own custom QC Patch using CoGe as a guide. Then I uncovered several articles pointing out that the WebView class that CoGe is using has been deprecated as of MacOS 10.10 and that WKWebView should be used instead.

    I then started researching WKWebView and discovered that in MacOS 10.13 a new method is being added that will apparently allow a call to take a snapshot of the viewport of a WKWebView which seems like it would be a good fit for outputting the image data in the custom QC Patch. I just updated to the Gold Master of MacOS 10.13 beta and am continuing my attempt at developing the QC Plugin that will allow for browser rendering in QC that I can then tie into a custom layer for mimoLive.

    I have very limited experience with Objective C and this is really my first project in this language, however I will not stop development on this until I make it work or until Boinx decides to build a browser source into their product.

    Now for my reasoning. I am a Twitch streamer and the vast majority of on stream alert systems are accessed via browser functionality. This means that without a browser source I am extremely limited in this area. At one point I was running 3 separate window captures with chromakeying to work around this. Each one was pointed at a separate page with some custom CSS injected so that I could chromakey them in mimoLive to allow them to sit over my stream. This resulted in a very poor quality stream due to the amount of CPU usage needed to do multiple window captures within mimoLive. I later worked around some of this issue by using a local html document with multiple iFrames to consolidate all of these alert/overlay pages into a single window for capturing and chromakeying. This helped, quite a bit. However I recently moved my stream from a mid-2011 iMac with an i7 to a 2016 15 inch MacBook Pro and believe it or not the MacBook Pro seems to be just a tiny bit slower than the much older iMac in terms of CPU (at least with regard to mimoLive usage. This has resulted in my stream quality decreasing significantly as I am maxing out my CPU when streaming. MimoLive manages to utilize around 250% CPU in my current setup and as such my system typically has less than 1% idle CPU while streaming.

    Now that I have upgraded to MacOS 10.13 my idle CPU is even less. I suspect that some of this is due to some data collection and debugging processes in the beta build but even so, I cannot stream with my full setup from a maxed out less than 1 year old laptop with my streaming software using this much CPU the result on a 720p 30fps stream is that my audio gets out of sync with the camera and mimoLive usually drops FPS output to around 20 and the quality degrades significantly because it cannot keep up with the inputs while encoding and doing a single window capture with chromakey.

    I will try to keep this thread updated on my development progress of both the QC Patch and the subsequent mimoLive layer.

  • @Stimpy Thanks for your efforts! Actually we tried to develop a Web-Browser-Plugin for Quartz Composer some years ago but unfortunately it turned out not to be stable so we never released it. As far as I can remember the problem was that the web view that will be rendered needs to be rendered in the main thread of the application but the video pipeline in mimoLive is running on a background thread and with it all the Quartz Composer stuff. Thats a bit tricky.

    It may actually a better way to have a native web browser source rather than the need for a QuartzComposer plugin. Sure enough we have it on our wish list!

Sign In or Register to comment.