Making Scott Hanselman's Powerpoint Greenscreen trick work with RevealJS via Blazor Server

Scott Hanselman posted a cool video the other day where he used magic (actually a websocket connection) to automatically switch scenes in OBS when moving between slides in Powerpoint.

Virtual PowerPoint Greenscreens! Change a PowerPoint Slide and Change an OBS scene *simultaneously* in 50 lines of C# https://t.co/yKyuDeJDN2 via @YouTube

— Scott Hanselman (@shanselman) September 14, 2020

This made it possible for him to use slides with green areas on, to create a kind of Powerpoint green screen, and put himself “inside” any of his slides.

Well, this is all well and good (I thought to myself) but I’m using slides.com these days, I wonder if it’s possible to pull off the same trick with that?

Well it turns out the answer is an emphatic yes, and what’s more it makes an excellent use case for Blazor Server.

Here’s the gist.

Take a simple slide deck, like this one….

As you can see there’s not much too it, just a couple of slides with some green areas…

The idea is to show this presentation via OBS, where we can use a chroma key filter to replace the green with something else (in this case my webcam).

To this end I can create a scene in OBS to show the browser window containing the presentation, plus my webcam in the correct place to appear in place of the green section of the slide.

For this to work the presentation needs to run full screen which we can easily do using this icon:

One thing you can’t see from the embedded presentation is the speaker notes.

Both slides have notes, and that’s where the relevant OBS scene name is stored.

So when we navigate to this slide we want OBS to auto-switch to the Centre scene.

If we don’t switch OBS scenes, we’re going to end up with problems like this…

I’ve highlighted the webcam here (in red) to show it hasn’t moved so there’s nothing to show in the green part of the second slide.

Crossing the streams

Scott used the OBS.WebSocket.NET Nuget package to make the connection to OBS (to trigger the scene change when moving between slides), and I wanted to do the same thing.

But that means using C#, and interacting with a slides.com embedded presentation relies on javascript.

If only there was a framework that made it fairly painless to communicate between javascript and C#.

Why hello Blazor!

I realised I could probably use Blazor for the OBS socket connection and a little JS Interop to react to slide change events from the embedded presentation.

First up I created a Blazor Server project.

Then added the slides embed code to the markup in Index.razor.

@page "/"

<iframe src="//slides.com/jonhilt/hello-and-welcome/embed?postMessageEvents=true" width="460" height="340" scrolling="no" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>

This handles showing the embedded presentation.

Now to react to slide change events. For this we need javascript.

We can inject IJSRuntime into the component, then use it to invoke a javascript method…

@page "/"
@inject IJSRuntime JsRuntime

<iframe src="//slides.com/jonhilt/hello-and-welcome/embed?postMessageEvents=true" width="460" height="340" scrolling="no" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>

@code {
    protected override async Task OnAfterRenderAsync(bool firstRender)
    {
        if (firstRender)
        {
            var dotNetObjectRef = DotNetObjectReference.Create(this);
            await JsRuntime.InvokeVoidAsync("ListenToSlidesEvents", dotNetObjectRef);
        }
    }
}

Note we have to put this call in OnAfterRenderAsync because of the way Blazor is pre-rendering this page on the server. If we try to do this in OnInitializedAsync we’ll get an error…

The if check is to make sure we only attempt this once when the component is first rendered and not on every (re)render.

Now this is going to expect to find a javascript function called ListenToSlidesEvents, so I added that to _Host.cshtml.

<script>
     window.ListenToSlidesEvents = (dotnetHelper) => {
         // this will be called from Index.razor via OnAfterRenderAsync
     }    
</script>

Now the job of ListenToSlidesEvents is to set up the javascript callbacks for the embedded presentation.

But just before we do that, I added a function to retrieve the slide notes for the current slide (showing in the embedded player).

This is important because we’re going to use the slide notes to indicate which OBS scene to use.

window.getSlideNotes = () => {
    document.querySelector('iframe')
            .contentWindow
            .postMessage(JSON.stringify({ method: 'getSlideNotes' }), '*');
}

This code is heavily based on the example provided on the slides.com help pages, so check that out for more details.

window.ListenToSlidesEvents = (dotnetHelper) => {

    window.addEventListener('message', function(event) {
        var data = JSON.parse(event.data);

        if (data.namespace === 'reveal') {
            if (data.eventName === 'slidechanged' || data.eventName === 'ready') {
                getSlideNotes();
            } else if (data.eventName === 'callback') {
                if (data.method === 'getSlideNotes') {
                    dotnetHelper.invokeMethodAsync('HandleSlideNotes', data.result);
                }
            }
        }
    });

}  

This makes sure we call getSlideNotes when the presentation is loaded initially (ready) and thereafter when the slide changes (slidechanged).

Then we listen for any callbacks, and when we get one we check if the method is getSlideNotes, if so this is the result of our getSlideNotes() function call.

Note the call to dotnetHelper.invokeMethodAsync

This is the magic which enables us to call into our C# component from javascript.

This will look for a method marked up as HandleSlideNotes in our Index component.

Let’s add that now.

@code {

    // other code omitted

    [JSInvokableAttribute("HandleSlideNotes")]
    public async Task HandleSlideNotes(string note)
    {
        if (note == null)
            return;
        
        if (note.StartsWith("OBS:"))
        {           
            var scene = new StringReader(note).ReadLine()?.Substring(4);
            // trigger scene change here 
        }
    }

}

So this will be invoked when we have slide/speaker notes, and will pick up the first line of the notes if they start with “OBS:” before grabbing the scene name from it.

Wire it up to OBS

From here we’re largely copying Scott’s code to actually communicate with OBS…

I grabbed a copy of Scott’s ObsLocal class (check it out in his repo here);

Back in Index.razor I then added a button and handler to attempt the connection to OBS and wired everything up in the HandleSlideNotes method.

Here’s my final markup for Index.razor.

@page "/"
@using SlidesToOBS
@using System.IO
@using OBS.WebSocket.NET
@inject IJSRuntime JsRuntime

<div>
    <h1>1. Connect to OBS ( @(_isConnectedToOBS ? "Connected" : "Not Connected"))</h1>
    <button @onclick="ConnectToOBS">Connect to OBS</button>
</div>

<div class="mt-4">
    <h1>2. Maximise this presentation</h1>
    <iframe src="//slides.com/jonhilt/hello-and-welcome/embed?postMessageEvents=true" width="460" height="340" scrolling="no" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>

</div>

And here’s the final @code.

@code
{
    private ObsLocal OBS;
    private bool _isConnectedToOBS => OBS != null;

    private async Task ConnectToOBS()
    {
        Console.Write("Connecting to OBS...");
        OBS = new ObsLocal();
        await OBS.Connect();
        Console.WriteLine("connected");
    }
    
    protected override async Task OnAfterRenderAsync(bool firstRender)
    {
        if (firstRender)
        {
            var dotNetObjectRef = DotNetObjectReference.Create(this);
            await JsRuntime.InvokeVoidAsync("ListenToSlidesEvents", dotNetObjectRef);
        }
    }

    [JSInvokableAttribute("HandleSlideNotes")]
    public async Task HandleSlideNotes(string note)
    {
        if (note == null)
            return;

        if (OBS == null)
            return;
        
        if (note.StartsWith("OBS:"))
        {
            try
            {
                var scene = new StringReader(note).ReadLine()?.Substring(4);
                OBS.ChangeScene(scene);
            }
            catch (ErrorResponseException e)
            {
                Console.WriteLine(note);
                Console.WriteLine(e.Message);
            }
            
        }
    }
}

That’s a wrap

So yes, you absolutely can adapt this technique to work with RevealJS/Slides.com presentations.

It just takes a little javascript interop but Blazor makes the OBS side of this nice and straightforward.

Here it is in action, and yes, it is necessary to pull funny faces when recording demos like this :-)

Here’s the source code.

Worth saying this is only a proof of concept, I’m sure there are big holes in the code!

Next up