Media Elements

MoonshineJS provides a few ways to transcribe audio from HTML elements like <audio> and <video>. The easiest is the VideoCaptioner–just pass it a <video> and MoonshineJS will overlay it with captions while it plays. If you need more control, use the MediaElementTranscriber to handle the transcriptions as they are generated.

VideoCaptioner

You can use the VideoCaptioner to add captions to a video with just a few lines. First you need an HTML <video> element:

<video id="myVideo" controls>
    <source src="/js/guide/transcription/eno.mp4" />
</video>

Then in JS you pass the element to a VideoCaptioner:

var captioner = new Moonshine.VideoCaptioner(
    document.getElementById("myVideo"), // pass in the video element
    "model/tiny"
);

Now when you play the video, it will display captions once the model has loaded:

MediaElementTranscriber

If you need more control of the design, you can use the MediaElementTranscriber to transcribe the output of <audio> and/or <video> elements. Let’s start by creating an <audio> element:

<audio id="myAudio" controls>
    <source src="/js/guide/transcription/beckett.wav" />
</audio>

Then in our JavaScript, we grab the element from the DOM and pass it to a new transcriber:

var transcriber = new Moonshine.MediaElementTranscriber(
    document.getElementById("myAudio"), // pass in the audio element
    "model/tiny",
    {
        onTranscriptionCommitted(text) {
            console.log(text);
        },
    }
);

When you create the MediaElementTranscriber, it will automatically bind to the element’s play and pause events so that transcription triggers accordingly when the element is played or paused. Try it below:

The process is the same for <video> elements; simply create a <video> element, fetch it from the DOM, and pass it into the transcriber. See the result below: