W3AXL commented 3 years ago

So far I have been unable to figure out how to use the metadata scripts in-browser, specifically IcecastMetadataReader.

I've tried to include IcecastMetadataReader.js in the html document, but it is a Node.js-specific file and results in console errors when included.

I also tried to "browserify" the script using the Node.js browserify tool. While including the resulting .js file no longer results in a console error on page load, I cannot use the IcecastMetadataReader class in any other scripts and it errors out as being undefined.

Help would be greatly appreciated in getting started here. I'm likely missing something simple but I can't figure out what it is.

W3AXL commented 3 years ago

Likely relevant to #34 as well

eshaz commented 3 years ago

34 will provide a separate module that can be used easily in vanilla javascript, which I believe is what you are attempting.

Referencing the library itself in the html via a script tag can certainly be done, but you would need to combine all of the other imports together and wrap the IcecastMetadataReader file in additional code so it's accessible in your html.

You could also create an npm project, install icecast-metadata-js using npm (https://github.com/eshaz/icecast-metadata-js/tree/master/src/icecast-metadata-js#installing), and then build your Javascript code with something like webpack: https://webpack.js.org. You would then use the output of the webpack build as the file to include in your HTML. I would highly recommend taking this route over manually combining the files together.

W3AXL commented 3 years ago

I'll give webpack a shot. #34 will probably be my best bet for a dead-simple player webapp, but in the meantime I'll see what I can do with a bundled-up version created via webpack.

Apologies for any misunderstandings, I'm still a novice when it comes to anything more advanced than basic javascript.

W3AXL commented 3 years ago

Currently either I'm doing something wrong, or webpack-ing the code doesn't seem to work currently.

I created an extremely basic test script using IcecastReadableStream:

import IcecastReadableStream from "icecast-metadata-js";

const endpointUrl = "https://streams.w3axl.com/icy/laportecounty";

var startButton = document.getElementById('startStream');
startButton.addEventListener('click', startStreaming);

function startStreaming() {
    console.log("beginning stream fetch");
    fetch(endpointUrl, {
        method: "GET",
        headers: {
            "Icy-MetaData": "1",
        }
    })
    .then(async (response) => {
        const icecast = new IcecastReadableStream(
            response,
            {
                parseStream,
                parseMetadata,
                metadataTypes: ["icy"]
            }
        );
        await icecast.startReading();
    })
}

function parseStream(stream) {
    //console.log(stream);
}

function parseMetadata(metadata) {
    console.log(metadata);
}

And when I attempt to load the resulting compiled script in a simple index.html webpage:

<html>
    <head>
        <meta charset="utf-8" />
        <title>Icecast Metadata Testing</title>
    </head>
    <body>
        <button id="startStream">Start Stream</button>
        <h4 id="metdata"></h4>
    </body>
    <script src="main.js"></script>
</html>

I receive the following error in the console:

Uncaught ReferenceError: module is not defined
    at eval (IcecastMetadataQueue.js:101)
    at Object../node_modules/icecast-metadata-js/src/IcecastMetadataQueue.js (main.js:29)
    at __webpack_require__ (main.js:74)
    at eval (index.cjs:1)
    at Object../node_modules/icecast-metadata-js/index.cjs (main.js:18)
    at __webpack_require__ (main.js:74)
    at eval (index.js:1)
    at main.js:97
    at main.js:98
    at main.js:100

It seems that the compiled webpack main.js is referencing IcecastMetadataQueue.js for whatever reason, which has a module.exports directive inside that causes Chrome to throw a fit. Not sure why that's happening but getting things working is obviously not as easy as "webpack it and go."

I think I'll have to wait until #34 is complete. I'm unfamiliar with Node.JS and I don't feel like bashing my head against the wall until I figure things out.

eshaz commented 3 years ago

Your code looks fine, actually this is a bug in the exports of icecast-metadata-js. Webpack is supposed to replace the CommonJS require and modules so the code works in a browser. I ran into the same issue while working on #34, and I'll be releasing a patch soon that fixes this.

W3AXL commented 3 years ago

Good to know it's not something on my end at least. Will be anxiously awaiting the updates!

eshaz commented 3 years ago

@W3AXL I pushed a new release that should fix this issue for you. I was able to verify it worked locally using the default webpack configuration that gets generated when running npx webpack init

W3AXL commented 3 years ago

I now have IcecastReadableStream working with my stream! Metadata is being parsed perfectly.

However, I do have one more question I hope you can help me with. I'm not at all familiar with the inner workings of Media Source Extensions, and I'm having trouble getting the stream chunks to play. Hopefully this is the best place to ask.

If I simply add the stream chunks to my sourceBuffer object with appendBuffer(value.stream) as they come in, I get the somewhat well-documented error Failed to execute 'appendBuffer' on 'SourceBuffer': This SourceBuffer is still processing an 'appendBuffer' or 'remove' operation.

If I try to implement a basic buffering setup, waiting for the source to finish processing, such as :

function pushStream(value) {
    bufferQueue.push(value.stream);
    bufferAvail = true;
    if (!streaming) {
        console.log("starting source buffering loop");
        streaming = true;
        processBuffer();
        sourceBuffer.addEventListener('updateend', processBuffer);
    }
    console.log("Current buffer size: " + bufferQueue.length);
}

function processBuffer() {
    // push in new data if there's any available
    if (bufferAvail) {
        console.log("processing next chunk");
        sourceBuffer.appendBuffer(bufferQueue.shift());
        // if we used up all the buffer, set to false
        if (!bufferQueue.length) {
            bufferAvail = false;
        }
    }
}

the buffer continuously increases in size and I only get a small blip of audio at the beginning of the stream.

I guess that implies my stream is giving me data faster than the sourceBuffer can process the chunks? I would think it should be able to handle a low-bitrate mp3 stream without any issues.

Any suggestions?

eshaz commented 3 years ago

@W3AXL That's great to hear that it's working with webpack now. The MSE api can be tricky, and the documentation / examples that I was able to find has been sporadic at best, at least that was the case when I originally wrote this code. The module I'm working on for #34 will do all of this for you, making this much simpler to just use off the shelf.

If you want to write this logic yourself, you can take a look here for a working example: https://github.com/eshaz/icecast-metadata-js/blob/master/src/demo/src/icecast/MetadataPlayer.js In the appendSourceBuffer method, there are a few calls to waitForSourceBuffer method that wraps the updateend event in a promise that resolves when the SourceBuffer is done updating. The SourceBuffer has to be done updating before any append or remove operations can happen. https://github.com/eshaz/icecast-metadata-js/blob/d9e7e45073a51f746712eb2628f6115e9509b9c1/src/demo/src/icecast/MetadataPlayer.js#L40-L66

W3AXL commented 3 years ago

Perfect, thanks for the link!

I did basically exactly what you did, minus the flac buffering, and it seems to be working pretty darn well. A little bit of skipping/crackling here or there, so I might add a half-second buffer or so, but otherwise the Metadata is perfectly sync-ed up!

eshaz commented 3 years ago

@W3AXL I just released the off-the-shelf module that you can include as a <script> tag in your HTML. I also added a few new demo pages that use regular HTML and give examples on how to use the new module. https://github.com/eshaz/icecast-metadata-js/tree/master/src/icecast-metadata-player

Since you're using this for a radio application, you may be interested in the metadata queue, which shows all of the buffered metadata updates. Usually for a low bitrate mp3 stream for P25 radio, the normal 32KB burst-size will give about 30 seconds of buffered audio / metadata. On your Icecast server, you can increase the burst-size to send more data to clients to increase the number of messages that show up in the queue.

eshaz / icecast-metadata-js

How to use icecast-metadata-js in-browser? #36

34 will provide a separate module that can be used easily in vanilla javascript, which I believe is what you are attempting.