Not getting the onSegment event, and unclear documentation

[kay-master]

Hello,

I'm trying to append an MP4 video that comes from <input type="file"> input tag, and reading it as arrayBuffer, then appending the buffer chunk of the file to mp4Box.appendBuffer(), I do get the onReady and onMoovStart events but for whatever reasons am not getting the onSegment event, here's a snippet of the code,

let mp4Box = mp4box.createFile()

 mp4Box.onReady = (info: any) => {
      movieInfo = info;
      console.log("Received File Information", info);
      mp4Box.onSegment = (id, user, buffer, sampleNum, is_last) => {
        let sb = user;
        sb.segmentIndex++;
        sb.pendingAppends.push({ id: id, buffer: buffer, sampleNum: sampleNum, is_last: is_last });
        console.info("Application", "Received new segment for track " + id + " up to sample #" + sampleNum + ", segments pending append: " + sb.pendingAppends.length);
        // this.chunkBuffer.push(buffer)
        this.onUpdateEnd(sb, true, false);
        // appendToBuffer()
      }
      initializeAllSourceBuffers()

    }

 let addBuffer = (mp4track) => {

      var track_id = mp4track.id;
      var codec = mp4track.codec;
      let mime = 'video/mp4; codecs="' + codec + '"';
      let ms = this.mediaSource;
      let sb;
      if (MediaSource.isTypeSupported(mime)) {
        try {
          console.info("MSE - SourceBuffer #" + track_id, "Creation with type '" + mime + "'");
          sb = ms.addSourceBuffer(mime);
          sb.addEventListener("error", function (e) {
            console.error("MSE SourceBuffer #" + track_id, e);
          });
          sb.ms = ms;
          sb.id = track_id;
          mp4Box.setSegmentOptions(track_id, sb);
          sb.pendingAppends = [];
        } catch (e) {
          console.error("MSE - SourceBuffer #" + track_id, "Cannot create buffer with type '" + mime + "'" + e);
        }
      } else {
        console.warn("MSE", "MIME type '" + mime + "' not supported for creation of a SourceBuffer for track id " + track_id);
      }
    }

let onInitAppended = (e) => {
      var sb = e.target;
      if (sb.ms.readyState === "open") {
        console.log("Init segment append ended");
        sb.sampleNum = 0;
        sb.removeEventListener('updateend', onInitAppended);
        sb.addEventListener('updateend', () => this.onUpdateEnd(sb, true, true));
        /* In case there are already pending buffers we call onUpdateEnd to start appending them*/
        this.onUpdateEnd(sb, false, true);
        sb.ms.pendingInits--;
        if (sb.ms.pendingInits === 0) {
          mp4Box.start();
        }
      }
    }

let initializeAllSourceBuffers = () => {
      if (movieInfo) {
        var info = movieInfo;
        for (var i = 0; i < info.tracks.length; i++) {
          var track = info.tracks[i];
          addBuffer(track);
        }

        let initSegs = mp4Box.initializeSegmentation();
        console.log(initSegs)
        for (let i = 0; i < initSegs.length; i++) {
          console.info("Init Segment ")
          let sb = initSegs[i].user
          sb.addEventListener("updateend", onInitAppended);
          sb.appendBuffer(initSegs[i].buffer);
          sb.segmentIndex = 0;
          sb.ms.pendingInits++;
          console.info("MSE - SourceBuffer #" + sb.id, "Appending initialization data");
          // this.chunkBuffer.push(initSegs[i].buffer)
          // saveBuffer(initSegs[i].buffer, 'track-'+initSegs[i].id+'-init.mp4');
        }
      }
    }

The code above gets triggered once we start appending to the mp4Box.appendBuffer(chunk) method.

Finally the code that reads on the file as array buffer and appending the above mentioned method.

// this.file is coming from the file input tag <input type="file">
let file = this.file, fileSize = file.size, _offset = 0, nextStart = 0;

let onparsedbuffer = (_mp4box, buffer) => {
      if (buffer.byteLength == 0) return

      buffer.fileStart = nextStart;
      console.log("File Start = " + nextStart + " and _mp4box appending: ", buffer)
      nextStart = _mp4box.appendBuffer(buffer);
    }

let readChunk = (_file, _chunk) => {
        let r = new FileReader();

        r.onload = (evt: any) => {

          if (evt.target.error == null) {

            let data = evt.target.result;

            _offset += data.byteLength;
            onparsedbuffer(mp4Box, data);

          } else {
            console.log("Read error: " + evt.target.error);
            return;
          }

          if (_offset >= fileSize) {
            // mp4Box.flush();
            let currentDate: any = new Date();
            console.log("Done reading file (" + fileSize + " bytes) in " + (currentDate - startDate) + " ms");

            return;
          }
          readChunk(_file, _chunk);
        };
        let blob = _file.slice(_offset, _chunk + _offset);
        r.readAsArrayBuffer(blob);

      }

readChunk(file, chunkSize);

I'm totally failing get this working, the code above is derived from the given a example at MP4Box.js - JavaScript MP4 Reader/Fragmenter, the difference here is that I want to append the local selected mp4 file to the Media Source Extension. I don't know were is the issue, but what I notice is, am not getting the onSegment event.

Note: I would like to improve the documentation of this project but I don't understand it yet the one that is available now its hard to understand.

[cconcolato]

It's hard to understand what's wrong in your code, without a version running somewhere or more info. Do you have a log of a run? Which browser are you using? What is the file size? what is the chunk size? how many tracks? how many samples do each track have? Why did you comment out the flush?

[kay-master]

My bad, i forgot to mention those things.

I've tested on both Chrome, Version 74.0.3729.169 (Official Build) (64-bit) and FireFox, 66.0.3 (64-bit).

I commented out flush because I was trying to learn how mp4box works, but even if its there I don't get the onSegment event. Chunk size is *1024 1024 = 1Mb** , has 3 tracks, wvtt which is not supported by Media Source, mp4a.40.2 and avc1.42c00d, with duration 26460160 and 15000000 respectively.

To do you better, am sharing the code am testing with here Test Code and the video I am testing with Test MP4 Video

Am not sure if this info is enough, and lastly I believe I did addSourceBuffer for each track

[kay-master]

I think I solved it, on the example MP4Box.js - JavaScript MP4 Reader/Fragmenter there is a use of sb.ms.pendingInits++ which on my case doesn't increment at all or decrement. Therefore mp4Box.start(); never gets called because this statement never becomes true

if (sb.ms.pendingInits === 0) {
          mp4Box.start();
        }

Maybe there's something I didn't see on the example. Thanks for responding because I was close on giving up.