Hello,
I apologize in advance if this is not a mux.js issue but my tests indicate that it might be.
In my current project I get mp2t packets via a websocket and I am converting those packets to fmp4 using mux.js which I then send to the browser (Windows 10 Chrome Version 60.0.3112.90) via the MSE pipeline. This works great and I am able to see the video for a little while until I get a PIPELINE_ERROR_DECODE error and the stream shuts down. This usually occurs within 30-40 minutes after the video starts playing.
In an effort to narrow down the problem I took the debug index.html (which works great when loading the whole file at a time) and modified it to take a file, read it and then cut it down to 40k slices which I then then feed to the mux.js at a fast rate. I take the output from the “done” event and append it to the MSE buffer. Doing this I also seem to get the PIPELINE_ERROR_DECODE from the browser after the video displays for a few minutes.
I cannot include the file I am using for testing (cause its too big) but any .ts file that is over 20 minutes long with 24 frames per second can be loaded to test this.
Here is the html page I have modified to slice the .ts file:
<!DOCTYPE html>
<!--[if IE 7]> <html class="no-js lt-ie9 lt-ie8"> <![endif]-->
<!--[if IE 8]> <html class="no-js lt-ie9"> <![endif]-->
<!--[if gt IE 8]><!-->
<html class="no-js">
<!--<![endif]-->
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<title></title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width">
<link rel="stylesheet" href="css/nomalize.min.css">
<link rel="stylesheet" href="css/main.css">
<script src="js/vendor/modernizr-2.6.2.min.js"></script>
<style>
fieldset {
padding: 10px;
border: 1px solid #ccc;
}
</style>
</head>
<body>
<!--[if lt IE 7]>
<p class="chromeframe">You are using an <strong>outdated</strong> browser. Please <a href="http://browsehappy.com/">upgrade your browser</a> or <a href="http://www.google.com/chromeframe/?redirect=true">activate Google Chrome Frame</a> to improve your experience.</p>
<![endif]-->
<div class="header-container">
<header class="wrapper clearfix">
<h1 class="title">Transmux Analyzer</h1>
</header>
</div>
<div class="main-container">
<div class="main wrapper clearfix">
<article>
<header>
<p>
This page can help you inspect the results of the transmuxing to mp4 files performed by videojs-contrib-hls. It's still a
bit tricky to create a MSE-compatible fragmented MP4. We've had luck with
<a
href="http://www.bento4.com/developers/dash/">Bento4</a>
and ffmpeg. If you have both of those utilities installed, you can create a working MP4 like this:
<pre>
ffmpeg -i movie.ts -vn -codec copy -absf aac_adtstoasc movie-audio.mp4
mp4fragment --track audio --fragment-duration 11000 movie-audio.mp4 movie-audio.m4s</pre>
<small>Looking for the <a href="legacy.html">FLV tool</a>?</small>
</header>
<section>
<h2>Inputs</h2>
<form id="inputs">
<legend>
The input with the checked radio box will be loaded into the player on this page.
</legend>
<fieldset>
<legend>M2TS Input</legend>
<div>
<input id="original-active" type=radio name=active checked value="original">
<label>
Your original .TS or .AAC segment:
<input type="file" id="original">
</label>
<button id="save-muxed" type="button">Save Transmuxer Output</button>
</div>
<div>
<label><input id="combined-output" type=checkbox name=combined checked value="combined"> Remux output into a single output?
</label>
</div>
<div>
Otherwise, output only:
<label><input id="video-output" type=radio name=output disabled checked value="video"> Video
</label>
<label><input id="audio-output" type=radio name=output disabled checked value="audio"> Audio
</label>
</div>
<div>
<label><input id="reset-tranmsuxer" type=checkbox name=reset checked value="reset"> Recreate the Transmuxer & MediaSource for each file open?
</label>
</div>
</fieldset>
<fieldset>
<legend>MP4 Input</legend>
<input id="working-active" type=radio name=active value="working">
<label>
A working, MP4 version of the underlying stream
produced by another tool:
<input type="file" id="working">
</label>
</fieldset>
<div>
<label>
Codecs:
<input id="codecs" type="text" value="avc1.64001f,mp4a.40.5">
</label>
</div>
</form>
</section>
<section id="video-place">
</section>
<section>
<h2>Comparison</h2>
<div id="comparison">
A diff of the structure of the two MP4s will appear here once you've specified an input TS file and a known working MP4.
</div>
</section>
<section>
<h2>Structure</h2>
<div class="result-wrapper">
<h3>videojs-contrib-hls</h3>
<pre class="vjs-boxes">
</pre>
</div>
<div class="result-wrapper">
<h3>Working</h3>
<pre class="working-boxes"></pre>
</div>
</section>
</article>
</div>
<!-- #main -->
</div>
<!-- #main-container -->
<div class="footer-container">
<footer class="wrapper">
<h3>footer</h3>
</footer>
</div>
<script src="../bower_modules/mux.js/dist/mux.js"></script>
<!-- Include QUnit for object diffs -->
<script src="../bower_modules/mux.js/node_modules/qunitjs/qunit/qunit.js"></script>
<script>
/*! @source http://purl.eligrey.com/github/FileSaver.js/blob/master/FileSaver.js */
window.saveAs = function (view) { "use strict"; if (typeof navigator !== "undefined" && /MSIE [1-9]\./.test(navigator.userAgent)) { return } var doc = view.document, get_URL = function () { return view.URL || view.webkitURL || view }, save_link = doc.createElementNS("http://www.w3.org/1999/xhtml", "a"), can_use_save_link = "download" in save_link, click = function (node) { var event = new MouseEvent("click"); node.dispatchEvent(event) }, is_safari = /Version\/[\d\.]+.*Safari/.test(navigator.userAgent), webkit_req_fs = view.webkitRequestFileSystem, req_fs = view.requestFileSystem || webkit_req_fs || view.mozRequestFileSystem, throw_outside = function (ex) { (view.setImmediate || view.setTimeout)(function () { throw ex }, 0) }, force_saveable_type = "application/octet-stream", fs_min_size = 0, arbitrary_revoke_timeout = 500, revoke = function (file) { var revoker = function () { if (typeof file === "string") { get_URL().revokeObjectURL(file) } else { file.remove() } }; if (view.chrome) { revoker() } else { setTimeout(revoker, arbitrary_revoke_timeout) } }, dispatch = function (filesaver, event_types, event) { event_types = [].concat(event_types); var i = event_types.length; while (i--) { var listener = filesaver["on" + event_types[i]]; if (typeof listener === "function") { try { listener.call(filesaver, event || filesaver) } catch (ex) { throw_outside(ex) } } } }, auto_bom = function (blob) { if (/^\s*(?:text\/\S*|application\/xml|\S*\/\S*\+xml)\s*;.*charset\s*=\s*utf-8/i.test(blob.type)) { return new Blob(["\ufeff", blob], { type: blob.type }) } return blob }, FileSaver = function (blob, name, no_auto_bom) { if (!no_auto_bom) { blob = auto_bom(blob) } var filesaver = this, type = blob.type, blob_changed = false, object_url, target_view, dispatch_all = function () { dispatch(filesaver, "writestart progress write writeend".split(" ")) }, fs_error = function () { if (target_view && is_safari && typeof FileReader !== "undefined") { var reader = new FileReader; reader.onloadend = function () { var base64Data = reader.result; target_view.location.href = "data:attachment/file" + base64Data.slice(base64Data.search(/[,;]/)); filesaver.readyState = filesaver.DONE; dispatch_all() }; reader.readAsDataURL(blob); filesaver.readyState = filesaver.INIT; return } if (blob_changed || !object_url) { object_url = get_URL().createObjectURL(blob) } if (target_view) { target_view.location.href = object_url } else { var new_tab = view.open(object_url, "_blank"); if (new_tab == undefined && is_safari) { view.location.href = object_url } } filesaver.readyState = filesaver.DONE; dispatch_all(); revoke(object_url) }, abortable = function (func) { return function () { if (filesaver.readyState !== filesaver.DONE) { return func.apply(this, arguments) } } }, create_if_not_found = { create: true, exclusive: false }, slice; filesaver.readyState = filesaver.INIT; if (!name) { name = "download" } if (can_use_save_link) { object_url = get_URL().createObjectURL(blob); setTimeout(function () { save_link.href = object_url; save_link.download = name; click(save_link); dispatch_all(); revoke(object_url); filesaver.readyState = filesaver.DONE }); return } if (view.chrome && type && type !== force_saveable_type) { slice = blob.slice || blob.webkitSlice; blob = slice.call(blob, 0, blob.size, force_saveable_type); blob_changed = true } if (webkit_req_fs && name !== "download") { name += ".download" } if (type === force_saveable_type || webkit_req_fs) { target_view = view } if (!req_fs) { fs_error(); return } fs_min_size += blob.size; req_fs(view.TEMPORARY, fs_min_size, abortable(function (fs) { fs.root.getDirectory("saved", create_if_not_found, abortable(function (dir) { var save = function () { dir.getFile(name, create_if_not_found, abortable(function (file) { file.createWriter(abortable(function (writer) { writer.onwriteend = function (event) { target_view.location.href = file.toURL(); filesaver.readyState = filesaver.DONE; dispatch(filesaver, "writeend", event); revoke(file) }; writer.onerror = function () { var error = writer.error; if (error.code !== error.ABORT_ERR) { fs_error() } }; "writestart progress write abort".split(" ").forEach(function (event) { writer["on" + event] = filesaver["on" + event] }); writer.write(blob); filesaver.abort = function () { writer.abort(); filesaver.readyState = filesaver.DONE }; filesaver.readyState = filesaver.WRITING }), fs_error) }), fs_error) }; dir.getFile(name, { create: false }, abortable(function (file) { file.remove(); save() }), abortable(function (ex) { if (ex.code === ex.NOT_FOUND_ERR) { save() } else { fs_error() } })) }), fs_error) }), fs_error) }, FS_proto = FileSaver.prototype, saveAs = function (blob, name, no_auto_bom) { return new FileSaver(blob, name, no_auto_bom) }; if (typeof navigator !== "undefined" && navigator.msSaveOrOpenBlob) { return function (blob, name, no_auto_bom) { if (!no_auto_bom) { blob = auto_bom(blob) } return navigator.msSaveOrOpenBlob(blob, name || "download") } } FS_proto.abort = function () { var filesaver = this; filesaver.readyState = filesaver.DONE; dispatch(filesaver, "abort") }; FS_proto.readyState = FS_proto.INIT = 0; FS_proto.WRITING = 1; FS_proto.DONE = 2; FS_proto.error = FS_proto.onwritestart = FS_proto.onprogress = FS_proto.onwrite = FS_proto.onabort = FS_proto.onerror = FS_proto.onwriteend = null; return saveAs }(typeof self !== "undefined" && self || typeof window !== "undefined" && window || this.content); if (typeof module !== "undefined" && module.exports) { module.exports.saveAs = saveAs } else if (typeof define !== "undefined" && define !== null && define.amd != null) { define([], function () { return saveAs }) }
</script>
<script>
'use strict';
var
$ = document.querySelector.bind(document),
inputs = $('#inputs'),
original = $('#original'),
working = $('#working'),
saveButton = $('#save-muxed'),
vjsParsed,
workingParsed,
diffParsed,
vjsBytes,
workingBytes,
saveConfig,
restoreConfig,
saveTA,
vjsBoxes = $('.vjs-boxes'),
workingBoxes = $('.working-boxes'),
workingBoxes = $('.working-boxes'),
muxedData,
muxedName,
transmuxer,
video,
mediaSource,
logevent,
prepareSourceBuffer;
logevent = function (event) {
console.log(event.type);
};
saveTA = function (tarr, n) { var b = new Blob([tarr], { type: 'application/octet-binary' }); return window.saveAs(b, n); }
saveConfig = function () {
var inputs = [].slice.call(document.querySelectorAll('input:not([type=file])'));
inputs.forEach(function (element) {
localStorage.setItem(element.id,
JSON.stringify({
value: element.value,
checked: element.checked,
disabled: element.disabled
}));
});
};
restoreConfig = function () {
var inputs = [].slice.call(document.querySelectorAll('input:not([type=file])'));
inputs.forEach(function (element) {
var state;
state = JSON.parse(localStorage.getItem(element.id));
if (state) {
element.checked = state.checked;
element.value = state.value;
element.disabled = state.disabled;
}
});
};
document.addEventListener('DOMContentLoaded', restoreConfig);
// output a diff of the two parsed MP4s
diffParsed = function () {
var comparison, diff, transmuxed;
if (!vjsParsed || !workingParsed) {
// wait until both inputs have been provided
return;
}
comparison = $('#comparison');
transmuxed = vjsParsed;
diff = '<p>A <del>red background</del> indicates ' +
'properties present in the transmuxed file but missing from the ' +
'working version. A <ins>green background</ins> indicates ' +
'properties present in the working version but missing in the ' +
'transmuxed output.</p>';
diff += '<pre class="mp4-diff">' +
QUnit.diff(muxjs.mp4.tools.textify(transmuxed, null, ' '),
muxjs.mp4.tools.textify(workingParsed, null, ' ')) +
'</pre>';
comparison.innerHTML = diff;
};
prepareSourceBuffer = function (combined, outputType, callback) {
var
buffer,
codecs,
codecsArray,
resetTransmuxer = $('#reset-tranmsuxer').checked;
if (typeof combined === 'function') {
callback = combined;
combined = true;
}
// Our work here is done if the sourcebuffer has already been created
if (!resetTransmuxer && window.vjsBuffer) {
return callback();
}
video = document.createElement('video');
video.controls = true;
mediaSource = new MediaSource();
video.src = URL.createObjectURL(mediaSource);
window.vjsVideo = video;
window.vjsMediaSource = mediaSource;
$('#video-place').innerHTML = '';
$('#video-place').appendChild(video);
mediaSource.addEventListener('error', logevent);
mediaSource.addEventListener('opened', logevent);
mediaSource.addEventListener('closed', logevent);
mediaSource.addEventListener('sourceended', logevent);
codecs = $('#codecs');
codecsArray = codecs.value.split(',');
mediaSource.addEventListener('sourceopen', function () {
mediaSource.duration = 0;
if (combined) {
buffer = mediaSource.addSourceBuffer('video/mp4;codecs="' + codecs.value + '"');
} else if (outputType === 'video') {
buffer = mediaSource.addSourceBuffer('video/mp4;codecs="' + codecsArray[0] + '"');
} else if (outputType === 'audio') {
buffer = mediaSource.addSourceBuffer('audio/mp4;codecs="' + (codecsArray[1] || codecsArray[0]) + '"');
}
buffer.addEventListener('updatestart', logevent);
buffer.addEventListener('updateend', logevent);
buffer.addEventListener('error', logevent);
window.vjsBuffer = buffer;
video.addEventListener('error', logevent);
video.addEventListener('error', function () {
document.getElementById('video-place').classList.add('error');
});
return callback();
});
};
var fileSlices = [],
sliceLength = 40000,
sliceIndex = 0,
outputType = 'video',
remuxedSegments = [],
remuxedBytesLength = 0,
remuxedInitSegment = null,
bytes,
createInitSegment = true, i, j,
sliceDataBuffer = null,
myTimerFunction = null,
sliceCounterPackets = 0,
sliceFlushLimit = 24,
sliceMAXCounterPackets = 60;
transmuxer = new muxjs.mp4.Transmuxer({ remux: false });
function sliceFile(segment)
{
if(segment && segment != null) {
var isSlicing = true;
while(isSlicing)
{
fileSlices.push(segment.slice(sliceIndex, sliceIndex+ sliceLength));
sliceIndex = sliceIndex + sliceLength;
if(sliceLength > segment.byteLength - sliceIndex)
{
fileSlices.push(segment.slice(sliceIndex, segment.byteLength));
isSlicing = false;
}
}
}
}
transmuxer.on('data', function (event) {
if (event.type === outputType) {
remuxedSegments.push(event);
remuxedBytesLength += event.data.byteLength;
remuxedInitSegment = event.initSegment;
}
});
transmuxer.on('done', function () {
var offset = 0;
if (createInitSegment) {
bytes = new Uint8Array(remuxedInitSegment.byteLength + remuxedBytesLength)
bytes.set(remuxedInitSegment, offset);
offset += remuxedInitSegment.byteLength;
createInitSegment = false;
} else {
bytes = new Uint8Array(remuxedBytesLength);
}
for (j = 0, i = offset; j < remuxedSegments.length; j++) {
bytes.set(remuxedSegments[j].data, i);
i += remuxedSegments[j].byteLength;
}
muxedData = bytes;
remuxedSegments = [];
remuxedBytesLength = 0;
if ($('#original-active').checked) {
prepareSourceBuffer(false, outputType, function () {
console.log('appending...');
try {
if(!window.vjsBuffer.updating) {
window.vjsBuffer.appendBuffer(bytes);
video.play();
} else {
console.log('dropped Packets...');
}
} catch (exp)
{
console.error("Append Error: "+ exp.stack);
}
});
}
});
function concatTypedArrays(a, b) { // a, b TypedArray of same type
var c = new (a.constructor)(a.length + b.length);
c.set(a, 0);
c.set(b, a.length);
return c;
}
original.addEventListener('change', function () {
var reader = new FileReader();
// do nothing if no file was chosen
if (!this.files[0]) {
return;
}
reader.addEventListener('loadend', function () {
var segment = new Uint8Array(reader.result),
combined = document.querySelector('#combined-output').checked,
outputType = document.querySelector('input[name="output"]:checked').value,
resetTransmuxer = $('#reset-tranmsuxer').checked,
remuxedSegments = [],
remuxedInitSegment = null,
remuxedBytesLength = 0,
createInitSegment = false,
bytes,
i, j;
sliceFile(segment);
myTimerFunction = setInterval( addSlicesToQueue, 100);
function addSlicesToQueue() {
if (fileSlices.length > 0) {
if (sliceDataBuffer == null) {
sliceDataBuffer = new Uint8Array(0);
}
sliceDataBuffer = concatTypedArrays(sliceDataBuffer, fileSlices.shift());
if (sliceCounterPackets > sliceFlushLimit) {
transmuxer.push(sliceDataBuffer);
transmuxer.flush();
sliceDataBuffer = null;
sliceCounterPackets = 0;
if (sliceFlushLimit < sliceMAXCounterPackets) {
sliceFlushLimit++;
}
} else {
sliceCounterPackets++;
}
} else {
clearInterval(myTimerFunction);
}
}
});
muxedName = this.files[0].name.replace('.ts', '.f4m');
reader.readAsArrayBuffer(this.files[0]);
}, false);
working.addEventListener('change', function () {
var reader = new FileReader();
reader.addEventListener('loadend', function () {
var bytes = new Uint8Array(reader.result);
if ($('#working-active').checked) {
prepareSourceBuffer(function () {
window.vjsBuffer.appendBuffer(bytes);
video.play();
});
}
workingBytes = bytes;
workingParsed = muxjs.mp4.tools.inspect(bytes);
console.log('working', workingParsed);
diffParsed();
// clear old box info
workingBoxes.innerHTML = muxjs.mp4.tools.textify(workingParsed, null, ' ');
});
reader.readAsArrayBuffer(this.files[0]);
}, false);
$('#save-muxed').addEventListener('click', function () {
if (muxedData && muxedName) {
return saveTA(muxedData, muxedName);
}
});
$('#combined-output').addEventListener('change', function () {
Array.prototype.slice.call(document.querySelectorAll('[name="output"'))
.forEach(function (el) {
el.disabled = this.checked;
}, this);
});
[].slice.call(document.querySelectorAll('input')).forEach(function (el) {
el.addEventListener('change', saveConfig);
});
</script>
</body>
</html>
Here is the error log I get from the chrome://media-internals when I run that page.
Hello, I apologize in advance if this is not a mux.js issue but my tests indicate that it might be.
In my current project I get mp2t packets via a websocket and I am converting those packets to fmp4 using mux.js which I then send to the browser (Windows 10 Chrome Version 60.0.3112.90) via the MSE pipeline. This works great and I am able to see the video for a little while until I get a PIPELINE_ERROR_DECODE error and the stream shuts down. This usually occurs within 30-40 minutes after the video starts playing.
In an effort to narrow down the problem I took the debug index.html (which works great when loading the whole file at a time) and modified it to take a file, read it and then cut it down to 40k slices which I then then feed to the mux.js at a fast rate. I take the output from the “done” event and append it to the MSE buffer. Doing this I also seem to get the PIPELINE_ERROR_DECODE from the browser after the video displays for a few minutes.
I cannot include the file I am using for testing (cause its too big) but any .ts file that is over 20 minutes long with 24 frames per second can be loaded to test this.
Here is the html page I have modified to slice the .ts file: <!DOCTYPE html>
Here is the error log I get from the chrome://media-internals when I run that page.
Thank you very much in advance for your help.