Bad read splice_insert: 16 expected: 20

[turbidwater]

When I parse this SCTE ("/DAlAAAAAAAAAP/wFAUAAqbVf+/+AAAAAH4AUmXAAAAAAAAAdIQsGg==") or any others from our stream, I get a parsed response from the library, but also get this error in the console:

Bad read splice_insert actual: 16 expected: 20
spliceInsert    @   scte35.js:111
parseSCTE35Data @   scte35.js:166
push../node_modules/scte35/build/lib/scte35.js.SCTE35.parseFromB64  @   scte35.js:194

Running the same cue in the CLI on master, I get:

$ npm run parse "/DAlAAAAAAAAAP/wFAUAAqbVf+/+AAAAAH4AUmXAAAAAAAAAdIQsGg=="

> scte35@0.2.0 parse C:\_libraries\scte35
> node ./scripts/cli.js "C:/Program Files/Git/DAlAAAAAAAAAP/wFAUAAqbVf+/+AAAAAH4AUmXAAAAAAAAAdIQsGg=="

C:\_libraries\scte35\lib\scte35.ts:284
        const splice = new DataView(bytes.buffer, offset, sis.spliceCommandLength);
                       ^
RangeError: Invalid DataView length undefined
    at new DataView (<anonymous>)
    at parseSCTE35Data (C:\_libraries\scte35\lib\scte35.ts:284:24)
    at Object.exports.SCTE35.parseFromB64 (C:\_libraries\scte35\lib\scte35.ts:322:12)
    at Object.<anonymous> (C:\_libraries\scte35\scripts\cli.js:22:42)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Function.Module.runMain (module.js:693:10)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! scte35@0.2.0 parse: `node ./scripts/cli.js "C:/Program Files/Git/DAlAAAAAAAAAP/wFAUAAqbVf+/+AAAAAH4AUmXAAAAAAAAAdIQsGg=="`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the scte35@0.2.0 parse script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\Nils\AppData\Roaming\npm-cache\_logs\2019-05-23T16_35_58_717Z-debug.log

I also tried on the splice-descriptors branch and got this error:

$ npm run parse "/DAlAAAAAAAAAP/wFAUAAqbVf+/+AAAAAH4AUmXAAAAAAAAAdIQsGg=="

> scte35@0.2.0 parse C:\_libraries\scte35
> node ./scripts/cli.js "C:/Program Files/Git/DAlAAAAAAAAAP/wFAUAAqbVf+/+AAAAAH4                                                                                                                                  AUmXAAAAAAAAAdIQsGg=="

C:\_libraries\scte35\lib\scte35.ts:266
        throw new Error(`Binary read error sectionLength: ${sis.sectionLength} +                                                                                                                                   3 !== data.length: ${bytes.byteLength}`);
              ^
Error: Binary read error sectionLength: 1003 + 3 !== data.length: 54
    at parseSCTE35Data (C:\_libraries\scte35\lib\scte35.ts:266:15)
    at Object.exports.SCTE35.parseFromB64 (C:\_libraries\scte35\lib\scte35.ts:34                                                                                                                                  4:12)
    at Object.<anonymous> (C:\_libraries\scte35\scripts\cli.js:22:35)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Function.Module.runMain (module.js:693:10)
    at startup (bootstrap_node.js:191:16)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! scte35@0.2.0 parse: `node ./scripts/cli.js "C:/Program Files/Git/DAlAAA                                                                                                                                  AAAAAAP/wFAUAAqbVf+/+AAAAAH4AUmXAAAAAAAAAdIQsGg=="`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the scte35@0.2.0 parse script.
npm ERR! This is probably not a problem with npm. There is likely additional log                                                                                                                                  ging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\Nils\AppData\Roaming\npm-cache\_logs\2019-05-23T16_35_15_6                                                                                                                                  29Z-debug.log

[turbidwater]

Comparing output against this serverside SCTE parser from Azure, looks like the problem may lie in the splice time object. The Azure parser sees time_specified_flag as true, but the JS parser does not. Another parser I tried also indicated the time_specified_flag is set to true.

[estobbart]

@turbidwater Thanks, this is interesting, I'll take a look. What's the source of the stream?

[estobbart]

Looks like it's a CLI problem on windows... I get this output..

$ npm run parse "/DAlAAAAAAAAAP/wFAUAAqbVf+/+AAAAAH4AUmXAAAAAAAAAdIQsGg=="

> scte35@0.2.0 parse /Users/estobb200/Development/scte35-js
> node ./scripts/cli.js "/DAlAAAAAAAAAP/wFAUAAqbVf+/+AAAAAH4AUmXAAAAAAAAAdIQsGg=="

Bad read splice_insert actual: 16 expected: 20
{
    "tableId": 252,
    "selectionSyntaxIndicator": false,
    "privateIndicator": false,
    "sectionLength": 37,
    "protocolVersion": 0,
    "encryptedPacket": false,
    "encryptedAlgorithm": 0,
    "ptsAdjustment": 0,
    "cwIndex": 0,
    "tier": 4095,
    "spliceCommandLength": 20,
    "spliceCommandType": 5,
    "spliceCommand": {
        "spliceEventId": 173781,
        "spliceEventCancelIndicator": false,
        "outOfNetworkIndicator": true,
        "programSpliceFlag": true,
        "durationFlag": true,
        "spliceImmediateFlag": false,
        "spliceTime": {
            "specified": false
        },
        "breakDuration": {
            "autoReturn": false,
            "duration": 126
        },
        "uniqueProgramId": 82,
        "available": 101,
        "expected": 192
    },
    "descriptorLoopLength": 0,
    "crc": 1954819098
}

In your log the CLI looks like it';s trying to parse the path.. > node ./scripts/cli.js "C:/Program Files/Git/DAlAAAAAAAAAP/wFAUAAqbVf+/+AAAAAH4AUmXAAAAAAAAAdIQsGg=="

I do get the..

Bad read splice_insert actual: 16 expected: 20

So probably two different issues.

[estobbart]

From the 2019 version @ 9.8.1, My interpretation of the splice time was..

splice_time() {
    time_specified_flag (1 bit)
    if(time_specified_flag == 1) {
        reserved (6 bits)
        pts_time (33 bits)
    } else
        reserved (7 bits)
}

if time_specified_flag, read 5 bytes, else read 1 byte. Looks like it parses if I always just read it as 5 bytes. lines 80 - 84

            offset++;
            // if (spliceTime.specified) {
            //     offset += 4;
            // }
            offset += 4;

Can you tell if the values beyond that point are correct, with that change?

        "breakDuration": {
            "autoReturn": false,
            "duration": 5400000
        },
        "uniqueProgramId": 0,
        "available": 0,
        "expected": 0

[turbidwater]

Hi @estobbart! Thanks for the quick reply. For breakDuration, I'm getting a duration of 8100000 and autoReturn of false. That object is different between the Azure parser I linked to and a different SCTE parser Azure Parser

"break_duration":{
      "auto_return":false,
      "duration":{
        "pts_time":8100000,
        "wall_clock_seconds_ext":90.0,
        "wall_clock_time_ext":"00:01:30:00000"
      }
    },

SCTE35-Parser

"break_duration":{
         "auto_return":0,
         "reserved":63,
         "duration":8100000
      },

The other properties line up, though.

As for the source, it is a broadcast stream for TV network saved as a replay. This is the SCTE tag for the beginning of an ad break.

[estobbart]

@turbidwater I'm not able to repro this with the other open source parser.. First I converted your base64 to hex..

/DAlAAAAAAAAAP/wFAUAAqbVf+/+AAAAAH4AUmXAAAAAAAAAdIQsGg==

Became..

fc 30 25 00 00 00 00 00 00 00 ff f0 14 05 00 02 a6 d5 7f ef fe 00 00 00 00 7e 00 52 65 c0 00 00 00 00 00 00 74 84 2c 1a

Then I used a different SCTE parser

> var test = "fc302500000000000000fff014050002a6d57feffe000000007e005265c000000000000074842c1a";
undefined
> JSON.stringify(s.parseFromHex(test))
'{"table_id":252,"section_syntax_indicator":0,"private_indicator":0,"section_length":37,"protocol_version":0,"encrypted_packet":0,"encryption_algorithm":0,"pts_adjustment":0,"cw_index":0,"tier":4095,"splice_command_length":20,"splice_command_type":5,"splice_command_type_text":"splice_insert","splice_event":{"splice_event_id":173781,"splice_event_cancel_indicator":0,"out_of_network_indicator":1,"program_splice_flag":1,"duration_flag":1,"splice_immediate_flag":0,"time_specified_flag":1,"pts_time":0,"break_duration":{"auto_return":0,"reserved":63,"duration":5400000},"unique_program_id":0,"avail_num":0,"avails_expected":0},"descriptor_loop_length":0,"descriptors":[]}'

The break duration is {"auto_return":0,"reserved":63,"duration":5400000} I can't tell how you're getting 8100000.

[turbidwater]

@estobbart My apologies. I must have gotten the output mixed up with another SCTE tag. I'm also seeing 5400000 for duration value, but also with additional wall clock info from the Azure parser.

      "duration": {
        "pts_time": 5400000,
        "wall_clock_seconds_ext": 60.0,
        "wall_clock_time_ext": "00:01:00:00000"
      }

I'm not sure how the wall clock values are calculated. Perhaps an optional parsing of the reserve? I think the pts_time / break duration is the important one, though.

I'm getting the same output as you did when I use the other JS SCTE Parser. Sorry about that. I was juggling too many tags.

[estobbart]

The transport stream PTS is a 90kHz clock, so 5400000 / 90000 gets you 60 seconds.

[turbidwater]

Aha! Thanks, that was puzzling me.

[turbidwater]

@estobbart Is there any estimated date for this to get into a new build on NPM?

[estobbart]

No date, I'll push something this weekend. Thanks for reminding me.

[turbidwater]

Thanks!

[turbidwater]

@estobbart any word on publishing a new build? Thanks!

[estobbart]

@turbidwater Apologies for the delay, I didn't actually have access to the npm repo. Should be published now.. https://www.npmjs.com/package/scte35/v/0.2.5

[turbidwater]

@estobbart Thanks! We will check it out