search

carsten-gross / ts2shout

Convert mpeg transport stream into shoutcast or native mpeg data stream
GNU General Public License v2.0
8 stars 2 forks source link

Problem with continuation frames #1

Closed keynet closed 4 years ago

keynet commented 4 years ago

Hi, I'm trying your utility with VDR & streamdev plugin which outputs a TS stream, which I think should work (both dvbsnoop and dvbinspector seem OK with the stream, and it plays in SMplayer). I seem to be running into a CRC problem as soon as a multi-frame packet is processed. Here's a trace with debug (note line 153 of ts2hout.c errors out when debug is on).

cat test1.ts | ./ts2shout [Sat Mar 21 19:24:46.291059 2020] [ts2shout:info] [pid 12799] init_structures(): Subscribing to MPEG-TS PID 0, 17, 18 (CHANNEL_TYPE_PAT, CHANNEL_TYPE_SDT, CHANNEL_TYPE_EIT) [Sat Mar 21 19:24:46.291107 2020] [ts2shout:info] [pid 12799] ts2shout version v0.85-1-g722354a compiled 21.03.2020 17:39 started [Sat Mar 21 19:24:46.291115 2020] [ts2shout:info] [pid 12799] MPEG streaming with shoutcast StreamTitles in FILTER mode without RDS support. PAT: Found data, table 0x00 (Section length 13), service_id 4164, section 0, last section 0 [Sat Mar 21 19:24:46.291148 2020] [ts2shout:info] [pid 12799] extract_pat_payload(): programme has transport_stream_id: 4164 [Sat Mar 21 19:24:46.291155 2020] [ts2shout:info] [pid 12799] add_channel(): Subscribing to MPEG-TS PID 1300 (Type CHANNEL_TYPE_PMT) PAT: Programme 1, MAP-PID 1300 PMT: Found data, table 0x02 (Section length 78), program number 6912, section 0, last section 0, INFO Length 0 [Sat Mar 21 19:24:46.291176 2020] [ts2shout:info] [pid 12799] add_payload_from_pmt(): Found mp1/mp2 audio stream in PID 1302 (service_id 6912) [Sat Mar 21 19:24:46.291182 2020] [ts2shout:info] [pid 12799] add_channel(): Subscribing to MPEG-TS PID 1302 (Type CHANNEL_TYPE_PAYLOAD) CHANNEL_TYPE_EIT: Found multi-frame-table 0x6e (last table 0x65), Section-Length: 1824, Section: 32 CHANNEL_TYPE_EIT: continued frame: offset: 187, counter: 1, section_length: 1824 [Sat Mar 21 19:24:46.291205 2020] [ts2shout:info] [pid 12799] Synced to MPEG-1 layer 2, 192 kbps, 48000 Hz, Joint Stereo CHANNEL_TYPE_EIT: continued frame: offset: 371, counter: 2, section_length: 1824 PAT: Found data, table 0x00 (Section length 13), service_id 4164, section 0, last section 0 PAT: Programme 1, MAP-PID 1300 CHANNEL_TYPE_EIT: continued frame: offset: 555, counter: 3, section_length: 1824 CHANNEL_TYPE_EIT: continued frame: offset: 739, counter: 4, section_length: 1824 CHANNEL_TYPE_EIT: continued frame: offset: 923, counter: 5, section_length: 1824 CHANNEL_TYPE_EIT: continued frame: offset: 1107, counter: 6, section_length: 1824 CHANNEL_TYPE_EIT: continued frame: offset: 1291, counter: 7, section_length: 1824 CHANNEL_TYPE_EIT: continued frame: offset: 1475, counter: 8, section_length: 1824 CHANNEL_TYPE_EIT: continued frame: offset: 1659, counter: 9, section_length: 1824 CHANNEL_TYPE_EIT: finished multi-frame table after offset 1843, counter: 10 EIT: crc32 FAIL (761318255, l: 1827)

(and subsequent crc fails on all such packets)

Attached very short start of .ts and matching dvbsnoop output

dvbsnoop-test1.txt test1.zip

Let me know what more tests I can help with Thanks

carsten-gross commented 4 years ago

Hi,

Am Samstag, 21. März 2020, 21:09:41 CET schrieb keynet:

Hi, I'm trying your utility with VDR & streamdev plugin which outputs a TS stream, which I think should work (both dvbsnoop and dvbinspector seem OK with the stream, and it plays in SMplayer). I seem to be running into a CRC problem as soon as a multi-frame packet is processed. Here's a trace with debug (note line 153 of ts2hout.c errors out when debug is on).

Thank you very much for your feedback. I'm into debugging this multi-frame issue - obviously there is a bug in my code that is not triggered by the mpeg frames generated by tvheadend but by the full EIT frames gathered from vdr.

I've got two sample files generated by VDR that are pointing me to the problem and I'm working to fix it.

Regards,

Carsten

keynet commented 4 years ago

Thanks for your reply. Hopefully the short sample I provided is enough, I can provide more if that helps, and help test/debug. Let me know... BTW This decodes the output conveniently: cat test1.ts | dvbsnoop -s ts -tssubdecode -if - > dvbsnoop-test1.txt

carsten-gross commented 4 years ago

Hello Keynet,

Am Sun, 22 Mar 2020, schrieb keynet:

Thanks for your reply. Hopefully the short sample I provided is enough, I can provide more if that helps, and help test/debug. Let me know...

The output of dvbsnoop was the key for my insight. I made a commit (ee8f8505888ea29cc1098206882b371e04090c10) that respects the pointer field of the ts frames containing EIT data. Now most(not all) EIT tables are working now. To get all EIT table combinations working a change of the internal handling of the ts frames is necessary (To get short EIT tables in a row also right).

At least two vdr files I got in the past are working now.

Viele Grüße,

Carsten Groß

-- Carsten Gross | http://www.siski.de/~carsten/

keynet commented 4 years ago

Great thanks. I recompiled, and now it's very nearly OK. The station is correct, but the first character of the currently playing data is not right - EIT: Current transmission `ront Row' The program is "Front Row" There's also some additional program info data - would be really good to recover: ... nt Row. Owen Sheers on adapting the much-loved classic series The Snow Sp...

[Thu Mar 26 19:32:38.490303 2020] [ts2shout:info] [pid 9926] init_structures(): Subscribing to MPEG-TS PID 0, 17, 18 (CHANNEL_TYPE_PAT, CHANNEL_TYPE_SDT, CHANNEL_TYPE_EIT) [Thu Mar 26 19:32:38.490482 2020] [ts2shout:info] [pid 9926] ts2shout version v0.86-1-gee8f850 compiled 26.03.2020 18:06 started [Thu Mar 26 19:32:38.490547 2020] [ts2shout:info] [pid 9926] MPEG streaming with shoutcast StreamTitles in FILTER mode without RDS support. [Thu Mar 26 19:32:38.490637 2020] [ts2shout:info] [pid 9926] extract_pat_payload(): programme has transport_stream_id: 4164 [Thu Mar 26 19:32:38.490697 2020] [ts2shout:info] [pid 9926] add_channel(): Subscribing to MPEG-TS PID 1300 (Type CHANNEL_TYPE_PMT) [Thu Mar 26 19:32:38.490768 2020] [ts2shout:info] [pid 9926] add_payload_from_pmt(): Found mp1/mp2 audio stream in PID 1302 (service_id 6912) [Thu Mar 26 19:32:38.490825 2020] [ts2shout:info] [pid 9926] add_channel(): Subscribing to MPEG-TS PID 1302 (Type CHANNEL_TYPE_PAYLOAD) [Thu Mar 26 19:32:38.490955 2020] [ts2shout:info] [pid 9926] Synced to MPEG-1 layer 2, 192 kbps, 48000 Hz, Joint Stereo [Thu Mar 26 19:32:38.491939 2020] [ts2shout:info] [pid 9926] SDT: Stream is station BBC Radio 4 from network . [Thu Mar 26 19:32:38.492583 2020] [ts2shout:info] [pid 9926] EIT: Current transmission `ront Row' [Thu Mar 26 19:32:38.507537 2020] [ts2shout:info] [pid 9926] filter_global_loop: short read, only got 16 bytes, instead of 188, trying to resync [Thu Mar 26 19:32:38.657752 2020] [ts2shout:info] [pid 9926] filter_global_loop: read from stream 1.50 MB, wrote 0.28 MB, no bytes left to read - EOF. Exiting.

carsten-gross commented 4 years ago

Am Donnerstag, 26. März 2020, 20:45:29 CET schrieb keynet:

Great thanks. I recompiled, and now it's very nearly OK. The station is correct, but the first character of the currently playing data is not right - EIT: Current transmission `ront Row' The program is "Front Row" There's also some additional program info data - would be really good to recover: ... nt Row. Owen Sheers on adapting the much-loved classic series The Snow Sp...

Ah, this is very likly a charset issue. Please send me a snippet of the transport stream (a few MB, up to 10) to check and correct this.

Most stations I've tested on Astra are using latin1. As you are listening to BBC, it is very likely that there is no charset required (7bit ascii only). I can see in my code that there is no handling there and latin1 is currently somewhat hardcoded.

EIT provides a short and long title for the current transmission - the short title is written to the log, but the full, long title is used in the shoutcast StreamTitle.

keynet commented 4 years ago

Attached is 5Megs of streamdev TS output for you.

Some other notes FYI... This is the output of the VDR's streamdev plugin - the usual way to stream - project is here: https://projects.vdr-developer.org/projects/plg-streamdev It can output TS, PES or ES, or via a transcoding script, which might be how I initiate ts2shout eventually. I use streamdev ES output with LMS right now and it works well, but streamdev only outputs the current channel at the time of connection, as an (icy) header, which LMS sees OK. It doesn't have the ability to inject the timed metadata as ts2shout does, as programs start etc, which is what I'm trying to achieve. streamdev TS output doesn't currently work with LMS, not the output of ts2shout. smplayer can see/play streamdev TS output (no meta), smplayer also doesn't like ts2shout output currently.

Apache errors: 233] SDT: Warning: Stream (also) contains unkown service with id 0x 1: /srv/www/cgi-bin/ts2shout [Fri Mar 27 12:45:48.379087 2020] [cgi:error] [pid 19706] [client 10.0.0.100:46980] AH01215: [Fri Mar 27 12:45:48.378907 2020] [ts2shout:info] [pid 20190] SDT: Warning: Stream (also) contains unkown service with id 0x 1: /srv/www/cgi-bin/ts2shout

test2.zip

carsten-gross commented 4 years ago

Am Freitag, 27. März 2020, 13:52:37 CET schrieb keynet:

Attached is 5Megs of streamdev TS output for you.

Thank you. I've no access to a stream without charset encoding (everything on Astra is using a charset encoding in one form or another). With your example I can test this now.

I fixed the code for your example file. It says "It's a Fair Cop" as short event (This is also written to syslog) and "It's a Fair Cop - 1/4. Former Humberside sergeant and stand-up comic Alfie Moore returns for another series where the audience are sworn in as police officers to face real life policing dilemmas" should be seen in the Shoutcast stream as full title.

I use streamdev ES output with LMS right now and it works well, but streamdev only outputs the current channel at the time of connection, as an (icy) header, which LMS sees OK. It doesn't have the ability to inject the timed metadata as ts2shout does, as programs start etc, which is what I'm trying to achieve. streamdev TS output doesn't currently work with LMS, not the output of ts2shout. smplayer can see/play streamdev TS output (no meta), smplayer also doesn't like ts2shout output currently.

Are the headers are all set correctly? You can also run ts2shout on the command line as filter and generate an audio file without shoutcast stuff. This file is playable with mplayer/vlc etc.

keynet commented 4 years ago

Brilliant - thanks for your updates. Now I have it streaming with both Title and Metadata - see attached screenshot (via Apache vhost) I'd like to log to Apache access log if at all possible rather than error log - is there a quick way to do that ? Or I might run from the command line/use a streamdev script. It also works in VLC BTW, but not in smplayer SQ

carsten-gross commented 4 years ago

Am Sonntag, 29. März 2020, 22:40:32 CEST schrieb keynet:

Brilliant - thanks for your updates.

You are welcome.

Now I have it streaming with both Title and Metadata - see attached screenshot (via Apache vhost) I'd like to log to Apache access log if at all possible rather than error log - is there a quick way to do that ? Or I might run from the command line/use a streamdev script.

A this is not that easy because the logging in apache error log is just the output on stderr.

Writing with openlog/syslog could be implemented - but actually I'm quite happy with the current approach.

carsten-gross commented 4 years ago

Fixed this in v0.88