xbmc / inputstream.ffmpegdirect

Supports streams opened by FFmpeg's libavformat or Kodi's cURL such as plain TS, HLS and DASH (non-DRM) as well as many others. There is support for Archive/Catchup services where there is a replay window and can timeshift across that span. Also provides timeshift for live streams where rewind/pause and fast-forward would not have been available.
GNU General Public License v2.0
58 stars 42 forks source link

How to open a video file? #149

Open markg85 opened 2 years ago

markg85 commented 2 years ago

Hi,

I've spend some time trying to figure out how this library works. From what i get it's very much tailored towards receiving data that is streaming in. Specifically for formats like HLS and DASH.

What i want to do is play a normal file (no stream). So for example, i have this playlist file

#EXTM3U
#KODIPROP:inputstream=inputstream.ffmpegdirect
#KODIPROP:mimetype=application/x-mpegURL
#KODIPROP:inputstream.ffmpegdirect.is_realtime_stream=false
#KODIPROP:inputstream.ffmpegdirect.open_mode=ffmpeg
https://dweb.link/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi

If you save that as bla.m3u and click it, it will play Big Buck Bunny.

If you open it in KODI it will show the information of the file. Like duration, container, speaker setup, etc... So it can definitely read it like this. But it doesn't play it. The logging looks as follows:

2021-10-11 22:09:15.105 T:454469    INFO <general>: VideoPlayer::OpenFile: /home/mark/Downloads/kodi_test/test.m3u
2021-10-11 22:09:15.105 T:454585    INFO <general>: Creating InputStream
2021-10-11 22:09:15.107 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: inputstream.ffmpegdirect: OpenStream() - Num Props: 2
2021-10-11 22:09:15.107 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: inputstream.ffmpegdirect property: inputstream.ffmpegdirect.is_realtime_stream = false
2021-10-11 22:09:15.107 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: inputstream.ffmpegdirect property: inputstream.ffmpegdirect.open_mode = ffmpeg
2021-10-11 22:09:15.107 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: Stream mimetype: application/x-mpegURL
2021-10-11 22:09:15.107 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: OpenWithFFmpeg - IO handled by FFmpeg's AVFormat
2021-10-11 22:09:15.219 T:454549    INFO <general>: CPythonInvoker(1, /home/mark/GitProjects/kodi_build/kodi_data/addons/service.xbmc.versioncheck/resources/lib/runner.py): script successfully run
2021-10-11 22:09:15.227 T:454549    INFO <general>: Python interpreter stopped
2021-10-11 22:09:15.330 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]: Input #0, matroska,webm, from 'https://dweb.link/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi':
2021-10-11 22:09:15.330 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:   Metadata:
2021-10-11 22:09:15.330 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     title           : Big Buck Bunny, Sunflower version
2021-10-11 22:09:15.330 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     COMMENT         : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
2021-10-11 22:09:15.330 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     MAJOR_BRAND     : isom
2021-10-11 22:09:15.330 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     MINOR_VERSION   : 512
2021-10-11 22:09:15.330 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     COMPATIBLE_BRANDS: isomiso2avc1mp41
2021-10-11 22:09:15.330 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     ARTIST          : Blender Foundation 2008, Janus Bager Kristensen 2013
2021-10-11 22:09:15.330 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     COMPOSER        : Sacha Goedegebure
2021-10-11 22:09:15.330 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     GENRE           : Animation
2021-10-11 22:09:15.330 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     ENCODER         : Lavf58.76.100
2021-10-11 22:09:15.330 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:   Duration: 00:10:34.65, start: -0.007000, bitrate: 2109 kb/s
2021-10-11 22:09:15.330 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:   Stream #0:0: Video: vp8, yuv420p(progressive), 1920x1080, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     Metadata:
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:       HANDLER_NAME    : GPAC ISO Video Handler
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:       VENDOR_ID       : [0][0][0][0]
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:       ENCODER         : Lavc58.134.100 libvpx
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:       DURATION        : 00:10:34.647000000
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:   Stream #0:1: Audio: opus, 48000 Hz, stereo, fltp (default)
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     Metadata:
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:       HANDLER_NAME    : GPAC ISO Audio Handler
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:       VENDOR_ID       : [0][0][0][0]
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:       ENCODER         : Lavc58.134.100 libopus
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:       DURATION        : 00:10:34.208000000
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]: Input #0, matroska,webm, from 'https://dweb.link/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi':
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:   Metadata:
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     title           : Big Buck Bunny, Sunflower version
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     COMMENT         : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     MAJOR_BRAND     : isom
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     MINOR_VERSION   : 512
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     COMPATIBLE_BRANDS: isomiso2avc1mp41
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     ARTIST          : Blender Foundation 2008, Janus Bager Kristensen 2013
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     COMPOSER        : Sacha Goedegebure
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     GENRE           : Animation
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     ENCODER         : Lavf58.76.100
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:   Duration: 00:10:34.65, start: -0.007000, bitrate: 2109 kb/s
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:   Stream #0:0: Video: vp8, yuv420p(progressive), 1920x1080, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     Metadata:
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:       HANDLER_NAME    : GPAC ISO Video Handler
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:       VENDOR_ID       : [0][0][0][0]
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:       ENCODER         : Lavc58.134.100 libvpx
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:       DURATION        : 00:10:34.647000000
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:   Stream #0:1: Audio: opus, 48000 Hz, stereo, fltp (default)
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:     Metadata:
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:       HANDLER_NAME    : GPAC ISO Audio Handler
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:       VENDOR_ID       : [0][0][0][0]
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:       ENCODER         : Lavc58.134.100 libopus
2021-10-11 22:09:15.331 T:454585    INFO <general>: AddOnLog: inputstream.ffmpegdirect: ffmpeg[733469A0]:       DURATION        : 00:10:34.208000000
2021-10-11 22:09:15.637 T:454585    INFO <general>: Creating Demuxer
2021-10-11 22:09:15.637 T:454585    INFO <general>: Opening stream: 0 source: 256
2021-10-11 22:09:15.637 T:454585    INFO <general>: Creating video codec with codec id: 139
2021-10-11 22:09:15.637 T:454585    INFO <general>: CDVDVideoCodecFFmpeg::Open() Using codec: On2 VP8
2021-10-11 22:09:15.637 T:454585    INFO <general>: Creating video thread
2021-10-11 22:09:15.638 T:454588    INFO <general>: running thread: video_thread
2021-10-11 22:09:15.638 T:454585    INFO <general>: Opening stream: 1 source: 256
2021-10-11 22:09:15.638 T:454585    INFO <general>: Finding audio codec for: 86076
2021-10-11 22:09:15.639 T:454585    INFO <general>: CDVDAudioCodecFFmpeg::Open() Successful opened audio decoder opus
2021-10-11 22:09:15.639 T:454585    INFO <general>: Creating audio thread
2021-10-11 22:09:15.639 T:454589    INFO <general>: running thread: CVideoPlayerAudio::Process()
2021-10-11 22:09:30.585 T:454585    INFO <general>: Process - eof reading from demuxer
2021-10-11 22:09:30.585 T:454585    INFO <general>: CVideoPlayer::OnExit()
2021-10-11 22:09:30.585 T:454585    INFO <general>: VideoPlayer: eof, waiting for queues to empty
2021-10-11 22:09:30.585 T:454585    INFO <general>: Closing stream player 1
2021-10-11 22:09:30.585 T:454585    INFO <general>: Waiting for audio thread to exit
2021-10-11 22:09:30.593 T:454589    INFO <general>: thread end: CVideoPlayerAudio::OnExit()
2021-10-11 22:09:30.593 T:454585    INFO <general>: Closing audio device
2021-10-11 22:09:30.593 T:454585    INFO <general>: Deleting audio codec
2021-10-11 22:09:30.593 T:454585    INFO <general>: Closing stream player 2
2021-10-11 22:09:30.593 T:454585    INFO <general>: waiting for video thread to exit
2021-10-11 22:09:30.593 T:454588   ERROR <general>: Got MSGQ_ABORT or MSGO_IS_ERROR return true
2021-10-11 22:09:30.593 T:454588    INFO <general>: thread end: video_thread
2021-10-11 22:09:30.593 T:454585    INFO <general>: deleting video codec
2021-10-11 22:09:30.595 T:454540    INFO <general>: Deleting settings information for files /home/mark/Downloads/kodi_test/test.m3u
2021-10-11 22:09:30.595 T:454585    INFO <general>: ADDON: Dll Destroyed - Inputstream FFmpeg Direct
2021-10-11 22:09:30.633 T:454469    INFO <general>: CVideoPlayer::CloseFile()

I'm guessing i'm missing just one single property to make things work, but i can't quite figure out which one.

Also, to be clear. I want to play a file over web but eventually want to go a step further. I want to add support for the ffmpeg crypto protocol which requires setting the decryption_key and decryption_iv properties on ffmpeg. And the easiest way to get that working in KODI seems to be through this addon. The URL that the M3U file contains would change to: crypto+https://dweb.link/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi

That is the real goal here. I'm also happy to hear your thought on that idea!

Best regards, Mark

phunkyfish commented 2 years ago

Will kodi play your url directly using an strm file? I.e. like this: https://kodi.wiki/view/Internet_video_and_audio_streams

What if you set the open mode to curl?

markg85 commented 2 years ago

Will kodi play your url directly using an strm file? I.e. like this: https://kodi.wiki/view/Internet_video_and_audio_streams

Yes

What if you set the open mode to curl?

Nope, won't work either. But that wouldn't work at all with my eventual idea to use ffmpeg's crypto protocol now does it?

I'm still digging through the code, trying to figure out why it won't start. I have a hunch (but i could be way off) that the code guesses a format from the mime type (here: https://github.com/xbmc/inputstream.ffmpegdirect/blob/Nexus/src/stream/FFmpegStream.cpp#L677) thus i think the iformat variable is filled with something that isn't this video file (it's VP8 encoded as you can see in the logging output from my initial post).

If i'm correct at all, which i doubt, then trying to just pass the iformat as NULL could make it work. This value is eventually passed to a avformat_open_input (see here) which forces the format to be what is provided. Unless NULL is provided, then it tries to detect it from the datastream.

Edit Yeah, that iformat idea (passing it as NULL) turned out to be nothing. I didn't check the value it had but i did comment out the part i linked too. That didn't change anything for me. Any hints as to where i should debug this?

phunkyfish commented 2 years ago

Using the crypto protocol wouldn't work anyway, at least I don't think it would. We would need to provide a mechanism with KODIPROPS to leverage it.

The default value of iformat is NULL so as you found it won't make a difference. I can't get the stream to open successfully with ffmpegdirect either.

The output is the same if it is opened with ffmpeg or curl. It correctly detects the format so I'm not sure where the problem is.

markg85 commented 2 years ago

Using the crypto protocol wouldn't work anyway, at least I don't think it would. We would need to provide a mechanism with KODIPROPS to leverage it.

I will make the patches to make that possible. Once i have it working without encryption first...

The default value of iformat is NULL so as you found it won't make a difference. I can't get the stream to open successfully with ffmpegdirect either.

The output is the same if it is opened with ffmpeg or curl. It correctly detects the format so I'm not sure where the problem is.

In my initial log you can see the (one and only) error Got MSGQ_ABORT or MSGO_IS_ERROR return true When searching through the codebase, that line occurs just twice. https://github.com/xbmc/xbmc/blob/9b080490af7838c10be233c627279d977e8fd101/xbmc/cores/VideoPlayer/VideoPlayerAudio.cpp#L255 and https://github.com/xbmc/xbmc/blob/2d988cefa91b6f73e13f3bbfcb364f0eee7e016a/xbmc/cores/VideoPlayer/VideoPlayerVideo.cpp#L341

The error line itself doesn't say (directly) if the error comes from the Video or Audio processing, but the thread id in front of it does spoil it. It's T:454588 and when looking at the other T:454588 ones we can see that it definitely is the video part writing this error log line.

I can't quite figure out what's going wrong there..

Just a hypothetical case. Could it be that the video is somehow read from the end and immediately stops (as there is nothing to play at the end)?

markg85 commented 2 years ago

I am a small step further!

Here's a small log snippet of the same video file (using STRM) that plays.

2021-10-13 13:07:09.693 T:6111     INFO <general>: running thread: CVideoPlayerAudio::Process()
2021-10-13 13:07:09.694 T:6107    DEBUG <general>: CVideoPlayer::SetCaching - caching state 1
2021-10-13 13:07:09.694 T:6107    DEBUG <general>: CDVDClock::SetSpeedAdjust - adjusted:0.000000
2021-10-13 13:07:09.694 T:6107    DEBUG <general>: CVideoPlayer::HandleMessages - player 2 reported state: 0
2021-10-13 13:07:09.694 T:6107    DEBUG <general>: CVideoPlayer::SetCaching - caching state 1
2021-10-13 13:07:09.694 T:6064    DEBUG <general>: OnAVChange: CApplication::OnAVChange
2021-10-13 13:07:09.694 T:6107    DEBUG <general>: CDVDClock::SetSpeedAdjust - adjusted:0.000000
2021-10-13 13:07:09.694 T:6110    DEBUG <general>: CVideoPlayerVideo - Stillframe left, switching to normal playback

And the same snippet when using m3u:

2021-10-13 13:02:41.567 T:5780     INFO <general>: running thread: CVideoPlayerAudio::Process()
2021-10-13 13:02:41.567 T:5776    DEBUG <general>: CVideoPlayer::SetCaching - caching state 2
2021-10-13 13:02:41.567 T:5776    DEBUG <general>: CDVDClock::SetSpeedAdjust - adjusted:0.000000
2021-10-13 13:02:41.567 T:5776    DEBUG <general>: CVideoPlayer::HandleMessages - player 2 reported state: 0
2021-10-13 13:02:41.567 T:5726    DEBUG <general>: OnAVChange: CApplication::OnAVChange
2021-10-13 13:02:41.568 T:5776    DEBUG <general>: CDVDDemuxClient::ParsePacket - (0) profile changed from -99 to 0
2021-10-13 13:02:41.577 T:5780    DEBUG <general>: CVideoPlayerAudio - CDVDMsg::GENERAL_PAUSE: false
2021-10-13 13:02:41.577 T:5780    DEBUG <general>: CDVDAudio::Pause - pausing audio stream
2021-10-13 13:02:41.578 T:5776    DEBUG <general>: CVideoPlayer::HandleMessages - player 1 reported state: 0
2021-10-13 13:02:41.867 T:5654    DEBUG <general>: Inhibiting OS screen saver

The thing i noticed here was the caching state. With m3u, it was set at 2. With STRM it's set at 1.

These values correspond to this enum:

  enum ECacheState
  {
    CACHESTATE_DONE = 0,
    CACHESTATE_FULL,     // player is filling up the demux queue
    CACHESTATE_INIT,     // player is waiting for first packet of each stream
    CACHESTATE_PLAY,     // player is waiting for players to not be stalled
    CACHESTATE_FLUSH,    // temporary state player will choose startup between init or full
  };

Where 1 is working in this case and 2 isn't. With the comment behind it i do get a sense that 1 should be used here, not 2.

Now i just need to figure out how that state is influenced from your addon. Any help would be really welcome here :)

phunkyfish commented 2 years ago

That state is influenced by Video Player. Not the addon.

The difference is that when using a STRM file you are using Kodi’s internal demuxer and when using the M3U you have the add-ons demuxer.

Similarly if you remove the ffmpegdirect kodi props from the M3U my guess is it would play the stream similar to the STRM file.

markg85 commented 2 years ago

You're right. Removing those ffmpegdirect properties indeed makes it play too. I still prefer to go the route of this addon because it seems the logical step to add crypto support later on.

I've been debugging quite a bit now with gdb. Yes, i can set breakpoints and follow the code. Both kodi and this addon so tht's really helping. Or so i thought. I'm not getting anywhere, lol.

Thus far i noticed one weird thing. When a frame is requested from the demuxer (which goes into this addon) it eventually enters this if statement

  if (!IsVideoReady())
  {
    m_packet.reset();
    DemuxPacket *pPacket = CDVDDemuxUtils::AllocateDemuxPacket(0);
    pPacket->demuxerId = m_demuxerId;
    return pPacket;
  }

which is in DVDDemuxClient.cpp (DemuxPacket* CDVDDemuxClient::Read())

It seems like it never parses a video frame...

phunkyfish commented 2 years ago

Correct, it never appears to read any frame data. I found the same and I don’t know why.

markg85 commented 2 years ago

I've been debugging it ever since my previous reply. Somehow a video frame is tried to be read from "stream 1" which is the audio in this particular video. But i can't find where it sets that stream to 1 to begin with...

Would you mind having another shot at it?

Also, are you on discord somewhere? That would help with debugging this.

phunkyfish commented 2 years ago

I can give it another try at the weekend. Not on discord I’m afraid.

markg85 commented 2 years ago

Awesome! I'll post here if i find something before that time.

markg85 commented 2 years ago

Have you found something? I spend a couple more hours without any success :(

phunkyfish commented 2 years ago

Same here. I will try again next weekend.

All I can think of to try now is it trace the code paths in kodi and then the addon and see where they differ when initially opening the stream.

markg85 commented 2 years ago

I am one - small - step further.

  if (!IsVideoReady())
  {
    m_packet.reset();
    DemuxPacket *pPacket = CDVDDemuxUtils::AllocateDemuxPacket(0);
    pPacket->demuxerId = m_demuxerId;
    return pPacket;
  }

Up till that function (in CDVDDemuxClient::Read()) you do actually have a frame. Yes, also a video frame. But after that function any frame you had is deleted because it enters that if statement.

This seems to be a point where the build in parser takes a different codepath. Both come in this function: CVideoPlayer::ReadPacket on this if-statement:

  if (m_pDemuxer)
    packet = m_pDemuxer->Read();

The build-in parser goes to CDVDDemuxFFmpeg::Read(). Your plugin goes via CDVDDemuxClient::Read()

Did this plugin ever work btw? Just curious because i've never seen anything work. Do you happen to have an m3u sample file that does work?

markg85 commented 2 years ago

Oh well, fast update, lol. Removing the if (!IsVideoReady()) check makes it work.... But that's internal Kodi code, not your plugin!...

Help, lol.

phunkyfish commented 2 years ago

Did this plugin ever work btw? Just curious because i've never seen anything work. Do you happen to have an m3u sample file that does work?

Yes, but it was not designed to open files. Just to support streams. Generally if it works in Kodi, it should in this add-on. You appear to have found an exception.

phunkyfish commented 2 years ago

Oh well, fast update, lol. Removing the if (!IsVideoReady()) check makes it work.... But that's internal Kodi code, not your plugin!...

Help, lol.

The same code is called from an addon or from kodi directly. Read is called constantly, removing that statement will cause serious issues BTW. The fix needed must be somewhere in the add-on, if its in Video Player we are in trouble as changes there are super hard to make.

markg85 commented 2 years ago

Oh well, fast update, lol. Removing the if (!IsVideoReady()) check makes it work.... But that's internal Kodi code, not your plugin!... Help, lol.

The same code is called from an addon or from kodi directly. Read is called constantly, removing that statement will cause serious issues BTW. The fix needed must be somewhere in the add-on, if its in Video Player we are in trouble as changes there are super hard to make.

Yeah, i'm taking your word for it that removing it will cause serious issues :) But i now have "something" working to continue fiddling with decryption.

I'll leave the actual fix in your capable hands. I hardly know anything about the internals of kodi and ffmpeg. Though that seems to be changing quite quickly with all the debugging, hehe

markg85 commented 2 years ago

So... this is super cool! With this patch: https://p.sc2.nl/nfkCy You'll get crypto support :D (not done yet, don't take the patch code as final)

Here's an example m3u file that works with it. Well, with that KODI change of disabling if (!IsVideoReady()) that is. And yes, i'm fully aware that there is a private key in the snippet below. That is OK in this very specific case.

#EXTM3U
#KODIPROP:inputstream=inputstream.ffmpegdirect
#KODIPROP:mimetype=application/x-mpegURL
#KODIPROP:inputstream.ffmpegdirect.crypto_key=b206ae6e28b1f6f5f0dc28899c4df346
#KODIPROP:inputstream.ffmpegdirect.crypto_iv=6cb5157f13fbc5f88ee54f33bcdc7826
#KODIPROP:inputstream.ffmpegdirect.is_realtime_stream=false
#KODIPROP:inputstream.ffmpegdirect.open_mode=ffmpeg
crypto:https://dweb.link/ipfs/bafybeigkmgo47w4nyzq3jim3hvucpbx37sam7lo6gymmtokyeopsoae64e

I'll obviously go through the proper review pull request stuff. I just wanted to post this as it's what i was aiming for all along when i started this debug adventure.

phunkyfish commented 2 years ago

Nice work. I did some tracing through the different paths this weekend (kodi vs the addon). I didn’t find the cause of the divergence yet but just a matter of time. Hopefully I’ll get some more time next week to look at it.

On 19 Oct 2021, at 23:22, Mark Gaiser @.***> wrote:

 So... this is super cool! With this patch: https://p.sc2.nl/nfkCy You'll get crypto support :D (not done yet, don't take the patch code as final)

Here's an example m3u file that works with it. Well, with that KODI change of disabling if (!IsVideoReady()) that is. And yes, i'm fully aware that there is a private key in the code below. That is OK in this very specific case.

EXTM3U

KODIPROP:inputstream=inputstream.ffmpegdirect

KODIPROP:mimetype=application/x-mpegURL

KODIPROP:inputstream.ffmpegdirect.crypto_key=b206ae6e28b1f6f5f0dc28899c4df346

KODIPROP:inputstream.ffmpegdirect.crypto_iv=6cb5157f13fbc5f88ee54f33bcdc7826

KODIPROP:inputstream.ffmpegdirect.is_realtime_stream=false

KODIPROP:inputstream.ffmpegdirect.open_mode=ffmpeg

crypto:https://dweb.link/ipfs/bafybeigkmgo47w4nyzq3jim3hvucpbx37sam7lo6gymmtokyeopsoae64e I'll obviously go through the proper review pull request stuff. I just wanted to post this as it's what i was aiming for all along when i started this debug adventure.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

markg85 commented 2 years ago

I really hope you'll get there :) I can't wait to submit this as a PR to get decrypting working in kodi!

Note, i'm also working on adding the IPFS and IPNS protocols in FFPMEG itself. It will take a while before that lands, but once it does i will make a patch for this project to add support for it too. Should be a simple patch though. But for that one i also plan to patch up kodi in the STRM files to have it working in there.

phunkyfish commented 2 years ago

Found a couple of problems today.

First is that kodi opens this as a file which means it doesn’t open it as using ffmpeg, I.e. directly using curl. This means that via kodi it could never using the ffmpeg protocols. So we have no working case of this kind of file working via ffmpeg.

Second is that removing if (!IsVideoReady()) just completely messes up the buffering, so it might allow you past the blocking point but there no way it’s a reasonable path forward.

markg85 commented 2 years ago

Found a couple of problems today.

Awesome!! Thank you for putting in some more time! We'll get there eventually 😁

First is that kodi opens this as a file which means it doesn’t open it as using ffmpeg, I.e. directly using curl. This means that via kodi it could never using the ffmpeg protocols. So we have no working case of this kind of file working via ffmpeg.

I don't get that. Kodi shouldn't open it using ffmpeg, right? Isn't that the whole point of this plugin to open it with the plugin and the extra ffmpeg things it allows setting? I don't see why Kodi should try ffmpeg too?

But even if Kodi does, ffmpeg allows playing http(s) urls. And I kinda vaguely remember seeing a supported list of protocols in Kodi (it's in your plugin too) that included http and https to allow for this via ffmpeg.

Second is that removing if (!IsVideoReady()) just completely messes up the buffering, so it might allow you past the blocking point but there no way it’s a reasonable path forward.

Oh for sure! I had no intention to even make a PR for this on the Kodi side. It was merely a way to move forward till we found the real issue.

phunkyfish commented 2 years ago

Kodi uses the same ffmpeg implementation internally. Files are opened with Curl and proxied to ffmpeg to decode. Streams are opened by ffmpeg directly which allows ffmpeg protocols to be used. So when the STRM file is used kodi open detects the file type and uses curl. This is when the file is successfully played properly.

What we need to do is open the file allowing ffmpeg to manage the file stream using the add-on. This would only work if ffmpeg supports opening this type of file directly. Luckily we can force kodi to open the stream directly using FFMPEG like this, so we know the add-on is the issue (or there is an issue with the inputstream interface, which I doubt):

#EXTINF:-1 tvg-chno="50",VP8 test
#KODIPROP:inputstream=inputstream.ffmpeg
#KODIPROP:mimetype=video/webm
https://dweb.link/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi

It's the inputstream.ffmpeg that lets us do this.

markg85 commented 2 years ago

If I understand that correctly then the example file that I tried to open using your plugin is now opened using the internal ffmpeg instead.

How do I expand that to also use that crypto stuff I made a patch for? I am still assuming that using your plugin is the proper path here, but it's it?

phunkyfish commented 2 years ago

If I understand that correctly then the example file that I tried to open using your plugin is now opened using the internal ffmpeg instead.

No, the patch is using the addon. The problem being kodi also needed to be patched to make it functional, we need to solve that.

How do I expand that to also use that crypto stuff I made a patch for? I am still assuming that using your plugin is the proper path here, but it's it?

The add-on is the right way to do this. Using kodi’s internal ffmpeg would not be accepted as it’s exposing parts of ffmpeg that should remain hidden.

markg85 commented 2 years ago

If I understand that correctly then the example file that I tried to open using your plugin is now opened using the internal ffmpeg instead.

No, the patch is using the addon. The problem being kodi also needed to be patched to make it functional, we need to solve that.

Oh awesome! Sounds like you solved it! Could you point me to the PR that fixes this? As i kinda need that that patched KODI to clean up and finalize my patch to support crypto.

How do I expand that to also use that crypto stuff I made a patch for? I am still assuming that using your plugin is the proper path here, but it's it?

The add-on is the right way to do this. Using kodi’s internal ffmpeg would not be accepted as it’s exposing parts of ffmpeg that should remain hidden.

Awesome, will keep doing that then!

phunkyfish commented 2 years ago

No sorry, not solved I’m afraid.

markg85 commented 2 years ago

I am getting a little bit confused now. Perhaps time to sum it up a bit and continue from there.

I thought this...:

#EXTINF:-1 tvg-chno="50",VP8 test
#KODIPROP:inputstream=inputstream.ffmpeg
#KODIPROP:mimetype=video/webm
https://dweb.link/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi

...was your way to get it working and let kodi play that file using m3u through your plugin. But that's not the case? Eh help, i'm confused.

Perhaps it's easier if you could sum up what needs to be patched where?

phunkyfish commented 2 years ago

No, what this does does is skip the addon completely and using the ffmpeg (not curl) in kodi core to open the file. All it proves is that’s it’s possible to to open it directly with ffmpeg.

The following entry is the one we need to work. I.e. using ffmpegdirect.

#EXTINF:-1 tvg-chno="50",VP8 test
#KODIPROP:inputstream=inputstream.ffmpegdirect
#KODIPROP:mimetype=video/webm
https://dweb.link/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi

Does this make sense?

markg85 commented 2 years ago

Yep, now it makes sense again! Thank you!

I'll see if i can figure out something more this weekend. But i surely don't mind if you beat me to it ;)

phunkyfish commented 2 years ago

A delay on my side. My laptop died. Will be a few weeks to get it fixed.

markg85 commented 2 years ago

Ahh, that sucks! Have fun reinstalling everything, that can be quite a challenge ;)

I'm debugging right now but i need a little bit of guidance i think. Could you tell me where that "inputstream.ffmpeg" is hidden (so not this addon but the one that is working). I can find "inputstream.adaptive" and some other "inputstream.*" ones but not a specific ffmpeg one.

Also, do you have some pointers for me for the most ideal place to start debugging? As i kinda don't get anywhere...

phunkyfish commented 2 years ago

Sure, you should look here: https://github.com/xbmc/xbmc/blob/master/xbmc/cores/VideoPlayer/DVDDemuxers/DVDDemuxFFmpeg.cpp

This code is not a separate inputstream add-on, it’s built into kodi itself. You’ll notice the code is very similar. Ffmpeg direct was my attempt to try running some of the feature set as an addon, also as I want to add some custom extensions.

The main difference with this code is that it can also call an inputstream addon, but the thought process is the same roughly.

As the code is similar you can debug it like you have debugged ffmpegdirect and hopefully find what is different

markg85 commented 2 years ago

After spending too many hours this weekend on it, and not getting anywhere, i'm about to give up... Which i don't want to because i really want to have encryption support! Do you have a chance to investigate it further?

Otherwise, are there more people that we can ask to have a look at this too?

Why is this issue this insanely difficult to figure out...

phunkyfish commented 2 years ago

I’m sure I can figure this out. But unfortunately not at the moment as I’m still waiting on my laptop being repaired.

Hopefully I’ll have it returned in the next couple of weeks.

markg85 commented 2 years ago

A ~month later. Just a friendly ping to keep this request alive :)

phunkyfish commented 2 years ago

Finally I have my laptop back so will look at this shortly.

On Sun, 12 Dec 2021 at 21:04, Mark Gaiser @.***> wrote:

A ~month later. Just a friendly ping to keep this request alive :)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/xbmc/inputstream.ffmpegdirect/issues/149#issuecomment-991970034, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDDKY2W3FRYQGUOG5W4KBTUQUE7PANCNFSM5FZAB2JA .

markg85 commented 2 years ago

That's good to hear! I'm really curious how fast you're going to figure out a proper patch and how many lines would change with it. In my experience these issues that take super long to debug (this issue certainly qualifies) are often one-liner patches...

phunkyfish commented 2 years ago

Yes, it’s likely some configuration required in the setup for ffmpeg or a permission that’s enabled by default in the kodi version. If it’s in the ffmpeg library that could expensive and testing changing requiring recompilation is expensive.

phunkyfish commented 2 years ago

@markg85 it appears the test stream is no longer working. Do you have a new one?

markg85 commented 2 years ago

Hi! Happy 2022! :)

You're still using this one i assume

https://dweb.link/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi

That link works just fine over here. If i open it in a browser it happily starts playing BigBuckBunny.

But just to be sure, here are some links to the same video file: https://dweb.link/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi https://ipfs.io/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi https://ipfs.sc2.nl/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi

They all work just fine for me. You can also play them in mpv (mpv <url>) or even just in a browser to be absolutely sure that the link is still working properly.

phunkyfish commented 2 years ago

The original URL would not work for me at all. It did work in VLC but took 10 seconds to start.

But the last URL you provided works OK, unfortunately still not via FFmpegdirect yet.

markg85 commented 2 years ago

But the last URL you provided works OK, unfortunately still not via FFmpegdirect yet.

You'll get there! You're the last hope ;)

phunkyfish commented 2 years ago

This is interesting. Via ffmpegdirect where it fails:

2022-01-07 16:48:03.830 kodi.bin[62187:11076201] 2022-01-07 16:48:03.830 T:11076201   DEBUG <general>: CVideoPlayerAudio - CDVDMsg::GENERAL_PAUSE: false
2022-01-07 16:48:03.830 kodi.bin[62187:11076179] 2022-01-07 16:48:03.830 T:11076179   DEBUG <general>: CVideoPlayer::SetCaching - caching state 2
2022-01-07 16:48:03.830 kodi.bin[62187:11076179] 2022-01-07 16:48:03.830 T:11076179   DEBUG <general>: CDVDClock::SetSpeedAdjust - adjusted:0.000000
2022-01-07 16:48:03.830 kodi.bin[62187:11076179] 2022-01-07 16:48:03.830 T:11076179   DEBUG <general>: CVideoPlayer::HandleMessages - player 2 reported state: 0
2022-01-07 16:48:03.830 kodi.bin[62187:11076179] 2022-01-07 16:48:03.830 T:11076179   DEBUG <general>: CVideoPlayer::HandleMessages - player 1 reported state: 0
2022-01-07 16:48:03.831 kodi.bin[62187:11076181] 2022-01-07 16:48:03.831 T:11076181   DEBUG <general>: OnAVChange: "CApplication::OnAVChange"
2022-01-07 16:48:03.831 kodi.bin[62187:11076179] 2022-01-07 16:48:03.831 T:11076179   DEBUG <general>: CDVDDemuxClient::ParsePacket - (0) profile changed from -99 to 0
2022-01-07 16:48:03.843 kodi.bin[62187:11076201] 2022-01-07 16:48:03.843 T:11076201   DEBUG <general>: CDVDAudio::Pause - pausing audio stream
2022-01-07 16:48:04.021 kodi.bin[62187:11048245] 2022-01-07 16:48:04.021 T:11048245   DEBUG <general>: Inhibiting OS screen saver
2022-01-07 16:48:05.955 kodi.bin[62187:11048245] 2022-01-07 16:48:05.955 T:11048245   DEBUG <general>: ------ Window Deinit (Pointer.xml) ------
2022-01-07 16:48:06.885 kodi.bin[62187:11048317] 2022-01-07 16:48:06.885 T:11048317   DEBUG <general>: CurlFile::Open - <http://192.168.1.201:80/web/currenttime>
2022-01-07 16:48:09.804 kodi.bin[62187:11076182] 2022-01-07 16:48:09.804 T:11076182   DEBUG <general>: CFileCache::Process - <https://ipfs.sc2.nl/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi> source read hit eof

If it hits an eof that would cause the stream to stop.

For the stream that works this occurs but not when it fails via ffmpegdirect: CDVDInputStreamFile::SetReadRate - set cache throttle rate to 290054 bytes per second

For comparison the stream working from kodi does not get this eof and it continues to set up the audio sink.

2022-01-07 16:49:21.313 kodi.bin[62187:11076830] 2022-01-07 16:49:21.313 T:11076830   DEBUG <general>: CVideoPlayerAudio - CDVDMsg::GENERAL_PAUSE: false
2022-01-07 16:49:21.313 kodi.bin[62187:11076809] 2022-01-07 16:49:21.313 T:11076809   DEBUG <general>: CVideoPlayer::SetCaching - caching state 1
2022-01-07 16:49:21.313 kodi.bin[62187:11076809] 2022-01-07 16:49:21.313 T:11076809   DEBUG <general>: CDVDClock::SetSpeedAdjust - adjusted:0.000000
2022-01-07 16:49:21.313 kodi.bin[62187:11076809] 2022-01-07 16:49:21.313 T:11076809   DEBUG <general>: CVideoPlayer::HandleMessages - player 2 reported state: 0
2022-01-07 16:49:21.313 kodi.bin[62187:11076809] 2022-01-07 16:49:21.313 T:11076809   DEBUG <general>: CVideoPlayer::HandleMessages - player 1 reported state: 0
2022-01-07 16:49:21.313 kodi.bin[62187:11076421] 2022-01-07 16:49:21.313 T:11076421   DEBUG <general>: OnAVChange: "CApplication::OnAVChange"
2022-01-07 16:49:21.313 kodi.bin[62187:11076809] 2022-01-07 16:49:21.313 T:11076809   DEBUG <general>: CVideoPlayer::SetCaching - caching state 1
2022-01-07 16:49:21.313 kodi.bin[62187:11076809] 2022-01-07 16:49:21.313 T:11076809   DEBUG <general>: CDVDClock::SetSpeedAdjust - adjusted:0.000000
2022-01-07 16:49:21.313 kodi.bin[62187:11076809] 2022-01-07 16:49:21.313 T:11076809   DEBUG <general>: CVideoPlayer::SetCaching - caching state 2
2022-01-07 16:49:21.313 kodi.bin[62187:11076809] 2022-01-07 16:49:21.313 T:11076809   DEBUG <general>: CDVDClock::SetSpeedAdjust - adjusted:0.000000
2022-01-07 16:49:21.313 kodi.bin[62187:11076821] 2022-01-07 16:49:21.313 T:11076821   DEBUG <general>: CVideoPlayerVideo - Stillframe left, switching to normal playback
2022-01-07 16:49:21.317 kodi.bin[62187:11076821] 2022-01-07 16:49:21.316 T:11076821    INFO <general>: CDVDVideoCodecFFmpeg::Open() Using codec: On2 VP8
2022-01-07 16:49:21.317 kodi.bin[62187:11076821] 2022-01-07 16:49:21.317 T:11076821   DEBUG <general>: CDVDVideoCodecFFmpeg - open frame threaded with 16 threads
2022-01-07 16:49:21.318 kodi.bin[62187:11076821] 2022-01-07 16:49:21.317 T:11076821   DEBUG <general>: CDVDVideoCodecFFmpeg - Updated codec: ff-vp8
2022-01-07 16:49:21.324 kodi.bin[62187:11076830] 2022-01-07 16:49:21.323 T:11076830   DEBUG <general>: CDVDAudio::Pause - pausing audio stream
2022-01-07 16:49:21.324 kodi.bin[62187:11076830] 2022-01-07 16:49:21.324 T:11076830   DEBUG <general>: CDVDAudio::Pause - pausing audio stream
2022-01-07 16:49:21.324 kodi.bin[62187:11076830] 2022-01-07 16:49:21.324 T:11076830   DEBUG <general>: CDVDAudio::Pause - pausing audio stream
2022-01-07 16:49:21.324 kodi.bin[62187:11076830] 2022-01-07 16:49:21.324 T:11076830   DEBUG <general>: ffmpeg[0x7feeef6b8600]: [opus] Could not update timestamps for skipped samples.
2022-01-07 16:49:21.324 kodi.bin[62187:11076830] 2022-01-07 16:49:21.324 T:11076830    INFO <general>: Creating audio stream (codec id: 86076, channels: 2, sample rate: 48000, no pass-through)
2022-01-07 16:49:21.324 kodi.bin[62187:11076830] 2022-01-07 16:49:21.324 T:11076830   DEBUG <general>: CVideoPlayerAudio:: synctype set to 0: clock feedback
2022-01-07 16:49:21.324 kodi.bin[62187:11048278] 2022-01-07 16:49:21.324 T:11048278    INFO <general>: CActiveAESink::OpenSink - initialize sink
2022-01-07 16:49:21.325 kodi.bin[62187:11048278] 2022-01-07 16:49:21.325 T:11048278   DEBUG <general>: CActiveAESink::OpenSink - trying to open device DARWINOSX:default
phunkyfish commented 2 years ago

So the issue could be that we ask ffmpeg to completely own managing the stream in ffmpegdorect but in Kodi it manages the stream and only uses ffmpeg to decode it.

markg85 commented 2 years ago

My first hunch when reading that is something somewhere in the plugin must be throwing it off.

Using ffmpeg directly, without any kodi, works fine too: ffplay https://ipfs.sc2.nl/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi plays

Could the plugin do something wrong in determining the eof?

phunkyfish commented 2 years ago

The plugin pretty much says "Hey, ffmpeg, you handle everything".

But we have more tools at our disposal now, I think iocontrol is now part of the add-on API so we can try it.

markg85 commented 2 years ago

Could it be that CFileCache::Process() gets some wrong information? Like perhaps the m_fileSize being 0? Or like m_pCache->CachedDataEndPosIfSeekTo(m_seekPos); not being filled yet before this point thus thinking it's empty?

Just guessing a bit. My kodi isn't running yet (updated my ffmpeg git and didn't recompile kodi yet) else i'd have a look.

phunkyfish commented 2 years ago

Hmmm, did appear to help, still get's stuck on:

2022-01-07 17:18:17.586 kodi.bin[63671:11089413] 2022-01-07 17:18:17.586 T:11089413 INFO <general>: running thread: CVideoPlayerAudio::Process()

Followed by: 2022-01-07 17:18:28.191 kodi.bin[63671:11089392] 2022-01-07 17:18:28.191 T:11089392 INFO <general>: Process - eof reading from demuxer