Merge AI Video commits into master

[j0sh]

The FPS / duration tests seem to have been written against a newer ffmpeg than what the LPMS AI Video branch is compiled with, which is causing them to fail locally, and for some reason CI is not running so we did not catch this earlier ... so let's port over the AI Video changes to master and work directly off master, to minimize further drift.

Cherry-picked the following from the ai-video branch:

• ffmpeg: Use helper to check for video metadata
• ffmpeg: Support image2 demuxer
• Add media duration to lpms_get_codec_info for GetCodecInfo (https://github.com/livepeer/lpms/pull/407[)](https://github.com/livepeer/lpms/commit/a17d3c6d4c522f582524ab1782c0fa673c8f9f31)
• https://github.com/livepeer/lpms/pull/414

Also since LPMS owns install_ffmpeg.sh now there is an install_ffmpeg.sh update in as the first commit, using the diff of the go-livepeer ai-video branch against master as a reference:

• install_ffmpeg: Compile in more stuff for AI

[j0sh]

Updated ffmpeg: Use helper to check for video metadata to correct some unit tests that had been failing all along - fixes in https://github.com/livepeer/lpms/pull/415/commits/5bfc051d802df7e6a95b96782c3b3caa9276de00

[j0sh]

This is passing for me locally, including with nvidia, however the failures in CI are consistent and a bit more concerning: it is always this one TestAPI_ConsecutiveMP4s

I would like to do a little more digging first before merging

Test cases passing locally:

josh@josh-gpu:~/aivideo-lpms$ time LD_LIBRARY_PATH=$HOME/compiled/ffmpeg-debug/lib PATH=$HOME/compiled/ffmpeg-debug/bin:$PATH PKG_CONFIG_PATH=$HOME/compiled/ffmpeg-debug/lib/pkgconfig go test -timeout 0 -tags nvidia ./...
?       github.com/livepeer/lpms/cmd/example    [no test files]
?       github.com/livepeer/lpms/cmd/example/simple     [no test files]
?       github.com/livepeer/lpms/cmd/transcoding        [no test files]
ok      github.com/livepeer/lpms/core   0.017s
?       github.com/livepeer/lpms/ffmpeg/proto   [no test files]
ok      github.com/livepeer/lpms/ffmpeg 1442.246s
ok      github.com/livepeer/lpms/segmenter      7.028s
ok      github.com/livepeer/lpms/stream 0.427s
ok      github.com/livepeer/lpms/transcoder     0.015s
ok      github.com/livepeer/lpms/vidlistener    1.587s
ok      github.com/livepeer/lpms/vidplayer      0.025s

real    24m11.747s
user    34m11.283s

only the known test that is incompatible with Ubuntu 24 fails

@rickstaa This should do the trick - from the test runner

echo '<?xml version="1.0" encoding="UTF-8"?><mime-info xmlns="http://www.freedesktop.org/standards/shared-mime-info"><mime-type type="video/mp2t"><comment>ts</comment><glob pattern="*.ts"/></mime-type></mime-info>' >> /usr/share/mime/packages/custom_mime_type.xml && update-mime-database /usr/share/mime

[rickstaa]

This is passing for me locally, including with nvidia, however the failures in CI are consistent and a bit more concerning: it is always this one TestAPI_ConsecutiveMP4s

I would like to do a little more digging first before merging

Test cases passing locally:

josh@josh-gpu:~/aivideo-lpms$ time LD_LIBRARY_PATH=$HOME/compiled/ffmpeg-debug/lib PATH=$HOME/compiled/ffmpeg-debug/bin:$PATH PKG_CONFIG_PATH=$HOME/compiled/ffmpeg-debug/lib/pkgconfig go test -timeout 0 -tags nvidia ./...
?       github.com/livepeer/lpms/cmd/example    [no test files]
?       github.com/livepeer/lpms/cmd/example/simple     [no test files]
?       github.com/livepeer/lpms/cmd/transcoding        [no test files]
ok      github.com/livepeer/lpms/core   0.017s
?       github.com/livepeer/lpms/ffmpeg/proto   [no test files]
ok      github.com/livepeer/lpms/ffmpeg 1442.246s
ok      github.com/livepeer/lpms/segmenter      7.028s
ok      github.com/livepeer/lpms/stream 0.427s
ok      github.com/livepeer/lpms/transcoder     0.015s
ok      github.com/livepeer/lpms/vidlistener    1.587s
ok      github.com/livepeer/lpms/vidplayer      0.025s

real    24m11.747s
user    34m11.283s

only the known test that is incompatible with Ubuntu 24 fails

@rickstaa This should do the trick - from the test runner

echo '<?xml version="1.0" encoding="UTF-8"?><mime-info xmlns="http://www.freedesktop.org/standards/shared-mime-info"><mime-type type="video/mp2t"><comment>ts</comment><glob pattern="*.ts"/></mime-type></mime-info>' >> /usr/share/mime/packages/custom_mime_type.xml && update-mime-database /usr/share/mime

Hey @j0sh take your time. I might merge this in the ai-video branch already to get a release out. I tested your command and it now works better but still gives an error:

--- FAIL: TestNvidia_ShortSegments (7.62s)
    api_test.go:521: Invalid argument
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x667020]

goroutine 341 [running]:
testing.tRunner.func1.2({0x6ccf80, 0x974350})
        /usr/local/go/src/testing/testing.go:1631 +0x24a
testing.tRunner.func1()
        /usr/local/go/src/testing/testing.go:1634 +0x377
panic({0x6ccf80?, 0x974350?})
        /usr/local/go/src/runtime/panic.go:770 +0x132
github.com/livepeer/lpms/ffmpeg.shortSegments(0xc0004ac340, 0x1, 0x6)
        /home/ricks/development/livepeer/ai/lpms/ffmpeg/api_test.go:523 +0x420
github.com/livepeer/lpms/ffmpeg.TestNvidia_ShortSegments(0xc0004ac340)
        /home/ricks/development/livepeer/ai/lpms/ffmpeg/nvidia_test.go:686 +0x79
testing.tRunner(0xc0004ac340, 0x75ec20)
        /usr/local/go/src/testing/testing.go:1689 +0xfb
created by testing.(*T).Run in goroutine 1
        /usr/local/go/src/testing/testing.go:1742 +0x390
FAIL    github.com/livepeer/lpms/ffmpeg 363.530s

I think it is related to my system as I'm on Ubuntu 24 so don't worry to much about it. I was just curious if there was a way to get the tests to pass on my system.

[j0sh]

@rickstaa On the error you are getting - from your previous logs from this one line it does seem like something with your GPU / driver setup, maybe it is hitting the encoder session limit?

OpenEncodeSessionEx failed: incompatible client key (21): (no details)

I will merge this to unblock AI and debug CI separately.

[j0sh]

@rickstaa merged. Since LPMS install_ffmpeg.sh was updated, the install_ffmpeg.sh in the go-livepeer AI Video branch will need to be updated to point to this - https://raw.githubusercontent.com/livepeer/lpms/bd7f8b07600446f018eab6075c22601aa3301bf6/install_ffmpeg.sh

[rickstaa]

@rickstaa merged. Since LPMS install_ffmpeg.sh was updated, the install_ffmpeg.sh in the go-livepeer AI Video branch will need to be updated to point to this - https://raw.githubusercontent.com/livepeer/lpms/bd7f8b07600446f018eab6075c22601aa3301bf6/install_ffmpeg.sh

Hey @j0sh thanks for the update. I rebased our go-livepeer/ai-video branch so this should be done (see https://github.com/livepeer/go-livepeer/commit/016259f52a0bd4849a23472ecff779fa48c378bd) 👍🏻.