bkauler commented 2 years ago

Success, #44 is now solved, source downloaded on January 3, cross-compiled, and now starts:

lives-20220103-startup-checks

However, I tried opening a very small video, and the "Opening audio" window never ends:

lives-20220103-open-audio

I tested executing directly:

# LANGUAGE=en LANG=en /usr/bin/mpv --no-config --quiet --no-video --vd-lavc-threads=4 --frames=3 --ao=pcm --ao-pcm-waveheader=no --audio-format=s16 --audio-channels=2 "/mnt/sda1/warehouse/misc/media-test/media-test-video/barsandtone.flv"
Compiled without libass.
There will be no OSD and no text subtitles.
     Video --vid=1 (vp6f 360x288 10.000fps)
 (+) Audio --aid=1 (mp3 2ch 44100Hz)
[ao/pcm] File: audiodump.pcm (RAW PCM)
[ao/pcm] PCM: Samplerate: 44100 Hz Channels: 2 Format: s16
AO: [pcm] 44100Hz stereo 2ch s16

Exiting... (End of file)
mpv: ../mpv-0.32.0/ta/ta.c:335: ta_dbg_check_header: Assertion `h->canary == CANARY' failed.
Aborted
#

mpv does play the video ok, like this:

# mpv barsandtone.flv 
Compiled without libass.
There will be no OSD and no text subtitles.
 (+) Video --vid=1 (vp6f 360x288 10.000fps)
 (+) Audio --aid=1 (mp3 2ch 44100Hz)
AO: [pulse] 44100Hz stereo 2ch float
VO: [gpu] 360x288 yuv420p
AV: 00:00:00 / 00:00:06 (0%) A-V:  0.000

Exiting... (End of file)
#

The video file is only 88KB, will attach it, not sure if can attach a .flv file, so appending a false ".gz" onto the filename:

barsandtone.flv.gz

salsaman commented 2 years ago

Seems to be an mpv issue: https://github.com/mpv-player/mpv/issues/7644https://github.com/mpv-player/mpv/issues/7644

salsaman commented 2 years ago

I recommend installing mplayer - mpv support is really only provided for limited compatibility. The test file provided opens with no issues using mplayer to separate the audio track.

bkauler commented 2 years ago

No, I won't be using mplayer. For some years, mpv has been emerging as the choice by users and distribution and package creators. For example, this site is typical:

https://www.slant.co/versus/13972/13973/~mplayer_vs_mpv

Quoting:

When comparing MPlayer vs MPV, the Slant community recommends MPV for most people. In the question“What are the best audio players for Linux?” MPV is ranked 20th while MPlayer is ranked 43rd.

Also, there is a perception that the mpv project is more active. I just checked, the latest release of mplayer is 1.4, released on 18/4/2019. MPV on the other hand, latest release yesterday.

I can't see how to browse mplayer commits, their svn does not offer a http interface.

Thinking back, years ago I changed to mpv, I seem to recall, it played more video formats, and played them properly.

bkauler commented 2 years ago

Found the mplayer commits, not much activity:

https://www.openhub.net/p/mplayer/commits

I seem to recall, it is only happy with it's own internal ffmpeg, another reason I went over to mpv.

A rather strong assessment of mplayer vs mpv:

https://github.com/mpv-player/mpv/issues/4510

In short: mpv is alive and throws out old broken code, mplayer is dead and clings onto old broken code like a hoarder.

One important change is that mpv, unlike mplayer, does not copy ffmpeg code into its codebase internally and update it once in a blue moon, but actually properly uses ffmpeg.

salsaman commented 2 years ago

This discussion is probably a better fit for the lives-devel mailing list. mpv lacks (or did lack) certain features which LiVES delegates specifically to mplayer, namely: firewire playback (fw://), tvcard playback, dvd playback, playing remote (e.g. rtsp) streams.

In addition, the fact that there are fewer changes is actually a benefit, since it means that the command line options don't require continual updates as is the case with mpv.

As regards the other tasks which LiVES uses it for - ripping the audio track(s), playing instant previews in the file chooser, as a fallback when all the decoder plugins fail, playing remote streams, and converting audio formats to wav, mplayer is more than adequate.

I am not sure what they are referring to, but at least for these uses I am not aware of any "broken code" in mplayer, in fact as can be seen from this current issue, mpv may throw out "broken code" but at the cost of introducing new bugs.

I'm happy to accept patches if you or somebody else wants to keep track of the continual updates to the mpv command line options.

Nevertheless, as I have stated above, that time would be better spent removing the need to rely on external binaries and instead incorporate the features into LiVES, using libav* and others directly, and reimplementing the specific items mentioned as standalone plugins.

This would involve:

create an audio decoder plugin which takes a URL, start offset and length, and produces a raw pcm output, returning channel layout, rate, format, etc. It should also have support for language / format detection and selection.
create a subtitle ripper plugin - similar to audio, but output would be an external text file in srt or sub format.
update the player code in LiVES so it can silently open the selected part of a clip, and embed the player widget in the file preview window (actually not too hard to implement, although the player code would need updating to handle playing standalone audio clips).
create a plugin which can convert audio formats to raw pcm. Possibly this could be the same audio ripper plugin mentioned earlier.
create plugins for playing from firewire, tvcards and dvd (impossible for me, as I have not got hardware to test with)
create a plugin capable of decoding remote streams (e.g. rtsp)
update the libav_format decoder and others so that they can handle every format which mplayer / mpv can decode.

None of these is particularly difficult, but they each require some knowledge of media formats and in some cases, specific hardware to test with, as well as an investment of time, which I would prefer to spend improving the core of LiVES.

Hence, for now, the recommendation is just to install mplayer and not get too concerned about whether mpv is superior for general use; for the specific purposes required by LiVES, mplayer is by far the better choice.

bkauler commented 2 years ago

Previous testing was with mpv 0.32.0. I have now compiled the latest version, 0.34.1, and this time included libass dependency. Same test that failed before, now works:

# LANGUAGE=en LANG=en /usr/bin/mpv --no-config --quiet --no-video --vd-lavc-threads=4 --frames=3 --ao=pcm --ao-pcm-waveheader=no --audio-format=s16 --audio-channels=2 barsandtone.flv 
     Video --vid=1 (vp6f 360x288 10.000fps)
 (+) Audio --aid=1 (mp3 2ch 44100Hz)
[ao/pcm] File: audiodump.pcm (RAW PCM)
[ao/pcm] PCM: Samplerate: 44100 Hz Channels: 2 Format: s16
AO: [pcm] 44100Hz stereo 2ch s16

Exiting... (End of file)
#

However, when I opened a Big Buck Bunny 1080p mp4, there is no audio, the speaker icon on the UI shows as muted, and nothing on the audio tracks. Using pulseaudio, and pavucontrol shows lives audio level is ok.

salsaman commented 2 years ago

Check in Preferences / Playback and make sure the audio is set to pulseaudio.

It's normal for the mute button to be visible, it should be partially transparent. If you hit the 'z' key you should see it toggle on and off. You can also set the state from the Play menu. The only time it should be insensitrive is if playing with a non-realtime audio player.

Also you can try this: click on menu option Advanced, and enable Developer Mode. This will show extra debug info and enable various hidden features, so it is a good idea when testing anyway.

With this option enabled, if you now click on Info / Show Clip Info, in the title bar of the dialog that opens, you will see the subdirectory in the working directory which holds the actual clip files. If you then examine the contents of this directory you will be able to determine if there are any files starting with audio*.. In addition, in the clip info window, there may be other useful info such as whether it was opened using a decoder plugin, audio track details (if any), etc.

You can also check the same directory in /run/shm/, the way LiVES works currently, the audio is decoded in shared memory then "migrated" into the actual directory. Since this is quite a new feature, it is possible (though unlikely) that the audio file was never migrated correctly.

As a final step, you can try going into Preferences / Startup / Reset and do a clean restart, which will eliminate any possibility that something went wrong during the initial setup.

salsaman commented 2 years ago

Also, when running the mpv test you mention above, please check to ensure that it does actually create the audiodump.pcm file as suggested.

bkauler commented 2 years ago

One thing that I should have mentioned, when open the Big Buck Bunny mp4, a window pops up stating "Inserting Silence" -- so that would be why no audio!

I did a Reset from Preferences, and when opened Big Buck Bunny it did that "Inserting Silence...". However, that other video, see #47 , does have the sound.

salsaman commented 2 years ago

You can try going into Preferences / Decoding, and uncheck "Automatic Trimming / Padding of Audio". The reason you are seeing that is because the audio track is shorter than the video, and LiVES is padding it with silence to match. That also suggests that some audio is being detected, so perhaps it is failing in the auto padding step.

bkauler commented 2 years ago

You can try going into Preferences / Decoding, and uncheck "Automatic Trimming / Padding of Audio". The reason you are seeing that is because the audio track is shorter than the video, and LiVES is padding it with silence to match. That also suggests that some audio is being detected, so perhaps it is failing in the auto padding step.

It is weird. With that option checked, I open the video and play it, the entire video plays, 717 frames, but no audio.

If I uncheck that option, then open the video. although it shows 717 frames, only about 1 second of video plays when I click the Play button, and that 1 second does have sound. The video track shows a line right to the end, but the two audio lines are only just over 1 second long then end.

The video file is only the beginning of the Big Buck Bunny cartoon, 22.7MB. I could upload it somewhere if that will help. It plays ok in all video players.

bkauler commented 2 years ago

I opened the Big Buck Bunny video in Avidemux, it plays perfectly. This is the info from Avidemux about the file:

===================================================== Video

Codec 4CC: H264 Image Size: 1920 x 1080 Aspect Ratio: 1:1 (1:1) Frame Rate: 24.000 fps Average Bitrate: 5844 kbps Total Duration: 00:00:29.999

===================================================== Video Codec Extradata

Size: 44 Extradata: 01 64 00 28 FF E1 00 1B 67 64 00 28 AC D9 40 78 02 27 E5 C0 44 00 00 03 00 04 00 00 03 00 C0 3C 60 C6 58 01 00 06 68 EB E3 CB 22 C0

===================================================== Audio (1 active track)

Codec: AAC Channels: 6 Bitrate: 24027 Bps / 192 kbps Frequency: 48000 Hz Total Duration: 00:00:30.078

...it shows the audio as slightly longer than the video.

bkauler commented 2 years ago

I have temporarily uploaded the Big Buck Bunny video here, with a false ".gz" appended to the filename (21.7MB):

https://distro.ibiblio.org/easyos/source/alphabetical/b/big-buck-bunny-1080p-24fps.mp4.gz

salsaman commented 2 years ago

Seems like there is a bug where the silence is replacing the original audio rather than being appended to it. I'll look into it.

salsaman commented 2 years ago

It would be useful if you could upload the clip, yes.

salsaman commented 2 years ago

Thanks, I missed that. When I open it with auto trim / pad disabled, I get approx. 10 seconds of audio and 30 seconds of video. I think what is confusing things is that the original clip has 6 channel audio which is being downmixed to 2. My guess that LiVES still assumes there aree six channels, thus it is playing 2 samples, skipping 4 samples, playing 2, etc. Something like that. Then when auto pad is on, it tries to insert the "missing" 20 seconds, but for some reason in this case the new audio is replacing the old instead of being inserted before - possibly it is getting rejected due to having too many audio channels.

salsaman commented 2 years ago

Fixed. aeb35131d4cd7337b74ae18549f8a1ee80fdb290

bkauler commented 2 years ago

Now using latest lives source.

With "Automatic trimming/padding of audio when possible", that bigbuckbunny mp4 file shows video 29.88sec, left 28.51sec, right 28.51 sec. Play the video it is truncated at 28.51sec and there is no sound.

With that setting unchecked, video is 29.88sec, left 1.37sec, right 1.37sec. Play the video, only plays for 1.37sec, truncated. This time do get audio, but only 1.37sec, again truncated.

salsaman commented 2 years ago

Appears to be caused by the same error as https://github.com/salsaman/LiVES/issues/47

The solution is to install mplayer and edit ~/.config/lives/settings replacing mpv with mplayer.

Closing as it is an mpv problem.

salsaman commented 2 years ago

Regarding earlier comments, I am not sure where this idea that mplayer uses some kind of incompatible libraries comes from. If you run 'ldd /usr/bin/mplayer', you will see amongst the list of linked libraries, something like the following:

libswscale.so.5 => /lib/x86_64-linux-gnu/libswscale.so.5 (0x00007fad27a34000)
libswresample.so.3 => /lib/x86_64-linux-gnu/libswresample.so.3 (0x00007fad27a12000)
libavformat.so.58 => /lib/x86_64-linux-gnu/libavformat.so.58 (0x00007fad2779a000)
libavcodec.so.58 => /lib/x86_64-linux-gnu/libavcodec.so.58 (0x00007fad261d9000)
libavutil.so.56 => /lib/x86_64-linux-gnu/libavutil.so.56 (0x00007fad260b0000)
libpostproc.so.55 => /lib/x86_64-linux-gnu/libpostproc.so.55 (0x00007fad2608e000)

which are just the standard libav* libraries, in fact the exact same ones which mpv uses.

bkauler commented 2 years ago

Please see my reply at #47

bkauler commented 2 years ago

I have just checked the debian repo, yes, they have compiled mplayer with system ffmpeg libraries, so same as mpv. It used to be that mplayer was only happy with its own inbuilt ffmpeg, but that is obviously out-of-date information.

The debian mplayer pkg is 2.2MB, so that it the overhead I would be introducing if decide to include it builtin to a build of EasyOS. Not too bad. It would be real nice if the problem with mpv could be solved though!

salsaman commented 2 years ago

I'll do my best. Apologies for misinterpreting your earlier comments. I am in the process of moving home and setting up a new workspace.

salsaman / LiVES

Opening audio never ends #46

===================================================== Video

===================================================== Video Codec Extradata

===================================================== Audio (1 active track)