AMR: Detect it within FFmpeg at Runtime. - Githubissues

jitsi / libjitsi

Advanced Java media library for secure real-time audio/video communication.

Apache License 2.0

628 stars 281 forks source link

AMR: Detect it within FFmpeg at Runtime. #498

Closed traud closed 4 years ago

traud commented 4 years ago

OpenH264 is an extension to FFmpeg. The bundled FFmpeg for Microsoft Windows includes support for the video codec H.264. However, that bundled FFmpeg does not come with the audio codec AMR(-WB). The change for OpenH264 simply disabled AMR-WB in general, at compile time (see #403 Review 97453424). Beside losing AMR-WB, that change was incomplete because AMR-WB is still added as a media format. For example, in Jitsi Desktop, AMR-WB is still selectable but non-functional.

This change here re-enables AMR-WB but detects whether the used FFmpeg supports AMR-WB, at runtime. Consequently, when FFmpeg is missing or when the used FFmpeg does not support AMR-WB, it does not show up in Jitsi Desktop anymore. When FFmpeg is present and when AMR-WB is supported, AMR-WB is listed and functional. On Linux, this allows the user to add/remove just AMR-WB (or even the whole FFmpeg) within the system without re-compiling Jitsi Desktop.

jitsi-jenkins commented 4 years ago

Hi, thanks for your contribution! If you haven't already done so, could you please make sure you sign our CLA (https://jitsi.org/icla for individuals and https://jitsi.org/ccla for corporations)? We would unfortunately be unable to merge your patch unless we have that piece :(.

damencho commented 4 years ago

The default logs that go in a file. Locations of those can be found in the faq.

traud commented 4 years ago

Ahh, I think, I know what you are about. I thought you are about whether these classes are still loaded, can be traced on the stack. However, you are about whether an Error/Exception is thrown which spams a default log file with a stack trace as error message.

Furthermore with ‘default’ logs, I guess, you are about the Logs in Jitsi Desktop → Menu → Tools → Options → Advanced → Logging → Archive Logs. This is the same as can be found in the file system via ~/.jitsi/log. Alternatively, one can edit the file logging.properties within the folder lib and change the ConsoleHandler from SEVERE to INFO. Then, the console prints the same as what is archived in the log files. For testing, I changed that to ALL because calling Jitsi Desktop with the parameter --debug did nothing.

After my Pull Request, there is a Throwable because of assertFindAVCodec(.). When removed, the log is not spammed anymore. Anyway, that issue existed before OpenH264 and therefore is an independent issue. Furthermore, that affects the audio codec MP3 as well. Shall I simply remove assertFindAVCodec(.) and update my Pull Request? Or are you creating a new Pull Request which removes that assertFindAVCodec(.) completely?

damencho commented 4 years ago

Nope, assertFindAVCodec should be there. I want to fix that, as the default is not to have AMR and will fill logs with those errors. Maybe it existed with some other codecs but those normally are there by default and do not throw that error. I still cannot find time to look into it, will try to squeeze it this or next week.

traud commented 4 years ago

assertFindAVCodec(.) is only used for AMR-WB and MP3. It does not make sense for AMR-WB anymore. Not sure what’s is use for MP3.

damencho commented 4 years ago

The idea is to not add that codec to the list of possible codecs when there is no ffmpeg part of that codec. This thing that I wanted to do earlier, not even try doing that ...

damencho commented 4 years ago

Do you have an example stack trace for the RuntimeException coming from assertFindAVCodec?

traud commented 4 years ago

That RuntimeException throws an ExceptionInInitializerError:

     [java] WARNING: [19] org.jitsi.impl.neomedia.codec.FMJPlugInConfiguration.log() Codec org.jitsi.impl.neomedia.codec.audio.amrwb.JNIDecoder is NOT successfully registered
     [java] java.lang.ExceptionInInitializerError
     [java]     at org.jitsi.impl.neomedia.codec.audio.amrwb.JNIDecoder.<init>(JNIDecoder.java:48)
     [java]     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
     [java]     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
     [java]     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
     [java]     at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
     [java]     at java.lang.Class.newInstance(Class.java:442)
     [java]     at org.jitsi.impl.neomedia.codec.FMJPlugInConfiguration.registerCustomCodecs(FMJPlugInConfiguration.java:253)
     [java]     at org.jitsi.impl.neomedia.MediaServiceImpl.setupFMJ(MediaServiceImpl.java:1624)
     [java]     at org.jitsi.impl.neomedia.MediaServiceImpl.<clinit>(MediaServiceImpl.java:253)
     [java]     at java.lang.Class.forName0(Native Method)
     [java]     at java.lang.Class.forName(Class.java:264)
     [java]     at org.jitsi.impl.libjitsi.LibJitsiImpl$ServiceLock.initializeService(LibJitsiImpl.java:173)
     [java]     at org.jitsi.impl.libjitsi.LibJitsiImpl$ServiceLock.getService(LibJitsiImpl.java:131)
     [java]     at org.jitsi.impl.libjitsi.LibJitsiImpl.getService(LibJitsiImpl.java:91)
     [java]     at org.jitsi.impl.libjitsi.LibJitsiOSGiImpl.getService(LibJitsiOSGiImpl.java:95)
     [java]     at org.jitsi.service.libjitsi.LibJitsi.invokeGetServiceOnImpl(LibJitsi.java:172)
     [java]     at org.jitsi.service.libjitsi.LibJitsi.getMediaService(LibJitsi.java:124)
     [java]     at net.java.sip.communicator.impl.neomedia.NeomediaActivator.start(NeomediaActivator.java:391)
     [java]     at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:697)
     [java]     at org.apache.felix.framework.Felix.activateBundle(Felix.java:2240)
     [java]     at org.apache.felix.framework.Felix.startBundle(Felix.java:2146)
     [java]     at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1373)
     [java]     at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
     [java]     at java.lang.Thread.run(Thread.java:748)
     [java] Caused by: java.lang.RuntimeException: Could not find FFmpeg encoder 0x12001!
     [java]     at org.jitsi.impl.neomedia.codec.audio.FFmpegAudioEncoder.assertFindAVCodec(FFmpegAudioEncoder.java:52)
     [java]     at org.jitsi.impl.neomedia.codec.audio.amrwb.JNIEncoder.<clinit>(JNIEncoder.java:35)
     [java]     ... 24 more
     [java] WARNING: [19] org.jitsi.impl.neomedia.codec.FMJPlugInConfiguration.log() Codec org.jitsi.impl.neomedia.codec.audio.amrwb.JNIEncoder is NOT successfully registered
     [java] java.lang.NoClassDefFoundError: Could not initialize class org.jitsi.impl.neomedia.codec.audio.amrwb.JNIEncoder
     [java]     at java.lang.Class.forName0(Native Method)
     [java]     at java.lang.Class.forName(Class.java:264)
     [java]     at org.jitsi.impl.neomedia.codec.FMJPlugInConfiguration.registerCustomCodecs(FMJPlugInConfiguration.java:253)
     [java]     at org.jitsi.impl.neomedia.MediaServiceImpl.setupFMJ(MediaServiceImpl.java:1624)
     [java]     at org.jitsi.impl.neomedia.MediaServiceImpl.<clinit>(MediaServiceImpl.java:253)
     [java]     at java.lang.Class.forName0(Native Method)
     [java]     at java.lang.Class.forName(Class.java:264)
     [java]     at org.jitsi.impl.libjitsi.LibJitsiImpl$ServiceLock.initializeService(LibJitsiImpl.java:173)
     [java]     at org.jitsi.impl.libjitsi.LibJitsiImpl$ServiceLock.getService(LibJitsiImpl.java:131)
     [java]     at org.jitsi.impl.libjitsi.LibJitsiImpl.getService(LibJitsiImpl.java:91)
     [java]     at org.jitsi.impl.libjitsi.LibJitsiOSGiImpl.getService(LibJitsiOSGiImpl.java:95)
     [java]     at org.jitsi.service.libjitsi.LibJitsi.invokeGetServiceOnImpl(LibJitsi.java:172)
     [java]     at org.jitsi.service.libjitsi.LibJitsi.getMediaService(LibJitsi.java:124)
     [java]     at net.java.sip.communicator.impl.neomedia.NeomediaActivator.start(NeomediaActivator.java:391)
     [java]     at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:697)
     [java]     at org.apache.felix.framework.Felix.activateBundle(Felix.java:2240)
     [java]     at org.apache.felix.framework.Felix.startBundle(Felix.java:2146)
     [java]     at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1373)
     [java]     at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
     [java]     at java.lang.Thread.run(Thread.java:748)

damencho commented 4 years ago

Thanks, this makes it easier for me. I will try getting to it tomorrow.