Video doesn't play on iOS or Mac after compress

[bipinvaylu]

I am using this library for one of my project and I discover one issue when i compress video using this library and try to play on iPhone or Safari browser.

Original video: https://drive.google.com/file/d/1MXmwkjxB8WKX6_Rf78JXZ1E6W4yg639a/view?usp=sharing Compressed video: https://drive.google.com/file/d/1zdX8gUgrEoGttWifHh_2JZP3hZqncSq7/view?usp=sharing

When we try to play this video in quick time player, It showing following error alert:

• LightCompressor version: 0.7.8
• Device: Nokia 6.1 Plus - 10.0
• Code:

  
  val sourceFile = File(video.url)
  val destinationFile =
  File(context.cacheDir.absolutePath + "/" + sourceFile.name)

VideoCompressor.start( srcPath = sourceFile.path, destPath = destinationFile.path, quality = VideoQuality.HIGH, isMinBitRateEnabled = false, listener = object : CompressionListener { override fun onCancelled() { if (isCancelled) return isCancelled = true emitter.onError(UnknownErrorException("Video compression process cancelled")) emitter.onComplete() }

    override fun onFailure(failureMessage: String) {
        VideoCompressor.cancel()
        if (isCancelled) return
        isCancelled = true
        emitter.onError(UnknownErrorException(failureMessage))
        emitter.onComplete()
    }

    override fun onProgress(percent: Float) {
        //Do nothing
    }

    override fun onStart() {
        //Do nothing
    }

    override fun onSuccess() {
        if (isCancelled) return
        compressedVideoMediaList.add(Video(destinationFile.path))
        if (videoMediaList.size == compressedVideoMediaList.size) {
            emitter.onNext(compressedVideoMediaList.toList())
            emitter.onComplete()
        }
    }

},

)

@AbedElazizShe Can you please help me here to figure out this issue? Let me know if you need any other details from my end.

[rickrl1]

Can confirm that this issue also occurs in the latest version 0.8.0 (worked fine in version 0.7.4). I have tried LOW and VERY_LOW quality level and the result is the same

[AbedElazizShe]

@rickrl1 @bipinvaylu thank you for the details and opening the issue.

@bipinvaylu could you please try versions 0.7.8 and/or 0.7.4 and let me know if you have the same problem? This could help me know what went wrong in 0.8.0 which contains a major change in transcoding the video.

Thanks in advance.

[rickrl1]

I tried 0.7.8 and it doesn't compress correctly at all - it just creates in a file that is 32 bytes. In 0.7.4, it compresses correctly and I am able to play the resulting video back on Android and iOS (Safari). In 0.8.0, it compresses correctly and I am able to play the resulting video on Android but it doesn't play back on iOS (Safari).

Hope this provides a bit more insight into what the issue may be.

[rickrl1]

FYI, 0.7.9 behaves the same as 0.8.0 i.e. can playback on Android but not iOS/Mac after compression

[bipinvaylu]

Hey @AbedElazizShe I just checkout v0.7.4 as mentioned by @rickrl1 and that is also not working for me. It plays video fine on Android device but not playing on iOS device.

[AbedElazizShe]

@rickrl1 @bipinvaylu thank you for all the useful details. Media codec is really tricky. I fix something in a set of devices causing others to fail. Have been experimenting for sometime now, that's why the library never reached version 1.0.0 yet. I will keep investigating and try to fix the issue in all devices, but as you can imagine, this might take some time.

Thank you a lot. Feel free to contribute to the project if you find a fix.

[rickrl1]

Thanks @AbedElazizShe. A bit more info....it seems that the result is also dependent on the Android version. I had a report where the compression wasn't working at all on a particular Android phone (not sure which one) but after that phone had its Android version updated (not if 10 or 11), compression was successful and I could also view the video on iOS/Mac. This is for version 0.8.0. But as I said earlier, version 0.8.0 worked on my older Android (Android 9, Samsung SMJ530-Y) but I could not view the video on iOS/Mac

I have experienced the difficulties with developing for Android in general so understand that trying to get the codec working on all devices and all Android versions is a very very difficult task.

[rickrl1]

Hi again, just a bit more info. Tried 0.8.0 again on my j5 Pro (Android 9) and when trying to play the compressed video in QuickTime player (on Mac), I do get the same alert as @bipinvaylu has shown above, but if I open the file anyway, it plays fine. The file doesn't play inside the Safari browser. Not sure if this helps at all.

[AbedElazizShe]

@rickrl1 @bipinvaylu I have resolved the issue. The problem is not related to the Android version rather than a specific encoder; c2.qti.avc.encoder. I have resolved this and tested on Mac as well. I have 70 more devices to test on before releasing the new version. Will do my best to finish that tomorrow.

[AbedElazizShe]

Issue fixed in version 0.8.1. Please let me know if you still have the issue. You can close the task if the problem is solved.

[oleh-mryhlod]

Hi @AbedElazizShe For me, compression isn't working anymore on my google pixel 4 device because of decoder c2.android.avc.decoder. Maybe the output from logcat related to the decoder will help:

D/CCodec: allocate(c2.android.avc.decoder)
I/CCodec: Created component [c2.android.avc.decoder]
I/CODECS decoder NAME:: c2.android.avc.decoder
I/MediaCodec: [c2.android.avc.decoder] setting surface generation to 17990657
D/CCodec: [c2.android.avc.decoder] buffers are bound to CCodec for this session
D/CCodecBufferChannel: [c2.android.avc.decoder#605] Created input block pool with allocatorID 16 => poolID 17 - OK (0)
D/CCodecBufferChannel: [c2.android.avc.decoder#605] Query output surface allocator returned 0 params => BAD_INDEX (6)
I/CCodecBufferChannel: [c2.android.avc.decoder#605] Created output block pool with allocatorID 18 => poolID 2267 - OK
D/CCodecBufferChannel: [c2.android.avc.decoder#605] Configured output block pool ids 2267 => OK
D/CCodecBuffers: [c2.android.avc.decoder#605:1D-Input.Impl[N]] codec released a buffer owned by client (index 1)
D/CCodecBuffers: [c2.android.avc.decoder#605:1D-Input.Impl[N]] codec released a buffer owned by client (index 2)
D/CCodecBuffers: [c2.android.avc.decoder#605:1D-Input.Impl[N]] codec released a buffer owned by client (index 0)
D/CCodecBuffers: [c2.android.avc.decoder#605:1D-Input.Impl[N]] codec released a buffer owned by client (index 1)
D/CCodecBuffers: [c2.android.avc.decoder#605:Output[N]] popFromStashAndRegister: output format changed to AMessage(what = 0x00000000) = {
      int32_t android._color-format = 2135033992
      int32_t android._video-scaling = 1
      Rect crop(0, 0, 1919, 1079)
      int32_t color-standard = 2
      int32_t color-range = 2
      int32_t color-transfer = 3
      int32_t height = 1080
      int32_t max-height = 240
      int32_t max-width = 320
      string mime = "video/raw"
      int32_t rotation-degrees = 90
      int32_t sar-height = 1
      int32_t sar-width = 1
      int32_t width = 1920
      int32_t android._dataspace = 258
      int32_t color-format = 2130708361
    }
D/CCodecBufferChannel: [c2.android.avc.decoder#605] MediaCodec discarded an unknown buffer
D/CCodecBufferChannel: [c2.android.avc.decoder#605] MediaCodec discarded an unknown buffer

Also I've tried different encoders and decoders, and seems like there is a working solution (compression actually works and the otput is playable on Mac and iOS): encoder still can be c2.android.avc.encoder but decoder should be as it was before - MediaCodec.createDecoderByType(inputFormat.getString(MediaFormat.KEY_MIME)!!) and the result for the decoder on Pixel 4 is c2.qti.hevc.decoder

But using encoder c2.android.avc.encoder slows down the compression, e.g. to compress 30sec video - 10sec, but to compress 30 sec video using c2.qti.avc.encoder - 6sec. But if using c2.qti.avc.encoder there is the issue with that green/black screen on iOS ¯_(ツ)_/¯

[AbedElazizShe]

@oleh-mryhlod thank you for thevery useful details. It took me weeks actually to figure out the issue but haven't tried your solution, I will try that and release a bug fix as soon as possible. Thank you.

[oleh-mryhlod]

Hi @AbedElazizShe I also tried a lib called LiTr https://github.com/linkedin/LiTr (someone mentioned it in one of the previous issues related to the green screen) - and it works fine, videos play fine on ios/mac after compression on pixel 4 android 11. And I noticed that it also chooses that c2.qti.avc.encoder and c2.qti.hevc.decoder on pixel 4 to compress video (and that codes are the fastest ones for pixel 4) and compressed video can be played on ios/mac. They have some utils to choose the encoder/decoder. So I don't know, the problem shouldn't be in codecs, I guess. I saw they are using MediaMuxer and it works, but I found some old answer on StackOverflow that using MediaMuxer causes that green screen on ios - https://stackoverflow.com/a/34487086 , but maybe now it changed and MediaMuxer should be used?)) Also, Telegram had the same issue 2-3 years ago if sending video from pixel 3.

[AbedElazizShe]

@oleh-mryhlod I changed the decoder as you mentioned and it seems fine, my videos compressed using my pixel4a are playing fine on windows and mac. I will release a new version shortly. I'd appriciate if you could try it.

[ashotterekyanlmc]

Still didn't work on version 1.0.0. Replaced line

val encoder = if (hasQTI) { MediaCodec.createByCodecName("c2.android.avc.encoder") } else { MediaCodec.createEncoderByType(MIME_TYPE) }

with val encoder = MediaCodec.createEncoderByType(MIME_TYPE)

now all is good