iOS app crashes when trying to compress video from the Photos app

alariej commented 2 years ago

Hello and thanks for publishing this library.

react-native-compressor crashes my iOS app when attempting to compress a video which is loaded from the iOS Photos app. Video is shot using the phone's camera. If I first save the video from the Photos app to a local folder on the device (using "Save to Files" in Photos' share menu), then the compression on that saved file is successful and I get no crash.

When debugging in xCode, the failure appears at line 295 in VideoCompressor.swift:

height: Int(resultHeight), width: Int(resultWidth) -> Thread 23: Swift runtime failure: Float value cannot be converted to Int because the result would be less than Int.min

It is as if the original videos from the Photos app were missing the height/width information. But note that the videos always play fine in my app's video player and the files always return the correct metadata (using getVideoMetaData), even the ones that crash the library.

The code I use to call the library:

const response = await Video.compress(
    file.uri,
    {
        compressionMethod: 'auto',
        maxSize: 1024,
        minimumFileSizeForCompress: 5,
    },
    progress => console.log(progress)
)
    .catch(err => console.log(err));

numandev1 commented 2 years ago

@alariej can you send me video so i can reproduce it on my side and fix it?

alariej commented 2 years ago

Hi and thanks for coming back so quickly.

I don't think sending you a video will really help, and the reason is following: If I make a copy of the video file and post it here, you will not get the error. I did try it myself for any video taken with the camera on the iPhone: Make a copy of the file and save it somewhere either locally or in Dropbox, try the compression and it works. The problem really appears when loading the video directly from the Photos library on the phone and trying to compress that "original" file.

The best way to reproduce the problem is to make a short video with the iPhone's camera, load it into a test app (using a native iOS Share Extension) and use the file's URI as the source for react-native-compressor.

numandev1 commented 2 years ago

@alariej Sorry, I did not get you. how do you load the video directly from the photos library? how can I load the video in the test app by native iOS Share Extension?

alariej commented 2 years ago

A Share Extension in iOS allows you to share content (links, images, videos, files) from other apps into your app, using the standard iOS "Share" button. For example, if you go to the Photos library on iPhone, pick an image or a video, press on the "Share" button (lower left), you can then select an app (Mail, WhatsApp, Safari, etc.) which you want to share the content with. You can implement that sharing functionality in React Native using, for example, https://www.npmjs.com/package/react-native-share-menu. They have good walk-through instructions at https://github.com/meedan/react-native-share-menu/blob/master/IOS_INSTRUCTIONS.md. I think this is a pretty standard way to load content into apps, so it would be great if your library could support it.

alariej commented 2 years ago

Additional info: I further tested this use case by trying compression: 'manual'. I get no crash, but also no compression, and no video (only audio) in the output file. Running getVideoMetaData on the output file returns an error: [Error: *** -[__NSArrayM objectAtIndex:]: index 0 beyond bounds for empty array], probably because there is no video info in the file.

numandev1 commented 2 years ago

@alariej I am too much busy today or maybe tomorrow too, but i will try to reproduce it on my side and fix it this weekend.

GrandPoohBear commented 2 years ago

My team has seen the issue where compressed files don't have any video track, and we've been trying to track it down. There was one video that would reproduce the situation every time, but only from our tester's phone. Any time we tried to repro on other devices, the compression worked fine. 🤷🏻

I opened an issue against NextLevelSessionExporter (the lib doing the heavy lifting on iOS), but haven't gotten anything to go on: https://github.com/NextLevel/NextLevelSessionExporter/issues/38

alariej commented 2 years ago

@nomi9995 The more I read about this issue, the more I think it has nothing to do with react-native-compressor. The problem seems to be with the way videos are stored in the Photos library on iOS. They cannot be imported into an app the same way as for images. Let me investigate this further, I will come back soon with what I can find out.

numandev1 commented 2 years ago

@alariej any update on this? did you investigate further?

alariej commented 2 years ago

There is not a lot of info on this problem. Most of the iOS apps (on Github) which import or share videos do not seem to apply compression to the file.

Perhaps the most interesting information I found is in the iOS share extension of the Signal chat app:

https://github.com/signalapp/Signal-iOS/blob/master/SignalShareExtension/ShareViewController.swift

There is a comment at line 936 which says that some videos in the Photos library need to be first copied outside of the Photos library before being processed (for compression). This seems to confirm what I am seeing in my app when trying to compress videos directly from the Photos library.

This would also confirm that the problem is not in your library, so I think you can go ahead and close the issue for now. I will try a solution similar to Signal in the next few days and let you know here if it works.

GrandPoohBear commented 2 years ago

I would definitely recommend copying the file to the app’s documents directory before doing anything else you need it for. The OS gives you no guarantees about how long other files in other places will be available!

On Feb 13, 2022, at 8:34 AM, Jean-François Alarie @.***> wrote:

There is not a lot of info on this problem. Most of the iOS apps (on Github) which import or share videos do not seem to apply compression to the file.

Perhaps the most interesting information I found is in the iOS share extension of the Signal chat app:

https://github.com/signalapp/Signal-iOS/blob/master/SignalShareExtension/ShareViewController.swift

There is a comment at line 936 which says that some videos in the Photos library need to be first copied outside of the Photos library before being processed (for compression). This seems to confirm what I am seeing in my app when trying to compress videos directly from the Photos library.

This would also confirm that the problem is not in your library, so I think you can go ahead and close the issue for now. I will try a solution similar to Signal in the next few days and let you know here if it works.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

numandev1 commented 2 years ago

@alariej can you share the URI of the video that you get from the share extension I wanna see the URI scheme and other things?

alariej commented 2 years ago

The videos in the Photos library are accessible from the Share Extension with a URI of this type:

file:///var/mobile/Media/PhotoData/OutgoingTemp/FC268169-1522-47C1-84E6-F71584EA5C69/Compatible/IMG_0985.MOV

However this URI cannot be forwarded from the Share Extension to the iOS app. Its content must first be copied to a folder ("container") shared between the Share Extension and the app. In Swift this is done by doing: FileManager.default.copyItem(sourceURI, destinationURI). The destination (or shared) URI then looks like:

file:///private/var/mobile/Containers/Shared/AppGroup/751639A9-C56D-47E8-8A4E-1BF1AA5084E3/share.MOV

It is this last URI which can be used in the app to play the video (for example, in a video player embedded in a WebView), or be posted to a remote server (for example, via rn-fetch-blob). Both these operations work without problem. However, when I try to use the same URI in react-native-compressor, it fails as if the video track in the file was missing.

The Signal iOS app works slightly differently. It does copy the video from the Photos library to a separate container in the Share Extension. But it applies the compression in the extension directly, and then shares the compressed video with the app. Not sure why they do it like that, as it probably forces the user to wait for the compression to be completed before having a visual confirmation of the content to be shared.

I'm kind of stuck here at the moment, but I'll keep digging in the next few days.

Note: Signal also uses AVAssetExportSession to compress videos in iOS -> https://github.com/signalapp/Signal-iOS/blob/master/SignalMessaging/attachments/SignalAttachment.swift#L1159

numandev1 commented 2 years ago

@alariej can you make a GitHub repro of share extension and compressor? I will make a solution for you if you make a GitHub repro.

alariej commented 2 years ago

Hmmm, give me a few days. I can't promise but I'll see what I can do.

alariej commented 2 years ago

@nomi9995 I now have a test app with an iOS Share Extension here:

https://github.com/alariej/sharedvideocompress

You might need to change the value of DEVELOPMENT_TEAM in project.pbxproj to your own ID
I tested the app in debug mode in xCode, on a physical iPhone, not on a simulator
The UI is empty at first, you see components only after having shared a video with the app
The app (sharedvideocompress) should appear in the list of sharing candidates when sharing videos only (no other media)
I tested sharing videos from the Photos app and from the Files app
Sharing from the Photos app crashes react-native-compressor at line 295 in VideoCompressor.swift
Sharing from the Files app works (you first need to get videos in there, for example by using "Save to Files" in the Photos app)

Here a few interesting discussions on related issues:

https://github.com/react-native-share/react-native-share/issues/634 https://stackoverflow.com/questions/60086748/can-i-get-absolute-uri-of-video-on-camera-roll-and-mutate-it-from-my-app

numandev1 commented 2 years ago

@alariej my provision profiles does not match with this app

alariej commented 2 years ago

There might be a solution here: https://stackoverflow.com/a/57196411

It seems you need to enable "App Groups" in your Apple developer account, in order to be able to use Share Extensions. (I probably also did that, a few years ago)

numandev1 commented 2 years ago

@alariej i am using my office' app provioning profiles therefore, i have no access to add share group😔

numandev1 commented 2 years ago

@alariej I tested on the simulator and it is working well on my side.

alariej commented 2 years ago

@nomi9995 I suspect your test does not reproduce the case where a user makes a video with the phone's camera (it would then be a .mov file, and not a .mp4) and tries to load it into the app via a Share Extension (by pressing the Share button in the Photos app). That's a problem with the simulator: It doesn't seem possible to simulate taking pictures or videos with the camera.

alariej commented 2 years ago

Submitted PR https://github.com/Shobbak/react-native-compressor/pull/98

numandev1 / react-native-compressor

iOS app crashes when trying to compress video from the Photos app #93