search

gorastudio-git / SCNRecorder

The best way to record your AR experience!
MIT License
201 stars 51 forks source link

Take photo crash and record issue #19

Closed 0k1019 closed 3 years ago

0k1019 commented 4 years ago

Hello, Thank you for wonderful project.

I bump up the library version to 2.0.0 to 2.2.0. After I update version, I can't take photo with crash log. And no video after video recording.

I have some devices and I test "SCNRecorder".

I test by checkout following project tag 2.0.0, 2.1.3 and 2.2.0. And run Example Project. In ARSCNView Example

On version 2.0.0

Nice working

Not working

On version 2.1.3

Nice working

Not working

2020-11-16 12:18:16.303563+0900 Example[529:45566] Metal API Validation Enabled
2020-11-16 12:18:18.087331+0900 Example[529:45847] [CAMetalLayerDrawable texture] should not be called after already presenting this drawable. Get a nextDrawable instead.
Fatal error: Unexpectedly found nil while unwrapping an Optional value: file SCNRecorder/ImageOutput.swift, line 60
2020-11-16 12:18:18.089814+0900 Example[529:45768] Fatal error: Unexpectedly found nil while unwrapping an Optional value: file SCNRecorder/ImageOutput.swift, line 60

StartVideoRecording Not working with black video.

On version 2.2.0

Nice working

Not working

2020-11-16 12:28:26.076353+0900 Example[555:49819] Metal API Validation Enabled
Fatal error: Unexpectedly found nil while unwrapping an Optional value: file SCNRecorder/ImageOutput.swift, line 60
2020-11-16 12:28:27.593305+0900 Example[555:50010] Fatal error: Unexpectedly found nil while unwrapping an Optional value: file SCNRecorder/ImageOutput.swift, line 60

StartVideoRecording Not working with black video.

Clue

I try to find clue I find TakePhoto VTCreateCGImageFromCVPixelBuffer function is not working correctly iOS 13.

I can't find any clue StartVideoRecording

I think OS version dependent on this project. Please check.

Hope you find reason soon!

v-grigoriev commented 3 years ago

Thanks for your findings! I was able to reproduce the issue and I'm working on fixing it.

v-grigoriev commented 3 years ago

Actually I was wrong. I was able to reproduce the issue using MTLPixelFormat.bgra10_xr_srgb, but it should be an impossible format for any SCNView related views. @0k1019 Can you please tell me what is the pixelFormat of CAMetalLayer you have in iOS 13 on iPad and iPhone? You can grab it, for example near takePhoto function call. e.g. print(sceneView.recordableLayer!.pixelFormat.rawValue)

0k1019 commented 3 years ago

@v-grigoriev I run print(sceneView.recordableLayer!.pixelFormat.rawValue). The result is 81 (MTLPixelFormat.bgra8Unorm_srgb) in iOS 13 on iPhone and iPad.

Below text and photo are about log and Fatal error screen shot.

2020-11-23 10:46:09.518798+0900 Example[277:5553] Metal GPU Frame Capture Enabled 2020-11-23 10:46:09.518993+0900 Example[277:5553] Metal API Validation Enabled before takePhoto function call sceneView.recordableLayer!.pixelFormat.rawValue: 81 after takePhoto function call sceneView.recordableLayer!.pixelFormat.rawValue: 81 Fatal error: Unexpectedly found nil while unwrapping an Optional value: file SCNRecorder/ImageOutput.swift, line 60 2020-11-23 10:46:18.099552+0900 Example[277:5703] Fatal error: Unexpectedly found nil while unwrapping an Optional value: file SCNRecorder/ImageOutput.swift, line 60

스크린샷 2020-11-23 오전 10 46 43
v-grigoriev commented 3 years ago

@0k1019 Can you please test with 2.3.0?

0k1019 commented 3 years ago

@v-grigoriev Thank you. It works really well in example project! Can you tell me what kinds of problem briefly?(I try to understand your commit, but there are a lot refactoring and fixing bug codes.)

v-grigoriev commented 3 years ago

Sure, starting with 2.1 to support iOS 14 on iPhone 12 and iPad Pro 2020 I changed the way to capture pixel buffers from MTLTexture. Instead of getBytes, I started using iosurface to create CVPixelBuffer. And it works on iOS 14 without any problem. But it seems iOS 13 Video Toolbox doesn't support such pixel buffers, so I reverted the change for iOS 13.

And now I have an idea of how to make all of this even better. I'm working on a solution that uses Metal Performance Shaders to properly and fast capture pixel buffers. And I'm going to add unit tests for testing with all supported pixel formats on various devices and iOS versions to prevent such crashes in the future.

0k1019 commented 3 years ago

Thank you wonderful explanation. I look forward to your roadmap(MPS, unit test...).

v-grigoriev commented 3 years ago

The update is ready. I hope it will prevent any future issues.

Screenshot 2020-12-07 at 2 51 43 PM