search

usc-imi / aeo-light

AEO-Light 2 is a new generation of optical sound extraction software developed by the University of South Carolina in close cooperation with Tommy Aschenbach. The project is made possible by the generous support of the National Endowment for the Humanities.
Other
41 stars 11 forks source link

I see that a newer version of FFmeg now has cineform support #17

Closed expertpcb150 closed 5 years ago

expertpcb150 commented 5 years ago

I 'd like to update using the latest FFmeg binaries for Mac OSX so that I could load Cineform file directly into AEO. I have done some Xcode programing builds. Not sure what I would have to do here but am willing to try with some support and guidance... or do you have and Archived directory that can be downloaded and then I just launch the project and update where necessary. I am not a total novice but have only worked in Xcode on a Mac. George

retokromer commented 5 years ago

As long as I know, the situation is still the same as a little more than three years ago, when CineForm was implemented into FFmpeg. There is a decoder for many flavours, but not the Bayer-based RAW, which would really be of interest for AEO-Light:

12/16-bit CFA Bayer RAW, log encoded and compressed at 12-bit progressive
expertpcb150 commented 5 years ago

Thanks for responding... I saw this https://fxhome.com/forum/discussion/45239/cineform-now-fully-open-source which seemed to imply that it would be possible to finally incorporate Full Cineform support in AEO. As you probably know going through the painful conversion of CineForm to a usuable format taht can be read by AEO is very time consuming.

retokromer commented 5 years ago

Yes, the SDK has been released on GitHub on 2017-10-15. On my end, I have tried to use it, but it was finally much simpler to hack it for our needs. Yet, sadly, I didn’t had the time and energy so far to make a clean PR for implementing the Bayer-based format we use into FFmpeg.

expertpcb150 commented 5 years ago

Unfortunately for me I have to do something... I have numerous 5K scans in Cineform format which require first converting them using Premiere to export in a custom 5K DV file (30 minute feature takes 5days approximately... and then sucking it into AEO and then extracting the audio (using 48k sampling) then takes another 3Days Approximately.... All of this to get decent sound quality on the music portion of the audio. There has to be a better way and I need to try to solve this.

BTW thanks for responding...

retokromer commented 5 years ago

30 minute feature takes 5days approximately

That’s very strange indeed! It should be possible to export 5K CineForm to DPX by Resolve between real time and its double.

expertpcb150 commented 5 years ago

OK I thought I had to be doing something wrong.... So I use Premiere and export as DPX? and NOT as DV correct? So I 'll have to look at how to do that. If that will cut the time down to just a few hours I can live with that. The AEO at high res takes a long while as well... any suggestions on how to speed that up?

Thanks

retokromer commented 5 years ago

So I use Premiere and export as DPX? and NOT as DV correct?

DPX or TIFF have a higher image quality, therefore a better sound quality can be extracted from them.

The AEO at high res takes a long while as well... any suggestions on how to speed that up?

I’ll try to check that tomorrow. (Then I’ll be out of office until Friday.) What computer model, how much RAM and what OS version do you use?

expertpcb150 commented 5 years ago

The original scan in CineForm is 4874 by 3232 I have exported at custom DV maintaining same resolution and progressive. I'm on MacOSX 10.13.6 with 8 Gig ram. 1 Terabyte internal and 5 Terabytes external MacPro Laptop 2.8 GHz Intel Core i7

Once I have the audio extracted and EQed ,then its added back to video and exported in cropped format with window bars as needed and centered accordingly..

retokromer commented 5 years ago

The original scan in CineForm is 4874 by 3232

4864 x 3232? Kinetta?

I'm on MacOSX 10.13.6 with 8 Gig ram.

I’ll try to test on a similar setting. 8 GB RAM is possibly the weak point, also because macOS 10.13 itself uses a lot of it.

expertpcb150 commented 5 years ago

Yes the scans were done on a Kinetta..

I haven't seen any recommendations as to what memory was needed... However not sure if that would make that much of a difference.

expertpcb150 commented 5 years ago

What would you advise? as to hardware and memory? See if I can get that funded for a future acquisition

retokromer commented 5 years ago

Sorry, I could not run a test today. I’ll be at the lab again only on Friday.

expertpcb150 commented 5 years ago

OK Thx

wilsbacher commented 5 years ago

AEO crunches the audio using OpenGL, so the capabilities of the graphics card provide the limiting factor for processing speed. Your kinetta software also relies heavily on the graphics card--and Jeff usually includes a pretty robust card. Try running AEO on the kinetta PC as a point of comparison--and DPX will be processed faster than an equivalent video source.

expertpcb150 commented 5 years ago

OK Thx

expertpcb150 commented 5 years ago

hi wondering if your tests led to some revised advice?

thx

wilsbacher commented 5 years ago

Closing this thread as it was initially about FFMpeg and Cineform. You can repost as a new thread relating to hardware configurations.

retokromer commented 5 years ago

FYI @expertpcb150 : This morning I could run tests on iMac (27-inch, late 2013) and I got from 16-bit DPX at Kinetta resolution a sound extraction speed of 4.21 fps in average.