ofVideoRecorder for macOS ~ help needed

[dimitre]

I'm researching ways of making a default video writer for macOS. Usually I'm using Syphon Recorder, it runs great but it would be better to have more control (rec start, stop, addframe, etc.).

The idea is having a basic simple interface for recording video (without audio at first). ofVideoRecorder / ofVideoWriter can be the base class with simple functions (setFbo, begin(fs::path), end, addFrame). Maybe a helper class like ofVideoWriterSettings

And having this base class extended to something like ofAVFoundationVideoWriter.mm (Objective-C or Objective-C++)

Video can be written using AVAssetWriter using AVAssetWriterInputPixelBufferAdaptor with kCVPixelFormatType_32BGRA for the faster video writing. We can have an internal shader to internally move pixel order to BGRA.

I'm wondering how to transfer data from openGL to AVAssetWriter (if possible without moving pixels to cpu)

Maybe Create CVPixelBufferRef from OpenGL https://gist.github.com/caojianhua/8494bb3077228ee01fda

I don't have experience in Obj-C I can make a repository with a sketch of everything so it can be improved by others.

Motivation: we now have hardware accelerated video encoders starting with apple silicon, so it would be great to be able to record videos in realtime, with good quality without any external tool or extra step.

Ideas? @2bbb @danoli3 @ofTheo @artificiel

[artificiel]

on apple silicon It is possible to allocate Metal textures (not OpenGL) that can be shared to both GPU and CPU. i presume AVAssetWriter can process a shared-allocated CVMetalTexture to disk (presumably GPU-compressed) with no "memory readback" per se (and thread-safe, double-buffered).

i don't know how approachable this is in the context of an openGL app?

[danoli3]

Something similar to https://github.com/julapy/ofxiOSVideoWriter/tree/master/src

[dimitre]

Yes @danoli3 something similar, simpler for macOS

[dimitre]

Just sketching an object that can potentially be part of OF core and be the base class for video recorders. Maybe it can work with automatic file names (software name plus year, month, day, h, m, s) it can have a helper class like ofVideoWriterSettings with options for framerate encoding, codec, bitrate, etc. We can pass only a fbo and the video recorder takes care of the rest (conversion to ofPixels if needed or some more advanced texture sharing when possible), so the fbo already has the attributes for color depth and width / height.

names are not very good. just starting some discussion to get some more ideas.

class ofVideoWriter {
public:
    ofVideoWriter();
    void setOutputFile(const of::filesystem::path & _fileName);
    void begin();
    void addFrame();
    void end();
    void setFbo(ofFbo * _f);
    void toggleRecording(); // maybe a way of starting or stopping video rec depending on internal status.

};