search

spite / ccapture.js

A library to capture canvas-based animations at a fixed framerate
MIT License
3.59k stars 407 forks source link

Capture an audio-reactive application #125

Open JetStarBlues opened 3 years ago

JetStarBlues commented 3 years ago

I have a Three.js program that uses the Web Audio API.

From previous issues, it seems there is currently no way to manipulate Web Audio as needed for capture:

[T]here's no way to "throttle" the internal workings of Web Audio, so the AnalyserNode will run in real time, desyncing from the canvas capture.

The meshes in my program "react" to the audio, so it is not possible to add the audio afterwards.

How was the Obsidian example in this project's README captured? The demo looks to be audio-reactive...

spite commented 3 years ago

usually just capture the video, and merge with the audio file afterwards. Obsidian is not audio reactive, it's scripted I have a branch that experiments with throttling the whole web audio graph, but it's not practical, yet. for now, unfortunately, there's no way to capture anything that reacts in real-time

JetStarBlues commented 3 years ago

Ah ok, thank you. Will try pre-generating the FFT data as suggested in this issue.

ElizabethHudnott commented 3 years ago

I'd like to be able to add audio to the video files rendered in the browser. Non-real-time is fine for my use case. I can render audio using an OfflineAudioContext but so far I haven't managed to accomplish the "merge" step inside the browser. I see that some peeps have ported ffmpeg to the browser using web assembly so I might try to get my head around that at some point, though it seems a bit over kill for such a simple task, but I really want my application's users to be able to download one completed file. I did some experiments with the browser's Media Encoder API, but my machine wasn't fast enough to transcode 50fps without audio glitches in real-time even just loading a WebM and an MP3 from disk.

Thanks for the cool library.