felixpalmer / android-visualizer

Takes the input from the Android MediaPlayer and displays visualizations, like in iTunes or WinAmp
816 stars 278 forks source link

Capture Handling with audio frame stack #15

Open dcerisano opened 9 years ago

dcerisano commented 9 years ago

Thanks for your work with the Visualizer class, especially the decoding of the FFT frame.

Your app synchronizes animation with the sound sampling rate with (invalidate).

   * Pass FFT data to the visualizer. Typically this will be obtained from the
   * Android Visualizer.OnDataCaptureListener call back. See
   * {@link Visualizer.OnDataCaptureListener#onFftDataCapture }
   * @param bytes
   */
  public void updateVisualizerFFT(byte[] bytes) {
    mFFTBytes = bytes;
    invalidate();
  }

The problem with this is that Visualizer's sound sampling mechanism is clunky. I have only seen a maximum sound buffer sampling rate of about 50ms which is crippling your graphics frame rate to about (20 fps) because of the synchronization with (invalidate).

The solution is to abandon Visualizer's data capture listener and set up a simple capture handler than runs at a nice smooth 60 fps.

Using a thread safe stack to store the capture frames will asynchronize the audio capturing and graphics rendering threads. Mirror smooth animation will result. Your graphics thread can feed on the stack asynchronously.

Your graphics are pretty basic and should be able to keep up, with room for even more fancy stuff.

It should result in peformance like this WebGL visualizer (which passes FFT data to the GPU as a texture!) https://www.shadertoy.com/view/Xds3Rr

felixpalmer commented 9 years ago

All sounds most sensible. If you get to implementing a more performant version, that will be most welcome