Consistent stutter - Githubissues

icecube45 commented 7 years ago

Hi! I'm noticing a consistent stutter with my strip, but not any other lag. Video: https://www.youtube.com/watch?v=a1HR7BQzPOY

Note: this is on the official code and pyaudio, not the fork I've been working on

scottlawsonbc commented 7 years ago

Are you controlling this with an ESP8266?

Can you please upload your config.py as a gist and link it here?

icecube45 commented 7 years ago

Yup, an esp8266.

https://gist.github.com/icecube45/0874da6c2f94b97ff823799c71104f8a

Only change I've made is the number of leds. Printing the fps over serial shows no drop in fps.

scottlawsonbc commented 7 years ago

Hmm this is very strange and I'm not able to reproduce the issue.

Could you try running the latest code in the experimental branch?

The experimental branch processes audio input and updates the LED strips using a different method than the master branch. It also has a brand new GUI and a complete overhaul of the visualizations. I'm curious whether you would still encounter the stuttering issue. If you do encounter the stuttering problem, then you probably have a network issue.

I just changed the settings in the experimental branch to match your particular settings. If you run the experimental branch hopefully you won't have to change any settings.

Important things to know about the experimental branch:

It's not finished yet
Settings are now in config/config.ini instead of in python/config.py
Run the visualization using main.py instead of visualization.py
The autocorrelation effect looks really nice with this song

Some GUI settings are incomplete:

Don't adjust the rise/fall sliders. They don't do anything yet.
Don't change the colormap, your selection will be ignored
Your max frequency range is ignored. Instead, the max frequency is automatically determined by measuring the spectral rolloff.

You may need to install two additional dependencies:

pip install matplotlib
pip install qdarkstyle

icecube45 commented 7 years ago

When trying to use autocorrelation I get this error


Traceback (most recent call last):
  File "main.py", line 220, in <module>
    audio_visualization(rolling_window)
  File "main.py", line 60, in audio_visualization
    led.pixels = effect_function(audio_frames) * 255
  File "C:\Users\Cole\Desktop\audio-reactive-led-strip-experimental\python\dsp.py", line 126, in wrapper
    filt.update(func(*args, **kwargs))
  File "C:\Users\Cole\Desktop\audio-reactive-led-strip-experimental\python\visualize.py", line 132, in Autocorrelation
    f = peak(f, len(f) // (config.N_PIXELS))
  File "C:\Users\Cole\Desktop\audio-reactive-led-strip-experimental\python\visualize.py", line 28, in peak
    y1 = np.empty((n, 2))
TypeError: 'float' object cannot be interpreted as an integer

icecube45 commented 7 years ago

I did replace my router just now, and the visualization is as smooth as butter, perhaps I'll try the experimental if the above error is solved.

scottlawsonbc commented 7 years ago

Are you is using Python 2.7 or 3? So far I have only tested the experimental code on Python 3

scottlawsonbc commented 7 years ago

It looks like the problem was caused by a simple type conversion error. It's fixed in the latest version of the experimental branch

joeybab3 commented 7 years ago

I'll have a look at the experimental branch tomorrow when I have access to my leds, I was having the stuttering issue occasionally as well.

icecube45 commented 7 years ago

@scottlawsonbc with the type conversion solved, I'm trying to run the experimental branch again, yet while it appears to be "running", I'm not sure if the effects are working at all: video of visualization

scottlawsonbc commented 7 years ago

Did you flash the firmware in the experimental branch? I switched the communication protocol for the ESP8266 firmware but I had not made the change on the Python. From your video it looks like a communication protocol mismatch. The new protocol supports more than 256 LEDs.

I just pushed this change https://github.com/scottlawsonbc/audio-reactive-led-strip/commit/08a37a87ffe14fa8c7dbefe9c1e377fb89116fb0 which updates Python to use the new communication protocol as well.

To get it working:

Download the latest experimental branch code which uses the new protocol
Upload the firmware that is included in the experimental branch

After doing that, does it work again?

icecube45 commented 7 years ago

Yup! that was the issue. Working great now

scottlawsonbc commented 7 years ago

Great!

By the way, you can tweak the response sensitivity by editing visualize.py

I'm using a custom python decorator to apply a lowpass filter to the output of the visualization function. You can adjust the filter's time constants to make the visualization more "blinky" or more "relaxed".

For example, with the autocorrelation effect in visualize.py:

@dsp.ApplyExpFilter(fall=0.07, rise=0.001, realtime=True)
def Autocorrelation(audio_frames):

You can make this more relaxed by changed the filter to

@dsp.ApplyExpFilter(fall=0.15, rise=0.001, realtime=True)

You only really need to change the fall time. A fall time range of 0.05 to 0.2 seems to work well. Physical interpretation of the fall time is the amount of time that the LEDs take to decay to 1/e (37%) of their current value in seconds. So fall=0.05 means 50 ms.

Atraii commented 7 years ago

I'm seeing stutter with the new experimental branch. It seems to be server side. I think the CPU is a bottleneck (all 8 cores at 100% here). I am regularly dropping down to 8-15 fps.

To get around the 256 LED limit I was seeing with the ESP8266, I did a quick write-up using NeoPixelBus. All out, the ESP8266 can handle 135 FPS with 256 LEDs (I haven't tested more because the python code is doing something weird).

Is this comparable with the I2S for you?

EDIT: I just tested the ESP8266 with the I2S library with 256 LEDs. The I2S is going strong at 210+ FPS, and my computer is the bottleneck, so it could potentially push out more.

SECOND EDIT: The 135 FPS is due to the update limit on the WS2812s. Going faster than 135 FPS for 256 pixels is actually impossible (due to the design) and the NeoPixelBus library limits you to that.

scottlawsonbc commented 7 years ago

To get around the 256 LED limit I was seeing with the ESP8266, I did a quick write-up using NeoPixelBus. All out, the ESP8266 can handle 135 FPS with 256 LEDs (I haven't tested more because the python code is doing something weird).

In my previous comment I mentioned that I updated the communication protocol and it now supports more than 256 LEDs. So far I have only made this change in the experimental branch.

I'm seeing stutter with the new experimental branch. It seems to be server side. I think the CPU is a bottleneck (all 8 cores at 100% here). I am regularly dropping down to 8-15 fps.

Are those FPS numbers measured from the computer side or the ESP8266 side (via serial)?

It's possible that you are updating the LEDs too frequently. Updating the LEDs hundreds of times per second can overwhelm the ESP8266's ability to receive and process incoming packets. This causes packets to be dropped and results in stuttering.

Also, the experimental branch decouples the audio framerate from the LED refresh rate. Your LEDs might be updating at 400 FPS even though you set the FPS to be 60. Setting the FPS only affects the rate at which new audio data is processed. When new audio data is not available, the code will interpolate and continue refreshing the LEDs as fast as possible. In the master branch it will not continue updating the LEDs and will instead pause until new audio data is available.

You can work around this by forcing a delay between every refresh. At the bottom of main.py you can change time.sleep(1e-3) to something like time.sleep(5e-3) to limit the LED refresh rate to below 200 FPS.

scottlawsonbc commented 7 years ago

What is the current status of this issue?

Atraii commented 7 years ago

The FPS numbers were measured computer side. I haven't tested tuning the sleep time yet, as I've been trying to get the protocol up and running without stutter on 853 LEDs using multiple outputs. Unfortunately, it doesn't appear to be possible to dump to both strips simultaneously and I'm still encountering a bottleneck. For the purposes here, I'm only testing with 256 LEDs now. I'm still seeing very low framerate serverside (but it seems to be massively improved if I run with gui=False.

Regardless, I'll try increasing the sleep time, as the root cause may be related. But ultimately, I'm seeing low FPS on the computer (9-11 FPS) with GUI. My machine is old (AMD FX-8120 circa 2011), but should still be sufficient for this level of processing.

scottlawsonbc commented 7 years ago

I think the FadeCandy would be your best option for controlling the 853 pixel LED strip. Each FadeCandy board can control up to 512 pixels through 8 output channels that can drive 64 pixels each. The FadeCandy boards perform temporal dithering, gamma correction, and run at +400 FPS. I have two FadeCandy boards myself and they look great.

Unfortunately there's no way to use more than 64 pixels per channel because of the inherent tradeoff between FPS and number of pixels. You would have to cut your strip every 64 pixels and run the data line all the way back to the FadeCandy board.

scottlawsonbc / audio-reactive-led-strip

Consistent stutter #14