Using nodemcu, noise problems and a few bugs causing web gui to be sporadic

[bbkiwi]

Hi Charles, You have a fantastic resource here. I have with much effort (due to little experience with git, c, virtualbox, linux, makefiles, websockets, electronics etc) managed to get both the embedded and non-embedded versions working. Especially useful was con-f-use/esp-clone.sh.

This has been a bit of a detour from my original goal of putting LEDs behind the head of my banjo uke that respond to the sound. I originally modified some adafruit 'color-organ' code so it would 'rotate' the pulsating LED images in response to the accents in my playing. https://www.youtube.com/edit?o=U&video_id=jLgvuNdCXMU https://www.youtube.com/edit?o=U&video_id=IRVPY4qavpc

I was in the process of getting it to react to freq when I saw your work. The idea of being able to connect via a web gui appealed.

What I have now (after changing your code as mentioned below) using a adafruit mic MAX4466 works perfectly when not serving up the webgui. If all the controls are closed (system 320hz), there is a bit of flashing of the LEDs from generated noise, but if I open the oscope, DFTs, LEDs or Notes, (running 170hz) lots of noise is generated and the LEDs flash from this noise. Even if I remove the mic, noise is still getting into the ADC.

At first the performance when serving the webgui was extremely inconsistent, it ran with 4 Hz sometimes, the controls and displays of oscope, etc didn't not always work. A few others have mentioned this behaviour in the earlier issues. I had to change the following:

1. Makefile with the specifed user.cfg produced an index.html that double loads main.js and menuinterface.js and this really messes up things (don't quite know why, as developer tools for chrome and firefox show it loaded ok). FIXED this by removing lines 5-7 from web/index.html and added to user.cfg PAGE_SCRIPTS = main.js

   2. line 92 in main.js is missing a ! It should contain: ... &&!pause_oscilloscope)?

   3. line 331 of main.js seems a bit different from the other commands, I think it should be if(!pause_notes )

2. A number of variables in main.js and menuinterface.js are undeclared my web developer console flagged this. . I added var in front. I don't know how important this it.

I've attached a diff between what is in your repo and my changes. I'm still not sure how can make my own repo and request a pull :-0 diffs.txt

Cheers, Bill

[cnlohr]

Wait... you mean to tell me you had problems and made fixes for them? I really need to dig into this this weekend. There's a couple other issues that I can probably address pretty quickly. I can understand the initial discomfort with checking out a repo because you would have to install git. Pull requests are always appreciated but this is still pretty cool!

One thing I wanted to check was if you use the GUI and then open the parameters tab, then click "RESTORE" and "SAVE" does that make it behave better? It should be very clear and smooth with something like a banjo!

[opus506]

I forked and applied all the changes bbkiwi suggested. The webui works as it should now and the system runs at very high hz now compared to the 20-30hz I was getting previously.

Now lets fix the AP scan and join problem, more power for colors. :)

[cnlohr]

I'm just unfamiliar with the AP scan and join problem?

[bbkiwi]

Hi Charles (and opus),

I'm glad the fixes seem to work. I'm glad I am able to help in a small way! Finding the duplicate loading problems took a lot of time and it seems to be the cause of the very strange behaviour but I don't really understand why is causes it. The other bugs seem to typos which got into the code, they may be infecting menuinterface.js in other examples of yours and others based on your code.

I've had to use the RESTORE and SAVE trick before, but only after ESP8266 connecting problems. It is basically working perfectly now when the controls are shut down. It responds nicely to the banjo uke playing.

I'm visiting my son next week who is much more computer experienced and I will try and set up forked repos for the various examples which I have cloned (your colorchord, esp82xx, esp8266ws2812i2s, con-f-use/esp82XX-basic)

If you have time a few questions:

1. My real interest is in tweaking the embeddedcommon routines. Some sections of the code are set inactive with comments by you as not suited for embedded systems. Is this because of time constraints? space constraints? I'm using embeddedlinux for testing as it is easier that reflashing all the time :-) but it has more grunt so runs the extra bits with no problem.

2. I still don't completely understand the memory usage of the ESP8266. Is everything in flash? How much writting can be done to flash before it is damaged? Is IRAM just part of the flash memory? Are all variables stored in flash or is there memory for variable that is not in flash that has unlimited R/W cycles?

3. I'd like to use the gui to alter parameters and then shut down all wifi activity (to save power and avoid interference). Is it possible to turn off the wifi (perhaps using a GPIO pin signal) and have it running as in you nosdk8266 blog.

4. Alternatively, I notice that when all controls are closed the websocket defaults to sending wx commands. Is this a 'heartbeat' to keep the websocket open? Could I just let the socket close and have a js button to reopen it?

5. In examples that I have cloned now all esp82xx as a subdirectory which does not have exactly the same code. Are they each using different submits?

Thanks, Cheers, Bill PS - Re what opus said, I have tried the 'scan for stations' and it has never worked for, but I've not looked into it as I know the SSID of what I'm trying to connect to.

[pjmole]

Hi Bill it's great to see a new fan to Charles', work.

Since I have sadly no close musician friends. My first suggestion I have for you is to try a standard "piezo quitar pickup" with the Colorchord? Many are home made. My belief is that one of those would drive the Colorchord with only the piozo bias applied!

I have been trying 3d visualizations with 4 colorchord and video. There is much to learn about this little project. One thing is to not forget that the javascript is run on the browser's machine. There is a large difference between my Pi3 on Firefox vs Chrome on Kobo7

I have finally got the entire toolchain working again. So I have built and burnt the latest release. and now trying to get 4 new colorchords running identically.

I also checked out the problem of display of APs search. The line number 428 is "setTimeout( WifiDataTicker, 500 );" should be much longer say 10000 if you wish to see the results.

[cnlohr]

@pjmole That does sound like a reasonable move... probably not 10 seconds, unless... do you need to wait until connection is fully reestablished before reading it?

Also, you guys are gonna have to start taking some video of this stuff!!! Get some hackadays.

[pjmole]

There are still problems with the toolchain I have. Even though I can build colorchord without errors my newer builds reboot after getting the wifi channel info, while the older ones come back from the scan. You can see some debug on ttyUSB0. After my reinstall of system I grabbed data from old esp8266 with esptool read_flash and used that to burn other devices with some success

Is there a way to get the librarys used on an esp8266 memory? pjm@Pi3-desktop:~/sketchbook/colorcord.hacks/embedded8266$ make clean esp82xx/common.mf:42: Warning: No shell variable 'ESP_ROOT', using '/home/pjm/esp8266/esp-open-sdk' rm -f esp82xx/fwsrc/uart.o esp82xx/fwsrc/esp82xxutil.o esp82xx/fwsrc/flash_rewriter.o esp82xx/fwsrc/http.o esp82xx/fwsrc/commonservices.o esp82xx/fwsrc/http_custom.o esp82xx/fwsrc/mdns.o esp82xx/fwsrc/mfs.o user/custom_commands.o user/user_main.o user/ws2812_i2s.o user/hpatimer.o user/adc.o ../embeddedcommon/DFT32.o ../embeddedcommon/embeddednf.o ../embeddedcommon/embeddedout.o image.elf image.map image.lst pjm@Pi3-desktop:~/sketchbook/colorcord.hacks/embedded8266$ make all esp82xx/common.mf:42: Warning: No shell variable 'ESP_ROOT', using '/home/pjm/esp8266/esp-open-sdk' /opt/Espressif/ESP8266_SDK/crosstool-NG/builds/xtensa-lx106-elf/bin/xtensa-lx106-elf-gcc -mlongcalls -Os -I/opt/Espressif/esp_iot_sdk_v1.4.0/include -Iesp82xx/include -I. -Iesp82xx/fwsrc -Iuser -I../embeddedcommon -DICACHE_FLASH -DWEB_PORT=80 -DCOM_PORT=7777 -DBACKEND_PORT=7878 -DSLOWTICK_MS=50 -DVERSSTR='"Version: b45c2-dev - Build Fri, 14 Apr 2017 21:15:24 -0400 with -DICACHE_FLASH -DWEB_PORT=80 -DCOM_PORT=7777 -DBACKEND_PORT=7878 -DSLOWTICK_MS=50"' esp82xx/fwsrc/uart.c esp82xx/fwsrc/esp82xxutil.c esp82xx/fwsrc/flash_rewriter.c esp82xx/fwsrc/http.c esp82xx/fwsrc/commonservices.c esp82xx/fwsrc/http_custom.c esp82xx/fwsrc/mdns.c esp82xx/fwsrc/mfs.c user/custom_commands.c user/user_main.c user/ws2812_i2s.c user/hpatimer.c user/adc.c ../embeddedcommon/DFT32.c ../embeddedcommon/embeddednf.c ../embeddedcommon/embeddedout.c -flto -Wl,--relax -Wl,--gc-sections -nostdlib -L/opt/Espressif/esp_iot_sdk_v1.4.0/lib -L/opt/Espressif/ESP8266_SDK/crosstool-NG/builds/xtensa-lx106-elf/lib/gcc/xtensa-lx106-elf/4.8.2/libgcc.a /opt/Espressif/esp_iot_sdk_v1.4.0/lib/libmain.a /opt/Espressif/esp_iot_sdk_v1.4.0/lib/liblwip.a /opt/Espressif/esp_iot_sdk_v1.4.0/lib/libssl.a /opt/Espressif/esp_iot_sdk_v1.4.0/lib/libupgrade.a /opt/Espressif/esp_iot_sdk_v1.4.0/lib/libnet80211.a /opt/Espressif/esp_iot_sdk_v1.4.0/lib/libwpa.a /opt/Espressif/esp_iot_sdk_v1.4.0/lib/libphy.a /opt/Espressif/esp_iot_sdk_v1.4.0/lib/liblwip.a /opt/Espressif/esp_iot_sdk_v1.4.0/lib/libcrypto.a /opt/Espressif/esp_iot_sdk_v1.4.0/lib/libpp.a /opt/Espressif/ESP8266_SDK/crosstool-NG/builds/xtensa-lx106-elf/lib/gcc/xtensa-lx106-elf/4.8.2/libgcc.a -T /opt/Espressif/esp_iot_sdk_v1.4.0/ld/eagle.app.v6.ld -Wl,-Map,output.map -B/opt/Espressif/esp_iot_sdk_v1.4.0/lib -o image.elf PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/opt/Espressif/ESP8266_SDK/crosstool-NG/builds/xtensa-lx106-elf/bin /home/pjm/esp8266/esp-open-sdk/esptool/esptool.py elf2image image.elf esptool.py v1.2 pjm@Pi3-desktop:~/sketchbook/colorcord.hacks/embedded8266$ ls -l image* -rwxrwxr-x 1 pjm pjm 319821 Apr 14 21:15 image.elf -rw-rw-r-- 1 pjm pjm 34640 Apr 14 21:15 image.elf-0x00000.bin -rw-rw-r-- 1 pjm pjm 199584 Apr 14 21:15 image.elf-0x40000.bin

just musing ....

[cnlohr]

I don't understand the question?

[pjmole]

I am still learning about the toolchain, as I suggested you could leave a copy of a successful compilation in your git. I would like to check if the wireless libraries are same as your builds.

my latest build reboots when wiresless scan is requested.

[cnlohr]

AH YES Ok. I will try to do that. I really just need to find time to do it. Actually, if any other lurkers have a good build, please pull request it.

[pjmole]

I'll by trying to get a decent build, have a test procedure in mind. I seem to have better wifi-scan now but having problems with netburn and web functions now trying out ESP8266_NONOS_SDK_V1.5.3_16_04_18 I am trying every sdk I can find that compiles an links OK. but can I use a single compiler chain for all builds and sdk's ?

ls -l /opt/Espressif/ drwxrwxr-x 11 pjm pjm 4096 Apr 6 12:04 ESP8266_NONOS_SDK drwxrwxr-x 12 pjm pjm 4096 Apr 9 15:17 ESP8266_NONOS_SDK_V1.5.3_16_04_18 drwxrwxr-x 13 pjm pjm 4096 Mar 27 13:55 ESP8266_RTOS_SDK drwxrwxr-x 12 pjm pjm 4096 Mar 26 17:44 ESP8266_SDK -rw-rw-r-- 1 pjm pjm 58453 Mar 16 09:54 esp_iot_sdk_v0.9.3_14_11_21_patch1.zip -rw-rw-r-- 1 pjm pjm 1070625 Mar 16 09:53 esp_iot_sdk_v0.9.3_14_11_21.zip drwxrwxr-x 12 pjm pjm 4096 Apr 8 19:45 esp_iot_sdk_v1.4.0 drwxrwxr-x 10 pjm pjm 4096 Mar 28 15:26 esp_nonos_sdk drwxrwxr-x 12 pjm pjm 4096 Mar 26 21:00 esp-open-sdk drwxr-xr-x 6 pjm pjm 4096 Apr 5 19:55 include drwxrwxr-x 2 pjm pjm 4096 Mar 4 14:38 lib drwxrwxr-x 7 pjm pjm 4096 Mar 4 14:42 source-code-examples

I do all of my hacking for now on the javascript side, and think that is where many things could be accomplished...caliope anyone? so I will be doing burns and burnwebs for to test build environment till happy.

[bbkiwi]

Hi Charles and pjmole, This issue has morphed a bit but I'm still lurking. My son helped me to fork the repo. I've been spending my time mainly modifying embeddedout.c fixing (I hope) SORT_NOTES and adding some new features which can be controlled by new parameters added to custom_commands.c . I've also made some mods to embeddedlinux to test this off the esp8266. Charles, I'm almost ready to submit a pull request after I fix up a few more things.

Concerning the wifi behaviour, it is reasonable for me but scan doesnt work for me, but doesn't force a reboot. To switch the esp2866 to use another SSID does not go smoothly. burn and burnweb work fine I can use -b 1000000 at home, but when I visit my son I have to switch to -b 115200.

Cheers, Bill PS: pjmole, looked up "caliope" and elegance would be nice :-)

[pjmole]

Duh! Sorry for the stupid misspelling. "calliope"

Now I will have to try to fulfil my own challenge... Rough outline ---- My webgl uses up to 4 colorchords. if I use only one colochord and limit output to the 4 octaves of interest, should be able to draw some cylinders (instead of spheres) for each pipe from DFT stack output of each octave in image like calliope form factor.

later monitor pipes to trigger MIDI or air valves from compressor.

PS: I have not bothered much with the notes stuf, looking forward to seeing your updates.

PPS: I am so amazed with the tools available to-day.

There is a fantastic javascript JS Dynamic Audio Synth. http://www.keithwhor.com/music/ that can be integrated into colorchord with little effort. One evening of hacking and practically no testing I have a system that is ready to accept commands like Synth.play(sound, note, octave, duration); and 4 mono synth profiles piano,organ,guitar, and edm.

It is simply amazing you can play the synth keyboard and see the colorchord react on Pi Firefox and Android chrome.!

[gusnaughton]

This is actually a really interesting discussion and I'm kind of thinking aloud here so I can come back when I have time and research this -- I know that there are some string LED lights that you'd be able to hang down from a ceiling.. but I'm not sure if they can be individually programmed like a ws2812 can. I think it'd be really interesting if we can take those 3D visualizations and feed them into an array of 3D string lights.

A project I'll be working on this month within this space is making a 2D array... as in, 8 or so instances of colorchord (8 esp8266s driving one strip of ws2812s), so it'd be super cool if there'd be a way to scale this to 3D :)

[pjmole]

Hi Gus, Perry here...

 A single colorchord will drive up to 255 ws2812s. I have user 24 and  60 at a time.  88 would easily show a piano keyboad.
If you want some real fun try building a POV.  To show all 192 DFT bins. You would only need a one controler and perhaps 2 or 4 8xLED strings in parallel. Start the motor and watdh the blinkin' show.

Or I have had great success with the "infinity mirror" concept only a few LEDs are needed and dual semi reflecting surfaces to multiply the percieved number of points.

[zombodotcom]

How did you build this for the nodemcu ?

[cnlohr]

The same way it is built for other systems. Need GCC for xtensa / the esp-open-sdk.