search

ZackFreedman / Somatic

Open-hardware data glove
MIT License
392 stars 56 forks source link

Running on ESP32 #2

Closed DieKatzchen closed 3 years ago

DieKatzchen commented 3 years ago

Running this on an esp32 should be doable, right? it would reduce the partslist since it could do the bluetooth keyboard and mouse itself, without needing the bluetooth mate.

[EDIT]Placing these here for future reference https://github.com/T-vK/ESP32-BLE-Mouse https://github.com/T-vK/ESP32-BLE-Keyboard [/EDIT]

sudomain commented 3 years ago

The teensy 4 used in the Somatic operates at 600 MHz while I've read esp32's can operate at 160MHz or 240MHz, but in the Somatic youtube video (at 6:25) Zach says he underclocks the teensy to 150MHz and it works fine. Something that'll probably be an issue is the 520KB of SRAM being inadequate. The teensy has 1024 KB and somewhere in the video he said it used 80% of the RAM.

DieKatzchen commented 3 years ago

That's true, I forgot about that. The esp32 can use external ram, but at what point are we no longer simplifying things and just complicating them in me and exciting ways?

DieKatzchen commented 3 years ago

Did some digging, there's an esp-wrover-b module with 8mib of ram already built in. So that can probably take care of it.

sudomain commented 3 years ago

I'm not sure if you saw (and are referring to) the issue I opened (#3), but it'd be interesting to see the performance. External RAM modules will make the wrist area of the glove more bulky, but then again so will an additional bluetooth module for sending the raw data to an instance of TensorFlow Lite on Android.

EDIT: Nevermind, I thought you were talking about adding a module to an ESP32 based Somatic, rather than replacing it.

DieKatzchen commented 3 years ago

I made a preliminary attempt at changing this to use an esp32 and the blackketter/ESP32-BLE-Combo library. It compiles, and seems to communicate with the training app. Obviously I will need to build one to test it. If you want to have a look it's on my fork, I just pushed it.

sudomain commented 3 years ago

Cool! I'm following your fork now. I think you could test it just with the IMU hooked up to the ESP32 since I think the hall sensors just act as push buttons.

DieKatzchen commented 3 years ago

Pretty much. Payday is Monday, will buy just enough to test with. May have to modify the code slightly to use the PSRAM, unsure if the tensorflow library automatically malloc()'s when necessary. Some research is required.

[Edit]Follow up after researching, it's really as simple as using malloc() to put the Arena on the PSRAM, tensorflow itself doesn't care where you put the Arena. see https://github.com/tensorflow/tensorflow/issues/37628 [/Edit]

Millimeter94 commented 3 years ago

@DieKatzchen: I just tried to compile your fork but i am missing these tree libraries: BleCombo.h elapsedMillis.h analogWrite.h Are those something special or can i use the first thing google spits out

DieKatzchen commented 3 years ago

Whoops, my bad for forgetting to document my dependencies!

"BleCombo" library is from here: https://github.com/blackketter/ESP32-BLE-Combo "ElapsedMillis" and "ESP32 AnalogWrite" are both available in the Library Manager.

ZackFreedman commented 3 years ago

I appreciate your work in porting and forking the project, but please continue this discussion on the relevant repo.