search

micropython / micropython-esp32

Old port of MicroPython to the ESP32 -- new port is at https://github.com/micropython/micropython
MIT License
673 stars 216 forks source link

ESP32 I2S support #185

Closed nickzoic closed 4 years ago

nickzoic commented 6 years ago

ESP32 has I2S hardware so it'd be nice to support it!

References:

MrSurly commented 6 years ago

@nickzoic Are you implementing, or is this a call to arms?

nickzoic commented 6 years ago

Bit of both :-) I'm hoping to get to it in the next couple of weeks, but if someone else does it in the meantime I'm not going to complain :-)

I've got some hardware on order but the protocol is pretty transparent so I can see if its working on the 'scope in the meantime.

MrSurly commented 6 years ago

What sorta hardware? I know some folks (ab)use I2S for neopixels.

nickzoic commented 6 years ago

I think that's a great idea :-)

I ordered one of these based on the PCM5102. Still on the lookout for a cheap line-in equivalent, but there's always these mics to try.

MrSurly commented 6 years ago

Hook them together ... pointless, but fun.

UnaKarlsen commented 6 years ago

I recently got a little E-Ink display that uses SPI that I want to use with the ESP32, so I'll probably be poking at this in the next week or 2.

MrSurly commented 6 years ago

Hi @SmilyBorg -- I worked on the HW SPI, if you could give that a test, that'd be great.

Is it this one?

TheBestJohn commented 6 years ago

This would be really neat to make some form of VoIP interface on a local pbx

hackrid commented 6 years ago

keep the massive parallel features in mind iggr and sprite used to input / output parallel data:

parallel input: https://github.com/igrr/esp32-cam-demo parallel output: https://esp32.com/viewtopic.php?t=3188

nickzoic commented 6 years ago

@SmilyBorg go for it ... contributions are very welcome!

nickzoic commented 6 years ago

Also note: there's some work going on re: those little E-ink displays and MicroPython:

aapris commented 6 years ago

We'll organise a noise sensor workshop in January 2018 (in Helsinki) and we are planning to use one or two "Adafruit I2S MEMS Microphone Breakout - SPH0645LM4H" which @nickzoic referenced. The easiest way to get some noise pressure readings is to use Arduino/Genuino MKR1000 or Adafruit Feather M0 (those have one I2S port I guess) and some code from this Adafruit I2S MEMS Microphone Breakout tutorial.

But I'd love to use ESP32, if I2S is available in micropython then, because we may have option to use two I2S ports and two MEMS microphones. This means we can have one microphone inside the building and one outside and we can compare both readings in real time. (Also ESP32 is much cheaper than suitable Arduinos and it has both WiFi and BLE.)

A Fast Fourier Transform (FFT) library for ESP32-Micropython would be useful for sound spectrum analysis, but this is a different topic.

nickzoic commented 6 years ago

I haven't had a chance to poke at this yet but I now have some i2s hardware now and might get some time in a few weeks.Probably too late for your January conference though. Anyone else who's interested is welcome to beat me to it :-) If your application is simple enough, it might be easy enough to write in C using ESP-IDF directly ... it's actually a pretty easy sdk to get started with.

UPDATE: found a local supplier of those MEMS microphones, so might get to this soon.

aapris commented 6 years ago

Any progress? :)

n1zzo commented 6 years ago

Can you share at least a work-in-progress version of this? I'm interested in helping develop this!

Avi-TelnT commented 5 years ago

Hi I am looking to hire an ESP32 Python programmer to help me to develop a firework interface for ESP32 I2S for a mic on I2S ICS-43434 IC and play wav using I2S MAX98357A IC. The target is work needed Google speech recognition and Google text to speech from ESP32. The development environment must run finally from: https://github.com/loboris/MicroPython_ESP32_psRAM_LoBo If you have an experience on ESP32 MicroPython_ESP32_psRAM_LoBo Python please reply my email nissim@telnt.com And call my Skype nissim.test
The work is fix price, milestone-based and payment will be as showing milestone below, We will provide the hardware for that:

  1. Expose to LoBO Python the ESP32-IDE /MicroPython/Tools/esp-idf/components/driver/include/driver/I2S.H and I2S.C by integrate them in machine module so dir(machine) will show the I2S interface and all commands to set I2S for recording and playing using the ESP32 DMA. With test Python app showing that all I2S in machine.I2S module is working. No new make file will be created, only add code to machine module.
  2. Use the above machine.I2S to record and play to a wav file using a digital mic on I2S ICS-43434 IC and play wav using MAX98357A IC. With test Python app showing that wav and playing are working from random files, as play("test.wav") and record ("rec.wav") both must work on the same time. Also support for Flak encoding, so if test.wav is Flak file it will be played ok. If a command record_flak ("rec_flak.wav") the rec_flak.wav will be flak file.
  3. Doing above but record and play from ram, with Flak encoding as needed by Google SR AND TTS. See https://cloud.google.com/speech-to-text/docs/encoding. With test Python app demo it working.
  4. Use ram Flak Ram recoding to do SR to get the recognized the Mic audio, as text returns from Google back to ESP32, modify the return text with meaningful information and send to be played by Google TTS. Like Siri. With ESP32. Replace the button function By voice keyword recognized as gate to SR. All from Python As:
  5. You Say "Lovey’s", recorded to Ram in a loop all the time. Lower audio level is a mark to send chunk of 1 sec ("Lovey’s") data to Google SR after flak encoding. Room normal audio level must be monitor all the time, in separate multi tasking way.
  6. Wait get recognize " Lovy's as return text, if not go to stage 1
  7. Play TTS "Yes"
  8. You Say to Mic I2S ICS-43434 “What is the baby temperature now?”
  9. Get back the text in ESP32 Python “What is the baby temperature now”
  10. ESP32 Send to TTS “The temperature now is 30 degree”
  11. The return wav or Flak been played to using MAX98357A over I2S
  12. Loop command can be continue until will be no command for 30 sec, than the keyword "Lovey’s", must say again.
  13. So, I believe to expose I2S code from ESP32 C to Python will not be a big work. a. Google API use wav Audio Encoding name: FLAC Free Lossless Audio Codec 16-bit or 24-bit required for streams. b. So, FLAC can be done in C or Python. But interface must be from Python. See https://goo.gl/mtrwVN
  14. Maybe this is a good source for FLAC in C https://goo.gl/cCYFTh Contact : Nissim Zur nissim@telnt.com Skype: nissim.test
miketeachman commented 5 years ago

As part of an air and noise pollution sensor project (https://hackaday.io/project/162059-street-sense) I needed to support an I2S microphone using the ESP32 port. Using the ESP-IDF I implemented support for both I2S Master Rx and Tx . I spent a good deal of effort to bring the C code from a "just works hobbyist" quality to "industrialized" quality.

I put in a PR to mainline uPy: https://github.com/micropython/micropython/pull/4471

I prepared detailed class usage documentation and working uPy examples for popular breakout microphone and audio DAC breakout boards. Here: https://github.com/miketeachman/micropython-esp32-i2s-examples

Feedback and comments welcome at the PR or at the documentation repo

Mike

nickzoic commented 5 years ago

Oh, fantastic! By a curious coincidence I'm just making a start on https://github.com/adafruit/circuitpython/issues/1035 I'll have a look at this tomorrow & do some testing w/ hardware.

-----Nick

AidanTek commented 5 years ago

Great Mike! - I think I can do some testing tomorrow evening as well, I have i2s modules for audio input and output

sixtyfive commented 5 years ago

A naive question from someone who's never done such a thing: is I2S support good enough already to be able to attach line-out from some kind of amplifier (hifi, mobile phone, laptop, etc.) to an ESP32 via a 24-bit audio ADC?

AidanTek commented 5 years ago

A naive question from someone who's never done such a thing: is I2S support good enough already to be able to attach line-out from some kind of amplifier (hifi, mobile phone, laptop, etc.) to an ESP32 via a 24-bit audio ADC?

I have some PCM1808 which are 24-bit 3Vpp analogue stereo in. Once I have some basic stuff working I am happy to try stuff out - what exactly did you want to do with the ADC?

Mike's examples suggest that you can record to SD and playback from .wav format - super exciting!!

sixtyfive commented 5 years ago

@AidanTek I'd like to use an ESP32 to send the output from my hi-fi amplifier to another esp32 connected to the same wifi network that would get a DAC and where i could plug headphones in. doing the SMD work is fine, i think i've learned all the necessary things for that. i've also worked with I2C before, but never with I2S...

AidanTek commented 5 years ago

In theory it is possible, but no idea how to implement it :) - I would start by looking at how bluetooth speakers work (decoding digital audio at the speaker end) and otherwise maybe internet radio?? Of course the radio capabilities of the ESP32 bring some interesting options to i2s applications

PS the SGTL5000 is a 24-bit i2s device that has both line-in and out and a built in headphones amp - but it is a QFN package which is achievable but pretty un-fun to reflow at home!! That's the chip used on the Teensy Audio Shield - it is very good!

BEMNETADEFIS commented 5 years ago

Dear mike I have question for you did you use pic32mx series mcus does microchips 32-bit mcus and sam32 and arm 32 imx6,im7,imx8 processors can be programmed using micropython does it feets tham combining fpga or cpld devices What is your recommedation on this becouse u bulit library for one of an applications of these devices used in commerical boards regards bemnet

On Feb 10, 2019 6:35 PM, "Mike Teachman" notifications@github.com wrote:

As part of an air and noise pollution sensor project ( https://hackaday.io/project/162059-street-sense) I needed to support an I2S microphone using the ESP32 port. Using the ESP-IDF I implemented support for both I2S Master Rx and Tx . I spent a good deal of effort to bring the C code from a "just works hobbyist" quality to "industrialized" quality.

I put in a PR to mainline uPy: micropython/micropython#4471 https://github.com/micropython/micropython/pull/4471

I prepared detailed class usage documentation and working uPy examples for popular breakout microphone and audio DAC breakout boards. Here: https://github.com/miketeachman/micropython-esp32-i2s-examples

Feedback and comments welcome at the PR or at the documentation repo

Mike

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/micropython/micropython-esp32/issues/185#issuecomment-462142467, or mute the thread https://github.com/notifications/unsubscribe-auth/APM4lwL6IkZ76N5vcC0uReqaEhZUnLJxks5vMDwxgaJpZM4Pil7t .

miketeachman commented 5 years ago

Hi Bemnet, I'm sorry, but I'm having some difficulty to understand your question. I think you would like to learn what microcontroller platforms support MicroPython. Is that right? This thread of discussion is focused on the I2S protocol, available on the ESP32. So, it might not be the best place to seek advice on MicroPython support. Do you know about the MicroPython Forum? That is an excellent place to ask general questions. There is a very active and supportive group of MicroPython experts who will be able to help.
https://forum.micropython.org/ Perhaps the "General Discussion and Questions" is a good place to visit? Good luck! Mike

AidanTek commented 5 years ago

On Tuesday at a local maker meet I had a go with your fork @miketeachman - I tried setting up to play a .wav from a micro-SD - I think I have a script that works based on tweaking your examples, but didn't get any sound, however ran out of time to troubleshoot - as I was walking home that night I realised that I had left some pins on the codec floating where I would normally have tied to ground, and need to look at my wiring again regardless - will report back soon. I'm using the Lolin D32 and a PCM5102.

sixtyfive commented 5 years ago

In theory it is possible, but no idea how to implement it :) - I would start by looking at how bluetooth speakers work (decoding digital audio at the speaker end) and otherwise maybe internet radio??

I'll take a look once we have our workshop set up. Atm there's no time for actual fiddling.

PS the SGTL5000 is a 24-bit i2s device that has both line-in and out and a built in headphones amp

Thanks for the suggestion! Might give it a try...

but it is a QFN package which is achievable but pretty un-fun to reflow at home!! That's the chip used on the Teensy Audio Shield - it is very good!

Won't get around learning how to reflow QFNs anyways. Too many useful ICs are QFN these days.

asturkoglu commented 5 years ago 
Ready to download this file,please wait!
..................................
download ok
exec(open('read-mono-mic-write-internal-flash.py').read(),globals())
T

I tried @miketeachman 's example with adafruit sph0645 and esp32 devkit. it resulted with above output, nothing happened. I don't know it is esp32 firmware related or not.

Edit: I've tried with different board. (ESP32 Lite CH340) This time I faced with an error about I2S. I guess "T" stands for Traceback, the main error was cannot import name I2S

curlyz commented 5 years ago

Hmmm, this problem is actually the uPyCraft bug, I have experienced this before, just use uPyCraft V0.28 or later. The letter "T" mean that there is a problem , Python output a traceback , but uPycraft only show the first letter of that message.

nickzoic commented 4 years ago

This is micropython/micropython#4170 now so I'm closing this to tidy up.