s60sc / ESP32-CAM_MJPEG2SD

ESP32 Camera motion capture application to record JPEGs to SD card as AVI files and stream to browser as MJPEG. If a microphone is installed then a WAV file is also created. Files can be uploaded via FTP or downloaded to browser.
GNU Affero General Public License v3.0
819 stars 201 forks source link

[FEATURE REQUEST] I2S Speaker Output #404

Open rjsachse opened 3 months ago

rjsachse commented 3 months ago

I'm not sure if this is a too far stretch for the great project, but I think having a speaker will would be great. either play audio through upd or wav from sd card, then would be able to make this the perfect project for a door cam or surveillance cam with 2 way audio and telemetry

just a idea...

s60sc commented 3 months ago

could you give a more specific use case for the speaker.

rjsachse commented 3 months ago

Some Ideas

  1. Doorbell camera. Push button and plays a selectable doorbell chime sound.
  2. Doorbell camera with 2 way audio so you can talk to the person at the door
  3. Intercom so one mjpeg2sd could talk to another in a different room
  4. Pet camera so when your away you can see and talk to your pets
  5. Application voice feedback like Wifi Connected, camera recording etc.
  6. Surveillance camera with human voice deterrent pre recorded like "Recording Started"
  7. Make car/truck/tank sounds like a horn or rev engine through remote control
  8. Robot with vision, hearing and a voice in 1 perfect package

I know a few ideas could be classed as 1 but trying to give user cases. To be able to play sounds from sd card or stream to device, would be nice but both would be great. I know it could be difficult to implement. Think this would be a great next step into this project. Using a i2s MAX98357A amp board. Would only need 1 extra pin if using with i2s mic too

Off Topic. I am starting to get my head around your code and started to add a oled screen, hlk-ld2410 radar, also updated telemetry.cpp to use just BME280 or BMP280 or GY91 not conflicting, not sure if you wanted a pull request or leave telemetry.cpp as a user edit. Happy to help in anyway I can.

s60sc commented 3 months ago

I'll start with browser on mic enabled pc to send stream to esp32

telemetry.cpp intended as a user edit

Ritashaiev commented 3 months ago

what pins are you using for for i2s on esp32s3 cam wroom?

rjsachse commented 3 months ago

You can pretty much use any free pins for i2s

Ritashaiev commented 3 months ago

You can pretty much use any free pins for i2s

i tried a couple but some of them not quite ideal. i modify them in the menu but there's what i got: pin 14 may cause the led out of control pin 3 or 4 may interrupt the camera it worked immediately after pressing the reset button, but not working after a reboot pin 46/47/21 works but it only records sound in the first 3 seconds i will try other pins later or maybe reinstall the app

rjsachse commented 3 months ago

What is the pin map of your board. Sorry I don't have that board.

Ritashaiev commented 3 months ago

What is the pin map of your board. Sorry I don't have that board.

Freenove ESP32-S3-WROOM CAM Board (Compatible with Arduino IDE), Onboard Camera Wireless, Python C Code, Detailed Tutorial, Example Projects https://amzn.asia/d/eibWUoQ i bought the esp32s3 from the link images i don't know if this pin map is correct, the board looks the same

rjsachse commented 3 months ago

Are you talking about microphone or speaker

Ritashaiev commented 3 months ago

Are you talking about microphone or speaker

sry, microphone. i didn't think about using i2s as a speaker. maybe i reply in a wrong issue

rjsachse commented 3 months ago

47,21,42,41,1,14 should be free pins according to pin map. Yes this a feature request for i2s speaker. Not issue about microphone, I have no issues with microphone that I have noticed

Ritashaiev commented 3 months ago

the tricky thing is when i used pin14 it kept led on. idk why

rjsachse commented 3 months ago

Wrong place for issue. Create a issue, if there is a issue with a selected board in appGlobals.h

s60sc commented 2 months ago

Implemented code in v9.7.1 to stream device microphone via browser to speaker on esp32. This works fine on my VoiceChanger app, but not on this app due to lag probably as too much else going on. The speaker code and notes is in audio.cpp

rjsachse commented 2 months ago

For the life of me, I could not find the setting. Need to change appSpecific.cpp config going to submenu 9, not 3. Just found the issue, but there is no time to fix.

s60sc commented 2 months ago

Sorry, I did that to hide the parameters so it wouldn't confuse anyone else, in your copy change 9 to 3

rjsachse commented 2 months ago

Glad I worked it out lol, sorry not going to be able to test till about Sunday

rjsachse commented 2 months ago

I had a quick go today. Sort of got it working through phone and pc, but the mic disconnects randomly and get this error message sometimes [14:39:24.739 ERROR @ webServer.cpp:413] websocket receive failed with ESP_ERR_INVALID_STATE

s60sc commented 2 months ago

As before there is too much going on in the app to handle the input stream in a timely manner. May need a separate ESP32 dedicated to remote mic. I will be uploading the latest voicechanger app in a few days, could use that as a basis.

rjsachse commented 2 months ago

Looking forward to trying the voicechanger app. Is there anyway to use the same serial clock and LR clock for both the mic and speaker like in this app esp32-walkie-talkie

s60sc commented 2 months ago

Voicechanger v1.3 uploaded.

Clock pin can be shared if compatible, eg mic I2S SCK & amp I2S BCLK

rjsachse commented 1 month ago

Seems to work OK. I get websocket errors sometimes. Still haven't had much time to play around with it as life gets in the way. Very thankful for what you have done, please don't remove code.

rjsachse commented 2 days ago

Ok been having troubles getting this working again since arduino v3. Thought is was my amp board, waited for a new one, with the same result with the amp just playing a high pitch squeal. found a bug and created pull request. What was happening is that I have the i2s pins predefined in camera pins and found a issue in audio.cpp sd and ws pins was swapped. Haven't had a chance to see if it works yet.

Also seems in the latest version the amp option is removed from web server

s60sc commented 2 days ago

I removed the amp option as it added clutter for something that few people would use as its too much work for the S3 with this app. Better to use a separate esp as an intercom. But if you can get it to work efficiently I'll reinstate it.