fnarify / VS1053-Arduino-Music-Player

General music player and recorder based on the VS1053 chip for Arduino
8 stars 1 forks source link

use an external SD reader for recording #1

Closed x6000c closed 3 years ago

x6000c commented 3 years ago

Hi, I am using your code to record ogg files with an Arduino UNO clone and a VS1053 mp3 shield (clone). Unfortunately my shield has a low quality on-board SD reader so I want to use an external one. I was not able to record any sound after having changed the last line of the following code snippet in SFEMP3ShieldConfig.h

`// otherwise use pinout of typical Sparkfun MP3 Player Shield.

else // otherwise use pinout of typical Sparkfun MP3 Player Shield.

define MP3_XCS 6 //Control Chip Select Pin (for accessing SPI Control/Status registers)

define MP3_XDCS 7 //Data Chip Select / BSYNC Pin

define MP3_DREQ 2 //Data Request Pin: Player asks for more data

if defined(__AVR_ATmega32U4__)

#define MP3_DREQINT          1 //Corresponding INTx for DREQ pin

else // swapped between Uno and Leonardo.

#define MP3_DREQINT          0 //Corresponding INTx for DREQ pin

endif

define MP3_RESET 8 //Reset is active low

if ( GRAVITECH == 1 )

#define SD_SEL               4 //select pin for SD card

else

#define SD_SEL               5 //select pin for SD card`

a file is being created though and it is also filled with bytes but there is no sound in it. Also it cannot be read by Audacity. When I use the original pin 9 at least some sound is recorded and the file can be played. I have tested the shield and the reader with the Adafruit VS1053 library and both work (on pin 5 as CS pin for example). The recordings are not safisfactory though.

So I would like to be able to use pin 5 (or 4 or 3) for recordings. How can that be done?

Thank you!

fnarify commented 3 years ago

It's been a long time since I've done anything with this library, or Arduino in general, so I'll see if I can help a bit.

Generally some more detail on the VS1053 module you're using specifically, same with the SD card module you're using, and whether the onboard SD card socket could do with a filtering capacitor, or maybe the pull-up resistors it's using are no good.

First up, you shouldn't need to change anything in the provided libraries, for SD_SEL you can just directly change the value on line 149 of the .ino file ( sd.begin() ) to the pin value you want, SD_SEL isn't used anywhere else that's important. What you can try doing is connecting the SD card's CS pin to pin 10 of the Arduino, as pin 10 can be used since it can only be used as an output. This might be useful as there aren't any other spare pins available. You could also disconnect the keypad and just use the serial monitor for testing to save on a few pins. The data lines for the SD card are fine with the defauult MISO and MOSI pins on the Arduino, they should be able to run over the top of the onboard SD card assuming the CS pin is different (which you might want to check for your module).

The only thing library wise that's important to check is around line 716 of SFEMP3Shield.h where these function prototypes listed should be changed from private to public: void getTrackInfo(uint8_t, char, uint8_t);
static void Mp3WriteRegister(uint8_t, uint16_t); static uint16_t Mp3ReadRegister (uint8_t);
uint8_t VSLoadUserCode(char
);

Now quality wise using the on-board SD card it might be poor due to whatever recording device you're using. The onboard microphone is pretty bad I found with my module, and if you're using an external one you'd want to check it's impendace as the module might not be able to supply it with enough power. The maximum rated impedance is 30 ohms for the VS1053 IC. Especially when you consider unless there's something wrong with the tracks or the circuit, an SD card socket is just a socket there's not a lot of variation unless there's an issue with the solder joints.

Similarly, it might be worthwhile reformatting the SD card (FAT16/32) as there might be issues with the wear level of the SD card causing it not to write to it correctly, although unlikely as it seems you've tested it fine otherwise.

Otherwise in terms of recording quality, there are quite a few different recording profiles available from the manufacturer's website (http://www.vlsi.fi/en/products/vs1053.html) which you can convert using their batch file which is also provided in the plugins folder of this repo. Details on the profiles can be found here: http://www.vlsi.fi/fileadmin/software/VS10XX/VorbisEncoder170c.pdf

The profile I converted to make oggenc.053 which is needed for recording is called venc44k2q05, which is 2-ch and had the highest bitrate at 135kb/s. Let's be honest though 135kb/s at 44.1kHz is just okay and nothing that special. Do make sure both patches.053 and oggenc.053 are in the root directory of the SD card as well, although I assume this is correct as you'd get a warning otherwise. Interestingly enough, the manufacturer is still updating the chip with new patches, and they could be worth trying, but I doubt it'd change that much.

Otherwise, in terms of other things to consider might be the write block it does for the recording. This is done in two writes of 256 bytes with an exception if it's the last data block. The reason for this was because any larger wouldn't work as there isn't enough spare memory with the Arduino Uno. It's possible this might be writing too often for the SD card causing it to skip data. There isn't really much of an alternative though, apart from adding a short wait between writes.

x6000c commented 3 years ago

Hi, and thank you for your reply!

the parts I'm using are these: https://de.aliexpress.com/item/32965676064.html https://de.aliexpress.com/item/32887226075.html https://de.aliexpress.com/item/32728051288.html

I forgot to mention: I am not using a keypad and no display. I only use the funcitons '*' for recording ogg and 'f' for playing it. So I removed the parts that are unnecessary in my case in the code.

the lines you mention which need to be changed from private to public have already been public.

I can use pins 10,5,4,3 for SD_SEL. Again: the file is written and filled with bytes as if it was recording but there seems to be no readable data. With the Adafruit_VS1053 library (using the external reader) I got almost acceptable recordings except for the hums that originate (most likely) from the file write function. In that Adafruit sketch I have experimented a lot to get rid of the hums, like you suggested, also changed the chunks of bytes to be written, inserted delays (which made it worse as additional high pitched noise occured) and so on. The only method I can think of to improve this is to reduce the period of time for the write process. And since your code uses SdFat and that is said to be much faster I was going to try that.

The onboard microphone is good imo. When I speak into it the speakers emit a clear sound of my voice. I think the noise/ hums occurs with the SPI bus or on the way to the ADC (where the analog signal might pick up the noise from the SPI).

Unfortunately I am still stuck. I can record no sound on the external SD reader. I have also tried a different SD card. Yes, I added the two files to the root directory. I think the mic is mono though. With the Adafruit sketch I use venc44k1q00.img (has to be renamed to v44k1q00.img because of file name restraints). But I also tried almost all other img files (also the stereo ones). The best results I got was with the 44k1q00 as it had good quality with a hum every second or so and a hum length of .4s.

btw, I've also started a topic here a few days ago: https://forum.arduino.cc/index.php?topic=722928.0

I have uploaded a test recording with the external reader here (no sound, unreadable with Audacity except in RAW mode): https://filebin.net/fum2hs5vwr8z8za5/TEST2.OGG and one with the on-board reader (poor sound but readable with Audacity): https://filebin.net/fum2hs5vwr8z8za5/TEST4.OGG

I have a suspicion that - once the external reader works - the sound may be better with venc44k1q00.img and your sketch as well. Are the respective patch files available in the library and if so which ones would they be?

I just noticed a line in your sketch: #if USE_MULTIPLE_CARDS sd.chvol(); // assign desired sdcard's volume.

could that perhaps have something to do with the external reader not recording audio data?

fnarify commented 3 years ago

The most important detail is likely in the section at the end, but I'll leave all this other stuff at the front.

Yes the microphone at least on my board is mono. I found it pretty tinny, your VS1053 module is the same as mine, although I had come across some with damaged components in the past, but that's not a huge surprise considering how cheap it is. I assume when you're recording that once you want to stop it you send the character 's' via the serial console to indicate end of recording?

The patches can be found at: http://www.vlsi.fi/en/support/software/vs10xxpatches.html There is a change log somewhere on their site, but I can't find it currently, the patch I supply in the repo would be from around early 2017 / late 2016.

that line on the def USE_MULTIPLE_CARDS isn't used anyway as it's always set to 0. and chvol() just sets the working directory. I wouldn't be surprised if it is a noise issue from the rest of the circuit, I do distinctly remember noticing the same type of noise when I was first putting the project together, but can't quite recall what I did to fix it.

This two posts might have a fair bit of relevance: https://electronics.stackexchange.com/questions/67914/mp3-player-shield-makes-noise https://www.eevblog.com/forum/projects/recording-audio-using-the-vs1053-encoder-chip/ Since when I did testing, I used a spare plugpack I had lying around.

I also didn't mention it in my previous reply, but on line 149 where the SD card is initialised, you can try changing SPI_HALF_SPEED to SPI_FULL_SPEED or more likely SPI_QUARTER_SPEED. I should be able to dig up my old prototype when I go into the office tomorrow. If I have time I'll grab a spare SD card module and see if I can get it to run. Although I'm a little baffled it wouldn't work, but this module had a lot of peculiarities I found when working on it, such as needing to be reset all the time after playing or recording audio.

When I was working on this project I had a lot of correspondence with another person who helped me track down quite a few bugs and led me to addd a large amount of features, lucky for me I found some of his emails I'll reproduce some of what was said below and see if I can dig up anything else tomorrow:

I would recommend trying the zener diode addition if you want, as that fixed the buzz for the person I was talking to at the time.

x6000c commented 3 years ago

yes, I type s to end the recordings. Yesterday I noticed a couple of things. The SD led lights up much brighter during writing when using the Adafruit sketch. I think that this is due to the slower speed of the SD library compared to SdFat. But it may indicate a problem with the printed wiring trace as you say and explain why the data isn't recorded properly and perhaps the speed of the SD library is just slow enough to work with the external reader and the Adafruit sketch.

I also checked if there is any data coming in via wordsWaiting on the Serial monitor with Serial.println inserted here:

wordsToWrite = wordsToRead / 2; Serial.println(wordsWaiting); for (uint8_t i = 0; i < 2; i++)

in the Adafruit sketch I had been using v44k1q00.img so far. To compare both sketches I have first tried yours, then the Adafruit with v44k1q05 and then v44k2q05. Your sketch uses v44k2q05.img, here is the output with it:

2422 2166 1910 1654 1398 1142 886 630 374 118 1632 1376 1120 864 608 352 96 3435 3179 2923 2667 2411 2155

while the Adafruit sketch (v44k1q05) something like this:

229 229 229 229 261 2017 2210 2117 2038 246 246 246 246 ... more of the 246 246 246 246 246 246 246 246 246 246 246 246 246 246 246 254 597 1787 2301 2253 2164 2084 2020 228 228 228 228 228 228

but when I replace the patch with v44k2q05.img in the Adafruit sketch I get something more similar to the output I get from your sketch AND there's a lot of chunks in the recording missing as well as audible distortions:

81 81 81 81 81 81 81 81 81 81 81 81 81 81 81 81 81 81 81 81 81 81 81 81 81 81 81 81 81 81 92 395 1869 2021 3799 2092 4045 4073 1707 2192 4068 2184 4052 2126 4059 2073 4008 3972 2117 2049

so maybe the external SD reader is not able to keep pace with the data influx at the higher qualities. The Adafruit sketch - although using the slower SD library - somehow still copes with the highest audio quality and outputs something that can be considered a recording and be read by Audacity (though unusable). I would have suspected the opposite, i.e. that using the SdFat library would be faster - maybe it is but for some reason the data transfer to the card seems slowed down.

so maybe the solution would be in using a lower quality patch like v44k1q00 and see if it works. But I wouldn't know how to do that i.e. creating the oggenc and the patch file. Or can I find the two files in the VLSI link you provided?

as for the recording quality itself I will definitely have a closer look at the links you provided. I already have some of these: https://de.aliexpress.com/item/32976461912.html maybe they will come in handy at some point. As for powering the on-board mic separately I wouldn't know how to go about that.

I had already tried the different SPI speeds as you suggested but to no avail. I also tried pin 9 as SD_SEL (which is the CS pin for the in-built SD reader I guess?) and access is ok on that pin. Furthermore I checked what happens if I Set USE_SOFTWARE_SPI to nonzero in SdFatConfig.h and noticed on initialising (starting the sketch) through the speakers the same kind of 'activity sound' like when I use the Arduino sketch. But then the sketch hung up, so I set it back to zero.

Yep, I noticed the reset problem, too. But this issue got less annoying when I used a different card, a 4 GB and a 32 GB SDHC instead of a 2 GB SD which may have been too slow.

The zener diode sounds like a good idea. Btw, I am using the external reader with 5V (does not run on 3.3V). But that doesn't mean that I have to insert the diode between 5V and ground, or does it?

x6000c commented 3 years ago

meanwhile I figured out how to convert plg files to 053. Had to install Perl for that though. I am not sure whether the converted file (venc44k1q00.plg to v44k1q00.053) corresponds to your patches.053 or oggenc.053. And if it's one of them, how do I create the other, since there are only .img and .plg files in the VS1053 Ogg Vorbis Encoder Application I found here: http://www.vlsi.fi/en/support/software/vs10xxapplications.html and .img files cannot be converted with the Perl command line application.

Which zener diode would you recommend?

I found these: https://de.aliexpress.com/item/1005001901464681.html and these: https://de.aliexpress.com/item/32632111642.html

would they do and if so, which voltage do I need? (please note last comment at bottom)

fnarify commented 3 years ago

I just used a simple 3.6V 1W zener diode, a 1N4729 type will do, it comes in a DO-14 package. If you have a local retailer for electronics parts it should be under a dollar for one. The recommendation for the zener diode was to solder it directly on the VS1053 module board, this means using the on-board SD card rather than an external. I at least know it fixed the hum issue on audio for one other person I dealt with.

If you're using the external SD card the zener won't make a difference, as this is specifically due to the 3.3V line reading too high and a 3.6V zener isn't going to help being connected to the 5V line. If you have a multimeter lying around, you can test the 3.3V/5V supply lines on the VS1053 and SD card module and see what voltages you get.

Also, while I doubt it's the main issue, trying different power sources is worthwhile, as getting power from a computer's USB port can lead to voltage swings that mess with badly designed boards. If you have a reliable plug pack (9-12V DC @ 1A; I think the 5V line has a maximum current draw of ~0.8A, so maybe the SD card is drawing too much power) that's worth trying, or some batteries. The type-B USB port on the Arduino can only draw about 0.5A, so potentially not enough for your setup.

This also might be relevant: https://arduino.stackexchange.com/questions/22426/sd-card-module-doesnt-work-with-external-power-supply

The files in the form venc44k1q00.plg etc correspond to oggenc.053. The patch files (corresponding to patches.053) can be found at: http://www.vlsi.fi/en/support/software/vs10xxpatches.html called vs1053b-patches290.zip Take one of the patch files, like the default "vs1053b-patches.plg" in that ZIP folder and run "vs_plg_to_bin.pl vs1053b-patches.plg patches.053" to convert it. There's some detail on the process at https://mpflaga.github.io/Arduino_Library-vs1053_for_SdFat/

x6000c commented 3 years ago

ok, I will go with this one then: https://aliexpress.com/item/32855215596.html , should be enough to experiment with

unfortunately the new patch didn't work either. It's a mystery to me why it would play ogg files without any issues but recording doesn't work properly, even with lower quality. I bet there is a minor change that has to be made or the design of the card reader has some flaw that only will show when using SdFat. If I should find out what causes it I will post it here.

I had tried many power sources (except for one with a voltage regulator). That includes two 18650 batteries as well. With all of them the hum occured in the same manner. Also tried powering the reader separately. What I haven't tried yet though is a powering the Arduino via the Vin pin with more than 5V. I have had some good experience with another project including a GPS module where the voltage supply by both USB and barrel jack were too weak.

If the zener diode works on the on-board SD I will be happy. Just one last question to make sure I don't mess this up. With 3.3v and ground do you mean the 3.3v and GND pins on the Arduino UNO (so I woud have to solder to the bottom side pins) or do you mean the input leads/ traces to the on-board SD reader (where would they be located? haven't found a pinout for this particular shield) in latter case I would think a SMD zener diode would be more suitable, right? something like this:

https://www.aliexpress.com/item/1335538281.html

fnarify commented 3 years ago

For the diode, between the 3.3v and GND pin of the VS1053-based module, you can just solder it on the underside. If you can wait till about tomorrow I can photograph what I did on my board, so I can double-check.

x6000c commented 3 years ago

yes, a photo would be very helpful. Sure, I can wait

x6000c commented 3 years ago

here's a direct comparison of the outputs from the Adafruit sketch and the SFEMP3Shield sketch: https://filebin.net/b6jbltuyfncm8qgg/spec.jpg

(side note: filebin uploads only last a few days)

both recording snippets only show the hums and both use the v44k1q00 patch (same audio quality)

the upper recording (Adafruit) shows indeed longer humming while the volume of the lower (SFEMP3Shield) is stronger. The Adafruit recording was made with the external SD reader, the SFEMP3Shield recording was made with the on-board SD reader.

so it indeed seems that the SdFat is somewhat faster. Also I was able to get the same audio quality (except for the hums that differ) as with the Adafruit sketch. For that I adjusted the volume to 1, 1 in MP3player.setVolume(1, 1); actually both functions look and probably work alike (corresponding Adafruit line: musicPlayer.setVolume(1,1);)

thus I definitely want to try that zener diode and I'm optimistic it will work. The additional advantage would be that less hardware is necessary (the external sd reader will be gone)

fnarify commented 3 years ago

It seems like on my original board I just slotted the zener diode into the header on the VS1053-based module. That's probably the easiest way to test it since it doesn't permanently change the board.

So in this case, take an equivalent 3.6V zener diode (rated from 0.5 to 1W) and just plug it into the female header on the VS1053 module (in my case a Geeetech shield). The anode of the diode goes to GND, and the cathode (black strip) goes to 3.3V. Since there should be a spare GND pin on the MP3 module and the 3.3V pin isn't being used by anything either.

Here's a quite poor drawing as an extra indication: https://imgur.com/a/N6OBNKy

There's 3 GND pins to choose from, so if you find it works and want something a bit neater, you can just use an SMD zener diode (although it might be a bit annoying to solder it on there) like you were asking before or cut the leads shorter on a through-hole one and just solder it to the top of the male headers on the module.

x6000c commented 3 years ago

that's excellent! I had already feared I would have to solder it on the tiny traces of the SD slot. Okay, I'm going to order those diodes now, perhaps I am also going to order the 5.1V ones to test if they also work with the external reader. In any case thank you for taking the time and help me with this. My original question was about the usage of the external reader and now - thanks to you - I have a potentially much better solution. Even if it should not work I will have some useful spare parts for future projects.

As soon as I have the diodes here - which may take a couple of weeks because I have to order them on aliexpress, couldn't find this particular type of diode on German websites (electronic stores are closed at the moment due to lock-down), except for may be this one https://www.pollin.de/p/zenerdiode-zpd3-6v-140763 I will report back here and let you know if it worked.

In the meantime we can discuss this topic further if you like. Or: I am not sure if this thread has to be "Closed with comment" to mark it as solved. If so, I can do so if you wish. Just let me know. But in that I case I wouldn't be able inform you about the result. and it seems I can't send personal messages. Anyways jlmk.

fnarify commented 3 years ago

I may as well wait till there's proper confirmation it fixes it for you and it's not another edge case. I normally get my parts from Digi-Key or Mouser if I need them in a rush, the only issue is you need to spend above a certain amount to get free postage, but they're pretty damn fast. I'm at least lucky there's a fair bit of local choice where I live in Australia.

There's also RS electronics and Futurlec, they might be fairly cheap to send internationally.

x6000c commented 3 years ago

Ok, will check, and report back asap. Thanks again and cya soon! Stephan

x6000c commented 3 years ago

Hi! got the diodes today and just gave it a try. Same as before, also switched K and A of the diode. Buzz still there. Maybe it's the wrong diode type (it's a 1N4729A, DO-41, 3.6V 1W), I don't know. However. I had tried a different mic module in the meantime which works flawlessly. So I guess I'll use the shield as a MP3 player now :-)

Not sure if the issue cannot be replied to when I close it - in case you want to reply. So I'll leave it open for another couple of days or if you can close it, you can do so of course.

Anyways, thanks again for your extensive help and maybe the method works for others better than for me.

Wish you all the best Stephan

fnarify commented 3 years ago

That's honestly a real shame, since it fixed the issue for me and another person. The only cause I could think of at this point is that the microphone on your module is just defective in general, or more likely that one of the PCB tracks is bad/damaged. I find these cheap modules can be very hit or miss at times, but they're so cheap for what they offer it's kind of a trade-off.

I'll do some testing on my own as I have a few spare modules, but it might be down to luck (probably bad luck) to find one that behaves similarly. Hopefully everything else works out well for you. If you need help with anything else in regards to this I'm more than willing to help.

x6000c commented 3 years ago

ok, thanks for your reply. Probably you're right and it's a PCB track that causes the problem. Since I have the other mic module it's ok plus the Zener diodes (actually I ordered 10 different types for voltages ranging from 3 to 47V) are something that may come in handy once in a while. Will close the issue now and thanks for offering help in case of possible future questions :-)