SD card Load/Save Button is intermittent from Web UI interface

[steve6375]

I have original h/w - ESP32-WROOM-32D + ILI9341 + touch + VS1053 decoder + MicroSD mounted in MiniSD card holder and contacts soldered directly. It is constructed using breadboard.

The problem is during testing I have always had problems with the Web UI using 'Save SD' and 'Load SD'. It is very intermittent. e.g. 'Load SD' button presses in sequence - 1 OK - 1 FAIL - 3 OK - 2 FAIL - 1 OK - 2 FAIL - 3 OK... etc. Different pattern every time I reboot. Playing from an mp3 file from the SD card seems OK. This 'Load from SD' intermittent behaviour is the same whether not playing radio statio, playing radio station, playing mp3 from SD file or with stopped mp3 play.

SD card always loads OK on startup or reset and icons on TFT screen always look OK - web UI icons also look OK (when page is successfully loaded). I have tried many SD card fixes, 10K and 5K pullups, different card shields, programmed D0 as INPUT not INPUT_PULLUP, different TF and micro-SD cards, etc. but no difference. SD cards seems OK 40MHz SDMMC_FREQUENCY 40000000 and slower frequencies all OK. I have used an oscilloscope to check SD card signals and they look OK.

e.g. Successful button click

1. I click 'Load from SD' button in web UI
2. SD_CLK signal starts first - then MOSI output signal is active then MISO - SD_CLK stops
3. Web browser lists stations in table - all OK

e.g. Unsuccessful button click

1. I click 'Load from SD' button in web UI (button goes red - then green again)
2. _No SDCLK signal starts - it seems SD card command is never started by esp32 ?
3. Web browser lists 'Not found' in web table (so it seems wifi server did get web UI command but could not return information?) - see Picture 2.

Incidentally, if I press F5 (refresh) in browser, I often get a browser error for a few seconds (intermittent).

I would appreciate any clues on where to look? Is this a wifi issue?

Does anyone else get similar issues when using web UI 'Load from SD', etc.? If it is working OK for you, what h/w are you using or did you also have problems?

[Hhaltt]

I have the same problem. Stations work and switch normally - but in the web interface "Not found". And also sometimes the web interface does not load

[steve6375]

Update: I put in some debug lines. When you click on 'Load from SD' it sends a POST command for SDstations.csv to the web service. However, it never gets this event!

When it does get the event then the list is filled. If you click on the 'Load from SD' button and the debug message does not appear then you get the error message in the web UI.

So for some reason, the POST message from the browser is not received or missed maybe?

//Events from websrv
void WEBSRV_onCommand(const String cmd, const String param, const String arg){  // called from html

    if(CORE_DEBUG_LEVEL == ARDUHAL_LOG_LEVEL_VERBOSE){
        SerialPrintfln("WS_onCmd:    " ANSI_ESC_YELLOW "cmd=\"%s\", params=\"%s\", arg=\"%s\"",
                                                        cmd.c_str(),param.c_str(), arg.c_str());
    }

    String  str;
....

    if(cmd.startsWith("SD")){       str = cmd.substring(2);
                                    SerialPrintfln("***********Get file " ANSI_ESC_CYAN "%s" ANSI_ESC_WHITE " SD command received", str.c_str());
                                    if(!webSrv.streamfile(SD_MMC, scaleImage(str.c_str()))){
                                        webSrv.streamfile(SD_MMC, scaleImage("/unknown.jpg"));}
                                    return;}

14:04:15 AUDIO_info:  new request: "https://hls-01-europaplus-top40.emgsound.ru/21/112/out183926.ts"
14:04:17 ***********Get file /stations.csv SD command received
14:04:21 AUDIO_info:  new request: "https://hls-01-europaplus-top40.emgsound.ru/21/112/out183927.ts"
14:04:27 AUDIO_info:  new request: "https://hls-01-europaplus-top40.emgsound.ru/21/112/out183928.ts"
14:04:31 ***********Get file /stations.csv SD command received
14:04:33 AUDIO_info:  new request: "https://hls-01-europaplus-top40.emgsound.ru/21/112/out183929.ts"
14:04:39 AUDIO_info:  new request: "https://hls-01-europaplus-top40.emgsound.ru/21/112/playlist.m3u8?hlssid=65635afdfa82474ab066fa4434faf98a"
[117603][E][vs1053_ext.cpp:1189] readPlayListData(): m3u8 playlists requires PSRAM enabled!
14:04:39 AUDIO_info:  new request: "https://hls-01-europaplus-top40.emgsound.ru/21/112/out183930.ts"

[steve6375]

P.S. I also found that you must make sure that you only have one browser or one browser pane open for the 192.168.1.xx webradio ip address. If you have multiple browser windows (multiple 'connections' open) then it seems to have a lot more problems??? - or maybe it's my imagination.

[steve6375]

My esp32 does not have PSRAM. Maybe it would be better if it did? Does yours have PSRAM?

[valera39]

steve6375, I also have the same problem as you described. One radio with the same configuration as yours, the other on ESP32S3 with unused PSRAM. I keep postponing this question to dear Wolle, because I have other more important questions

[steve6375]

I notice that some web UI buttons always work. These seem to use socket.send in the html code. The UploadTextFile, loadGridFileFromSD, uploadCanvasImage and index page form (approx. line 1861 in index.h) all use POST.

[steve6375]

The following code works by resending the request if it fails. This fixes the problem for loading the SD card button

function loadGridFileFromSD () { // load from SD
  var XLrowObject
  var rawFile = new XMLHttpRequest()
  var errorCount = 0
  rawFile.onerror = function (event) {
  openAndSend(rawFile, ++errorCount)
  }
  rawFile.onload = function (event) {
    if (rawFile.status < 200 || rawFile.status >= 300)
    openAndSend(rawFile, ++errorCount)
  }
  rawFile.ontimeout = function (event) {
    openAndSend(rawFile, ++errorCount)
  }
  rawFile.onreadystatechange = function () {
    if (rawFile.readyState === 4) {
      var rawdata = rawFile.responseText
      var workbook = XLSX.read(rawdata, {
        raw: true,
        type: 'string',
        cellDates: false,
        cellText: true
      })
      workbook.SheetNames.forEach(function (sheetName) {
        XLrowObject = XLSX.utils.sheet_to_row_object_array(workbook.Sheets[sheetName])
      })
      var strJSON = JSON.stringify(XLrowObject)
      var objJSON = JSON.parse(strJSON)
      $('#jsGrid').jsGrid({
        data: objJSON
      })
      updateStationlist()
    };
  }
openAndSend(rawFile, ++errorCount);
}

function openAndSend(request, errorCount) {
  var limit = errorCount
    ? Math.pow(2, errorCount - 1) * 10
    : 0
  limit = limit > 1000 ? 1000 : limit
  var pause = Math.random() * limit
  setTimeout(function () {
requestopen('POST', '/SD/stations.csv', true)
request.send()
  }, pause)
}

Here is the uploadTextFile code

function uploadTextFile (fileName, content) {
  var fd = new FormData()
  fd.append('Text=', content)
  var theUrl = 'uploadfile?' + fileName + '&version=' + Math.random()
  var xhr = new XMLHttpRequest()
  var errorCount = 0

  xhr.onerror = function (event) {
    xhrSend(xhr, ++errorCount, theUrl, fd)
  }
  xhr.onload = function (event) {
    if (xhr.status < 200 || xhr.status >= 300)
    xhrSend(xhr, ++errorCount, theUrl, fd)
  }
  xhr.ontimeout = function (event) {
    xhrSend(xhr, ++errorCount, theUrl, fd)
  }

  xhr.onreadystatechange = function () { // Call a function when the state changes.
    if (xhr.readyState === 4) {
      if (xhr.responseText === 'OK') alert('OK: ' + fileName + ' successfully uploaded')
 //     else alert('ERROR: ' + fileName + ' not uploaded')
    }
  }

console.log (fd)
xhrSend(xhr, ++errorCount, theUrl, fd);
}

function xhrSend(request, errorCount, myurl, data) {
  var limit = errorCount
    ? Math.pow(2, errorCount - 1) * 10
    : 0
  limit = limit > 1000 ? 1000 : limit
  var pause = Math.random() * limit
  console.log (data)
  setTimeout(function () {
request.open('POST', myurl, true)
request.send(data)
  }, pause)
}

[valera39]

Where does this code need to be added?

[steve6375]

It replaced the existing code in index.h index.zip Here is mine which I have added many changes You can easily modify the 'Steve's MiniWebRadio' string and in Info/About screen!

It now works very well - just initial load of index.html when first type 192.168.1.xxx into a browser is still a problem - need to just keep hitting F5 refresh until the esp32 'sees' the request!

[steve6375]

The esp32 is still missing packets when using a lot of the other buttons, e.g. treble, bass, etc.

This is easy to see if you press F12 in the browser and click the Console button you can see frequent error messages after clicking buttons. The browser never gets a reply (presumably because it missed the packet?). The Vol+, Vol- and Mute buttons use socket.send and these seem 100% reliable. So it seems to be the POST and GET commands which are intermittent. It seems solution is to change index.h to use socket.send for all POST/GET commands?

I have experimented with yield() and also not initialising IR, ftp and soap and still get missed packets but sockets seem to be reliable (I think). Not sure where the problem is! Any hints or help appreciated. This is a great project and it would be really nice to make it reliable!

[steve6375]

index.zip I have added a simple retry to the other web commands. This index.h is for the latest small_buttons skew. It seems to be much more reliable now. I am not a html programmer so my code may be buggy but it seems to work. It would be better if the xmlhttprequests used sockets instead maybe, but I dont know how to do that! I see others have complained about the esp32 WROOMs missing wifi packets and they think its a problem in the wifi library. I have tried to remove all other things from setup and loop (e.g. ir, soap, fttp, etc.) and the esp32 still misses packets.