Data folder is not flashed using Coral Dev Micro + Wireless Add-on board

arthurkafer commented 1 year ago

Description

Good afternoon,

I'm trying to create a simple HTTP server using Wi-Fi (and the add-on board). Code flashing is fine, but I suspect the /data files in the project are not being flashed to the board when the Dev Board Micro + Wireless Add-on is set on Arduino IDE.

Using the normal Dev Board Micro flashing the server works as intended, because the /data folder is flashed correctly.

Everytime I'm flashing using Sketch+Data option:

I'll put the code I'm using and the flash logs for both boards below.

Dev Board Micro + Wireless Add-on -> CameraStreamingHttp using WiFi

Dev Board Micro -> CameraStreamingHttp without WiFi.zip

Is there something missing in Arduino IDE to flash the /data folder together?

Click to expand!

### Issue Type Bug ### Operating System Windows 10 ### Coral Device Dev Board Micro ### Other Devices _No response_ ### Programming Language C++ ### Relevant Log Output ```shell *Selecting "Dev Board Micro + Wireless Add-on" board* Section: 0x0 iMX bootable image generated successfully Boot Section 0x00000000: FILL | adr=0x00010000 | len=0x00000004 | ptn=0xc2000105 FILL | adr=0x00010004 | len=0x00000004 | ptn=0x00010020 FILL | adr=0x00010008 | len=0x00000004 | ptn=0x00040004 FILL | adr=0x0001000c | len=0x00000004 | ptn=0x00080004 FILL | adr=0x00010020 | len=0x00000004 | ptn=0xc0010021 ENA | adr=0x00010000 | cnt=0x00000004 | flg=0x0110 ERAS | adr=0x00000004 | cnt=0x00000004 | flg=0x0110 ERAS | adr=0x00000008 | cnt=0x00000004 | flg=0x0110 LOAD | adr=0x00000004 | len=0x000178c4 | crc=0xbb66d509 | flg=0x0110 LOAD | adr=0x00000008 | len=0x000178c4 | crc=0xa1f260bf | flg=0x0110 Inject command 'receive-sb-file' Preparing to send 193248 (0x2f2e0) bytes to the target. Successful generic response to command 'receive-sb-file' (1/1) 0% (1/1) 1% (1/1) 1% (1/1) 2% (1/1) 2% (1/1) 3% (1/1) 3% (1/1) 4% (1/1) 4% (1/1) 5% (1/1) 5% (1/1) 6% (1/1) 6% (1/1) 7% (1/1) 7% (1/1) 8% (1/1) 8% (1/1) 9% (1/1) 9% (1/1)10% (1/1)10% (1/1)11% (1/1)12% (1/1)12% (1/1)13% (1/1)13% (1/1)14% (1/1)14% (1/1)15% (1/1)15% (1/1)16% (1/1)16% (1/1)17% (1/1)17% (1/1)18% (1/1)18% (1/1)19% (1/1)19% (1/1)20% (1/1)20% (1/1)21% (1/1)21% (1/1)22% (1/1)23% (1/1)23% (1/1)24% (1/1)24% (1/1)25% (1/1)25% (1/1)26% (1/1)26% (1/1)27% (1/1)27% (1/1)28% (1/1)28% (1/1)29% (1/1)29% (1/1)30% (1/1)30% (1/1)31% (1/1)31% (1/1)32% (1/1)32% (1/1)33% (1/1)34% (1/1)34% (1/1)35% (1/1)35% (1/1)36% (1/1)36% (1/1)37% (1/1)37% (1/1)38% (1/1)38% (1/1)39% (1/1)39% (1/1)40% (1/1)40% (1/1)41% (1/1)41% (1/1)42% (1/1)42% (1/1)43% (1/1)43% (1/1)44% (1/1)45% (1/1)45% (1/1)46% (1/1)46% (1/1)47% (1/1)47% (1/1)48% (1/1)48% (1/1)49% (1/1)49% (1/1)50% (1/1)50% (1/1)51% (1/1)51% (1/1)52% (1/1)52% (1/1)53% (1/1)53% (1/1)54% (1/1)54% (1/1)55% (1/1)56% (1/1)56% (1/1)57% (1/1)57% (1/1)58% (1/1)58% (1/1)59% (1/1)59% (1/1)60% (1/1)60% (1/1)61% (1/1)61% (1/1)62% (1/1)62% (1/1)63% (1/1)63% (1/1)64% (1/1)64% (1/1)65% (1/1)65% (1/1)66% (1/1)67% (1/1)67% (1/1)68% (1/1)68% (1/1)69% (1/1)69% (1/1)70% (1/1)70% (1/1)71% (1/1)71% (1/1)72% (1/1)72% (1/1)73% (1/1)73% (1/1)74% (1/1)74% (1/1)75% (1/1)75% (1/1)76% (1/1)76% (1/1)77% (1/1)78% (1/1)78% (1/1)79% (1/1)79% (1/1)80% (1/1)80% (1/1)81% (1/1)81% (1/1)82% (1/1)82% (1/1)83% (1/1)83% (1/1)84% (1/1)84% (1/1)85% (1/1)85% (1/1)86% (1/1)86% (1/1)87% (1/1)87% (1/1)88% (1/1)89% (1/1)89% (1/1)90% (1/1)90% (1/1)91% (1/1)91% (1/1)92% (1/1)92% (1/1)93% (1/1)93% (1/1)94% (1/1)94% (1/1)95% (1/1)95% (1/1)96% (1/1)96% (1/1)97% (1/1)97% (1/1)98% (1/1)98% (1/1)99% (1/1)100% Completed! Successful generic response to command 'receive-sb-file' Response status = 0 (0x0) Success. Wrote 193248 of 193248 bytes. Finding all necessary files Creating Filesystem STATE_CHECK_FOR_ANY STATE_CHECK_FOR_ELFLOADER STATE_RESET_ELFLOADER OpenHidDevice vid=18d1 pid=9307 ... OpenHidDevice vid=18d1 pid=9307 opened STATE_CHECK_FOR_SDP STATE_LOAD_FLASHLOADER STATE_CHECK_FOR_FLASHLOADER STATE_PROGRAM STATE_LOAD_ELFLOADER STATE_PROGRAM_DATA_FILES OpenHidDevice vid=18d1 pid=9307 ... OpenHidDevice vid=18d1 pid=9307 opened Transferring /third_party/cyw-bt-patch/BCM4345C0_003.001.025.0144.0266.1MW.hcd /third_party/cyw-bt-patch/BCM4345C0_003.001.025.0144.0266.1MW.hcd 10% /third_party/cyw-bt-patch/BCM4345C0_003.001.025.0144.0266.1MW.hcd 20% /third_party/cyw-bt-patch/BCM4345C0_003.001.025.0144.0266.1MW.hcd 30% /third_party/cyw-bt-patch/BCM4345C0_003.001.025.0144.0266.1MW.hcd 40% /third_party/cyw-bt-patch/BCM4345C0_003.001.025.0144.0266.1MW.hcd 50% /third_party/cyw-bt-patch/BCM4345C0_003.001.025.0144.0266.1MW.hcd 60% /third_party/cyw-bt-patch/BCM4345C0_003.001.025.0144.0266.1MW.hcd 70% /third_party/cyw-bt-patch/BCM4345C0_003.001.025.0144.0266.1MW.hcd 80% /third_party/cyw-bt-patch/BCM4345C0_003.001.025.0144.0266.1MW.hcd 90% /third_party/cyw-bt-patch/BCM4345C0_003.001.025.0144.0266.1MW.hcd done Transferring /third_party/firmware/cypress/43455C0.bin /third_party/firmware/cypress/43455C0.bin 10% /third_party/firmware/cypress/43455C0.bin 20% /third_party/firmware/cypress/43455C0.bin 30% /third_party/firmware/cypress/43455C0.bin 40% /third_party/firmware/cypress/43455C0.bin 50% /third_party/firmware/cypress/43455C0.bin 60% /third_party/firmware/cypress/43455C0.bin 70% /third_party/firmware/cypress/43455C0.bin 80% /third_party/firmware/cypress/43455C0.bin 90% /third_party/firmware/cypress/43455C0.bin done Transferring /third_party/firmware/cypress/43455C0.clm_blob /third_party/firmware/cypress/43455C0.clm_blob 10% /third_party/firmware/cypress/43455C0.clm_blob 20% /third_party/firmware/cypress/43455C0.clm_blob 30% /third_party/firmware/cypress/43455C0.clm_blob 40% /third_party/firmware/cypress/43455C0.clm_blob 50% /third_party/firmware/cypress/43455C0.clm_blob 60% /third_party/firmware/cypress/43455C0.clm_blob 70% /third_party/firmware/cypress/43455C0.clm_blob 80% /third_party/firmware/cypress/43455C0.clm_blob 90% /third_party/firmware/cypress/43455C0.clm_blob done Transferring /usb_ip_address /usb_ip_address done STATE_RESET_TO_FLASH OpenHidDevice vid=18d1 pid=9307 ... OpenHidDevice vid=18d1 pid=9307 opened Flashing to flash storage complete, the device is restarting to execute your application. *Selecting "Dev Board Micro" board* Section: 0x0 iMX bootable image generated successfully Boot Section 0x00000000: FILL | adr=0x00010000 | len=0x00000004 | ptn=0xc2000105 FILL | adr=0x00010004 | len=0x00000004 | ptn=0x00010020 FILL | adr=0x00010008 | len=0x00000004 | ptn=0x00040004 FILL | adr=0x0001000c | len=0x00000004 | ptn=0x00080004 FILL | adr=0x00010020 | len=0x00000004 | ptn=0xc0010021 ENA | adr=0x00010000 | cnt=0x00000004 | flg=0x0110 ERAS | adr=0x00000004 | cnt=0x00000004 | flg=0x0110 ERAS | adr=0x00000008 | cnt=0x00000004 | flg=0x0110 ERAS | adr=0x0000000c | cnt=0x00000040 | flg=0x0110 LOAD | adr=0x00000004 | len=0x000178c4 | crc=0x16eeb661 | flg=0x0110 LOAD | adr=0x00000008 | len=0x000178c4 | crc=0xddd6d5ce | flg=0x0110 Inject command 'receive-sb-file' Preparing to send 193264 (0x2f2f0) bytes to the target. Successful generic response to command 'receive-sb-file' (1/1) 0% (1/1) 1% (1/1) 1% (1/1) 2% (1/1) 2% (1/1) 3% (1/1) 3% (1/1) 4% (1/1) 4% (1/1) 5% (1/1) 5% (1/1) 6% (1/1) 6% (1/1) 7% (1/1) 7% (1/1) 8% (1/1) 8% (1/1) 9% (1/1) 9% (1/1)10% (1/1)10% (1/1)11% (1/1)12% (1/1)12% (1/1)13% (1/1)13% (1/1)14% (1/1)14% (1/1)15% (1/1)15% (1/1)16% (1/1)16% (1/1)17% (1/1)17% (1/1)18% (1/1)18% (1/1)19% (1/1)19% (1/1)20% (1/1)20% (1/1)21% (1/1)21% (1/1)22% (1/1)23% (1/1)23% (1/1)24% (1/1)24% (1/1)25% (1/1)25% (1/1)26% (1/1)26% (1/1)27% (1/1)27% (1/1)28% (1/1)28% (1/1)29% (1/1)29% (1/1)30% (1/1)30% (1/1)31% (1/1)31% (1/1)32% (1/1)32% (1/1)33% (1/1)34% (1/1)34% (1/1)35% (1/1)35% (1/1)36% (1/1)36% (1/1)37% (1/1)37% (1/1)38% (1/1)38% (1/1)39% (1/1)39% (1/1)40% (1/1)40% (1/1)41% (1/1)41% (1/1)42% (1/1)42% (1/1)43% (1/1)43% (1/1)44% (1/1)45% (1/1)45% (1/1)46% (1/1)46% (1/1)47% (1/1)47% (1/1)48% (1/1)48% (1/1)49% (1/1)49% (1/1)50% (1/1)50% (1/1)51% (1/1)51% (1/1)52% (1/1)52% (1/1)53% (1/1)53% (1/1)54% (1/1)54% (1/1)55% (1/1)56% (1/1)56% (1/1)57% (1/1)57% (1/1)58% (1/1)58% (1/1)59% (1/1)59% (1/1)60% (1/1)60% (1/1)61% (1/1)61% (1/1)62% (1/1)62% (1/1)63% (1/1)63% (1/1)64% (1/1)64% (1/1)65% (1/1)65% (1/1)66% (1/1)67% (1/1)67% (1/1)68% (1/1)68% (1/1)69% (1/1)69% (1/1)70% (1/1)70% (1/1)71% (1/1)71% (1/1)72% (1/1)72% (1/1)73% (1/1)73% (1/1)74% (1/1)74% (1/1)75% (1/1)75% (1/1)76% (1/1)76% (1/1)77% (1/1)78% (1/1)78% (1/1)79% (1/1)79% (1/1)80% (1/1)80% (1/1)81% (1/1)81% (1/1)82% (1/1)82% (1/1)83% (1/1)83% (1/1)84% (1/1)84% (1/1)85% (1/1)85% (1/1)86% (1/1)86% (1/1)87% (1/1)87% (1/1)88% (1/1)89% (1/1)89% (1/1)90% (1/1)90% (1/1)91% (1/1)91% (1/1)92% (1/1)92% (1/1)93% (1/1)93% (1/1)94% (1/1)94% (1/1)95% (1/1)95% (1/1)96% (1/1)96% (1/1)97% (1/1)97% (1/1)98% (1/1)98% (1/1)99% (1/1)100% Completed! Successful generic response to command 'receive-sb-file' Response status = 0 (0x0) Success. Wrote 193264 of 193264 bytes. Finding all necessary files Creating Filesystem STATE_CHECK_FOR_ANY STATE_CHECK_FOR_ELFLOADER STATE_RESET_ELFLOADER OpenHidDevice vid=18d1 pid=9307 ... OpenHidDevice vid=18d1 pid=9307 opened STATE_CHECK_FOR_SDP STATE_LOAD_FLASHLOADER STATE_CHECK_FOR_FLASHLOADER STATE_PROGRAM STATE_LOAD_ELFLOADER STATE_PROGRAM_DATA_FILES OpenHidDevice vid=18d1 pid=9307 ... OpenHidDevice vid=18d1 pid=9307 opened Transferring /coral_micro_camera.html /coral_micro_camera.html 10% /coral_micro_camera.html 20% /coral_micro_camera.html 30% /coral_micro_camera.html 40% /coral_micro_camera.html 50% /coral_micro_camera.html 60% /coral_micro_camera.html 70% /coral_micro_camera.html 80% /coral_micro_camera.html 90% /coral_micro_camera.html done Transferring /default.elf /default.elf 10% /default.elf 20% /default.elf 30% /default.elf 40% /default.elf 50% /default.elf 60% /default.elf 70% /default.elf 80% /default.elf 90% /default.elf done Transferring /usb_ip_address /usb_ip_address done STATE_RESET_TO_FLASH OpenHidDevice vid=18d1 pid=9307 ... OpenHidDevice vid=18d1 pid=9307 opened Flashing to flash storage complete, the device is restarting to execute your application. ```

hjonnala commented 1 year ago

please refer to this comment: https://github.com/google-coral/coralmicro-out-of-tree-sample/issues/6#issuecomment-1515293630

arthurkafer commented 1 year ago

Yeah, Show Sketch Folder puts me in the correct folder, and the file is there in the /data folder.

I found the /data folder is being "replaced"/"overwritten" by the /data folder inside the Arduino core libraries -> C:\Users\arthu\AppData\Local\Arduino15\packages\coral\hardware\coral_micro\1.0.1\variants\coral_micro_wifi\data

I tried putting the html file inside this folder and it was flashed correctly, but that's not supposed to work that way, I guess

FStefanni commented 1 year ago

Hi,

I confirm this bug. I have done a little analysis, here what I have found:

In file boards.txt, the key coral_micro_wifi.upload.pattern passes two times the option --data_dir:
- The first time, as for the micro (i.e. without wifi), to pass the tensorflow data model
- The second time, to pass the data dir inside the variants folder, I suppose to load some specific firmware to enable the WiFi
- So, in the end, it seems that passing two times the same option, does not work, since the second occurrence overwrites the first occurrence
I have run the flashtool program, with --help to check this fact, but the option --data_dir is not listed. i suppose this is a customization of a tihrd-party tool, and who have done the customization has forgotten to describe this option in the help
To confirm the bug I have done two checks:
- If I run the upload as is, then I see in the log that the files inside the data dir of variants is actually loaded, but not the model inside the project dir
- If I remove the second occurrence of --data_dir, I see the model is loaded, the files inside data of variants are not loaded, and, in this case, without using the wifi, the tensorflow model works fine

I note the tags of this issue, and I suggest to change them, since it does not work regardless of the platform (I am using MacOS and Linux, the original reported of this issue Windows, and it is normal that this cannot work regardless of the platform).

Regards

VisualMicro commented 3 weeks ago

It seems with a few modifications to the boards.txt and platform.txt this can be remedied, by implementing the below approach: 1) Copy all firmware files to a single "data" folder in the build folder using an additional prebuild hook. 2) Remove the secondary "--data_dir" from the flashtool command for the coral_micro_wifi variant. 3) Add quotes to the windows prebuild.2 recipe to ensure spaces in sketch paths are supported

We have documented this on this page, with the changes needed also below.

Boards.txt: coral_micro_wifi.upload.pattern={cmd.path} --elfloader_path {elfloader.file} --flashloader_path {flashloader.file} --build_dir . --elf_path {build.path}/{build.project_name}.elf --toolchain {build.compiler_path} --arduino --data_dir={build.path}/data {upload.extra_flags}

Platform.txt: recipe.hooks.sketch.prebuild.2.pattern.windows=cmd.exe /c if exist "{build.source.path}\data" ( xcopy /Y /E "{build.source.path}\data*" "{build.path}\data\" ) recipe.hooks.sketch.prebuild.3.pattern.windows=cmd.exe /c if exist "{data.dir}" ( xcopy /Y /E "{data.dir}*" "{build.path}\data\" )

We will create a Pull Request for this if there are no objections.

google-coral / coralmicro

Data folder is not flashed using Coral Dev Micro + Wireless Add-on board #47

Description