search

maxgerhardt / pio-esp32-esp8266-filesystem-downloader

Script for expanding PlatformIO's ability to download LittleFS and SPIFFS filesystems from ESP32 and ESP8266 chips and their contents
29 stars 9 forks source link

"downloaded_fs_0x300000_0xfa000.bin": no such file or directory #3

Closed leohxj closed 1 year ago

leohxj commented 2 years ago

I’ve tried it on my own project by copy download_fs.py to my project and adding extra_scripts = download_fs.py as described in https://github.com/maxgerhardt/pio-esp32-esp8266-filesystem-downloader#using-in-a-different-project.

I run custom -> Download Filesystem, but got below error:

Processing nodemcuv2 (platform: espressif8266; board: nodemcuv2; framework: arduino)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif8266/nodemcuv2.html
PLATFORM: Espressif 8266 (3.2.0) > NodeMCU 1.0 (ESP-12E Module)
HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash
PACKAGES: 
 - framework-arduinoespressif8266 3.30002.0 (3.0.2) 
 - tool-esptool 1.413.0 (4.13) 
 - tool-esptoolpy 1.30000.201119 (3.0.0) 
 - toolchain-xtensa 2.100300.210717 (10.3.0)
Converting main.ino
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 35 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <ESP8266WebServer> 1.0
|   |-- <ESP8266WiFi> 1.0
|-- <ESP8266WiFi> 1.0
Building in release mode
Auto-detected: /dev/cu.usbserial-0001
command_download_fs(["downloadfs"], [])
Entrypoint
Retrieving filesystem info for ESP8266.
FS_START: 0x300000
FS_END: 0x3fa000
FS_PAGE: 0x100
FS_BLOCK: 0x2000
Recognized SPIFFS filesystem.
Parsed FS info: FS type FSType.SPIFFS Start 0x300000 Len 1024000 Page size 256 Block size 8192 Tool: /Users/leohxj/.platformio/packages/tool-mkspiffs/mkspiffs
Executing flash download command.
"/Users/leohxj/.platformio/penv/bin/python" "/Users/leohxj/.platformio/packages/tool-esptoolpy/esptool.py" --chip esp8266 --port "/dev/cu.usbserial-0001" --baud 115200 --before default_reset --after hard_reset read_flash 0x300000 0xfa000 "/Users/leohxj/Documents/PlatformIO/Projects/access-point/downloaded_fs_0x300000_0xfa000.bin"
scons: *** [downloadfs] "/Users/leohxj/.platformio/penv/bin/python" "/Users/leohxj/.platformio/packages/tool-esptoolpy/esptool.py" --chip esp8266 --port "/dev/cu.usbserial-0001" --baud 115200 --before default_reset --after hard_reset read_flash 0x300000 0xfa000 "/Users/leohxj/Documents/PlatformIO/Projects/access-point/downloaded_fs_0x300000_0xfa000.bin": No such file or directory
==================================================================================== [FAILED] Took 0.81 seconds ====================================================================================
The terminal process "platformio 'run', '--target', 'downloadfs', '--environment', 'nodemcuv2'" terminated with exit code: 1.

Where is downloaded_fs_0x300000_0xfa000.bin file, or how to build or generate it??

maxgerhardt commented 2 years ago

The script is supposed to trigger the generation of that file by starting esptool.py, but in the output you can see it failed.

scons: *** [downloadfs] "/Users/leohxj/.platformio/penv/bin/python" "/Users/leohxj/.platformio/packages/tool-esptoolpy/esptool.py" --chip esp8266 --port "/dev/cu.usbserial-0001" --baud 115200 --before default_reset --after hard_reset read_flash 0x300000 0xfa000 "/Users/leohxj/Documents/PlatformIO/Projects/access-point/downloaded_fs_0x300000_0xfa000.bin": No such file or directory

I think I remember a bug report saying that quoting the executable (python) does not work like that on a Mac (but does on my Windows machine).

Can you change this line in the method of the SPIFFSInfo class

https://github.com/maxgerhardt/pio-esp32-esp8266-filesystem-downloader/blob/1eba44dbc32027bfe3b26e246829fd9b9764dc05/download_fs.py#L73-L74

to

    def get_extract_cmd(self, input_file, output_dir):
        return f'{self.tool} -b {self.block_size} -p {self.page_size} --unpack "{output_dir}" "{input_file}"'

and retry?

leohxj commented 2 years ago

yes, I am on MacOS. I change the code below, but it also gets errors like before.

then I print log in get_extract_cmd, but it does not execute.

maxgerhardt commented 2 years ago

Can you manually open a CLI and execute the command? 

/Users/leohxj/.platformio/penv/bin/python "/Users/leohxj/.platformio/packages/tool-esptoolpy/esptool.py" --chip esp8266 --port "/dev/cu.usbserial-0001" --baud 115200 --before default_reset --after hard_reset read_flash 0x300000 0xfa000 "/Users/leohxj/Documents/PlatformIO/Projects/access-point/downloaded_fs_0x300000_0xfa000.bin"

what does it output?

leohxj commented 2 years ago

@maxgerhardt It works! and the output is:

 /Users/leohxj/.platformio/penv/bin/python "/Users/leohxj/.platformio/packages/tool-esptoolpy/esptool.py" --chip esp8266 --port "/dev/cu.usbserial-0001" --baud 115200 --before default_reset --after hard_reset read_flash 0x300000 0xfa000 "/Users/leohxj/Documents/PlatformIO/Projects/access-point/downloaded_fs_0x300000_0xfa000.bin"
esptool.py v3.1
Serial port /dev/cu.usbserial-0001
Connecting....
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: 98:cd:ac:26:2a:2f
Uploading stub...
Running stub...
Stub running...
1024000 (100 %)
1024000 (100 %)
Read 1024000 bytes at 0x300000 in 91.5 seconds (89.5 kbit/s)...
Hard resetting via RTS pin...
maxgerhardt commented 2 years ago

Weird that it works when the command is executed in the commandline, if it's the same command it should work.

What is the full output now of the "Download Filesystem" task?

nighi commented 2 years ago

I have the same issue under linux (Manjaro). I could execute the cmds directly via shell, but I think the problem is already the download cmd, not only the extraction cmd.

Removing the quotes around the python-path doesn't help, neither in the mentioned get_extract_cmd method nor in download_fs. But with changing the call to subprocess.call(cmd, shell=True) fixed the issue with download and extraction (lines 297 & 274). Executing via shell is considered as a security issue because it could enable shell injection, but I see no problem in this case. It could be a problem for Windows users, don't know. Maybe there is a better option escaping the cmd correctly or use the arguments of subprocess.call ?

With the script working there is another issue: The extracted files are only a part of the file system. But perhaps I should open a new Issue because they don't seem related.

maxgerhardt commented 2 years ago

I'll double-check this on Linux today to see where / why it fails, but thanks for the hint on shell = True. Sadly I won't have a Mac to test on, but maybe it equally resolves the issue with that.

maxgerhardt commented 1 year ago

Fixed per PR above