platformio / platform-espressif32

Espressif 32: development platform for PlatformIO
https://registry.platformio.org/platforms/platformio/espressif32
Apache License 2.0
891 stars 600 forks source link

AddPreAction hook executed after the build (buildfs) #1158

Open bmedici opened 1 year ago

bmedici commented 1 year ago

The execution order is: 1) Building FS image from 'data' directory to .pio/build/ser/spiffs.bin 2) before_buildfs(["buildfs"], [".pio/build/ser/spiffs.bin"]) 3) after_buildfs(["buildfs"], [".pio/build/ser/spiffs.bin"])

Where I would have expected before_buildfs first, then "Building FS image", and finally after_buildfs.

Some context:

platformio.ini

[env:esp32]
    board = esp32dev
    framework = arduino
    board_build.partitions = shared/partitions.csv
    extra_scripts =
    shared/extra_script.py

shared/extra_script.py

import os
Import("env")

print(">>> Current CLI targets", COMMAND_LINE_TARGETS)
print(">>> Current Build targets", BUILD_TARGETS)
# print(env.Dump())
# host = env.GetProjectOption("custom_ping_host")

def before_buildfs(source, target, env):
    print("\n>>>  before_buildfs: build-www")
    env.Execute("ruby shared/build-www.rb")

    print("\n>>> before_buildfs: copy assets")
    env.Execute("rsync --delete -av shared/assets/ data/assets/")

    print("\n>>> before_buildfs: gzip assets")
    env.Execute("find data/ -type f \( -name \*.css -o -name \*.js -o -name \*.html \) -exec gzip -f \"{}\" \;")

    print("\n>>> before_buildfs: total size")
    env.Execute("du -sh data")
    env.Execute("du -sh data/*")

env.AddPreAction("buildfs", before_buildfs)

def after_buildfs(source, target, env):
    print("\n>>> after_buildfs: unzip assets")
    # env.Execute("find data/ -type f -name *.gz -exec gunzip -f \"{}\" \;")

env.AddPostAction("buildfs", after_buildfs)

execution output for platformio run --target buildfs --environment ser

Processing ser (board: esp32dev; framework: arduino; platform: espressif32  @6.3.0)
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Library Manager: Installing jfturcot/SimpleTimer @ *
Library Manager: Warning! Could not install `{'owner': 'jfturcot', 'name': 'SimpleTimer', 'version': '*'}` dependency for the`vanbox-lib` package
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32dev.html
PLATFORM: Espressif 32 (6.3.0) > Espressif ESP32 Dev Module
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES:
 - framework-arduinoespressif32 @ 3.20009.0 (2.0.9)
 - tool-esptoolpy @ 1.40501.0 (4.5.1)
 - tool-mkspiffs @ 2.230.0 (2.30)
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 57 compatible libraries
Scanning dependencies...
Dependency Graph
      [...]
Building in release mode
>>> Current CLI targets ['buildfs']
>>> Current Build targets ['buildfs']
Building FS image from 'data' directory to .pio/build/ser/spiffs.bin
/setup.html.gz
      [...]

before_buildfs(["buildfs"], [".pio/build/ser/spiffs.bin"])

>>>  before_buildfs: build-www
ruby shared/build-www.rb
> build_page  [index]
  layout  [2023-07-26 19:39:41 +0200]
  page    [2023-07-24 21:34:53 +0200]
  output  [0]
> build_page  [location]
  layout  [2023-07-26 19:39:41 +0200]
  page    [2023-07-22 01:19:54 +0200]
  output  [0]
> build_page  [setup]
  layout  [2023-07-26 19:39:41 +0200]
  page    [2023-07-21 21:35:06 +0200]
  output  [0]

>>> before_buildfs: copy assets
rsync --delete -av shared/assets/ data/assets/
building file list ... done
deleting css/style.css.gz
deleting css/jquery-ui.min.css.gz
deleting css/helpers.css.gz
deleting css/bootstrap.min.css.gz
css/
css/bootstrap.min.css
css/helpers.css
css/jquery-ui.min.css
css/style.css

sent 174360 bytes  received 114 bytes  348948.00 bytes/sec
total size is 189322  speedup is 1.09

>>> before_buildfs: gzip assets
find data/ -type f \( -name \*.css -o -name \*.js -o -name \*.html \) -exec gzip -f "{}" \;

>>> before_buildfs: total size
du -sh data
 88K    data
du -sh data/*
 56K    data/assets
 20K    data/img
4.0K    data/index.html.gz
  0B    data/js
4.0K    data/location.html.gz
4.0K    data/setup.html.gz
after_buildfs(["buildfs"], [".pio/build/ser/spiffs.bin"])

>>> after_buildfs: unzip assets
=================================================================================================================== [SUCCESS] Took 3.50 seconds ===================================================================================================================

Environment    Status    Duration
-------------  --------  ------------
ser            SUCCESS   00:00:03.495
stale[bot] commented 9 months ago

This issue has been automatically marked as stale because it has not had recent activity. Please provide more details or it will be closed if no further activity occurs. Thank you for your contributions.

ArnoCtr commented 8 months ago

Same issue.

bmedici commented 8 months ago

Any update on this (looks-like-a-) bug ?

valeros commented 6 months ago

I'd recommend using the path to the file system binary instead of the buildfs target (see an example):

env.AddPreAction("$BUILD_DIR/${ESP32_FS_IMAGE_NAME}.bin", before_buildfs)
env.AddPostAction("$BUILD_DIR/${ESP32_FS_IMAGE_NAME}.bin", after_buildfs)