oxidecomputer / hubris

A lightweight, memory-protected, message-passing kernel for deeply embedded systems.
Mozilla Public License 2.0
3.02k stars 175 forks source link

GPIO IRQ is causing task rebuilds #1828

Closed mkeeter closed 4 months ago

mkeeter commented 4 months ago

I recently noticed that a no-op rebuild (cargo xtask dist ...) is doing a bunch of work.

The culprit appears to be anything with a dependency on drv-stm32xx-sys-api:

building crate task-jefe
    Finished `release` profile [optimized + debuginfo] target(s) in 0.19s
target/thumbv7em-none-eabihf/release/task-jefe -> target/sidecar-b-dev/dist/jefe.elf
building crate drv-stm32xx-sys
   Compiling drv-stm32xx-sys-api v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32xx-sys-api)
   Compiling drv-stm32xx-sys v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32xx-sys)
    Finished `release` profile [optimized + debuginfo] target(s) in 1.28s
target/thumbv7em-none-eabihf/release/drv-stm32xx-sys -> target/sidecar-b-dev/dist/sys.elf
building crate stm32h7-update-server
    Finished `release` profile [optimized + debuginfo] target(s) in 0.20s
target/thumbv7em-none-eabihf/release/stm32h7-update-server -> target/sidecar-b-dev/dist/update_server.elf
building crate drv-auxflash-server
   Compiling drv-stm32xx-sys-api v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32xx-sys-api)
   Compiling drv-auxflash-server v0.1.0 (/Users/mjk/oxide/hubris/drv/auxflash-server)
    Finished `release` profile [optimized + debuginfo] target(s) in 1.39s
target/thumbv7em-none-eabihf/release/drv-auxflash-server -> target/sidecar-b-dev/dist/auxflash.elf
building crate task-net
   Compiling drv-stm32xx-sys-api v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32xx-sys-api)
   Compiling drv-stm32h7-spi-server-core v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32h7-spi-server-core)
   Compiling task-net v0.1.0 (/Users/mjk/oxide/hubris/task/net)
    Finished `release` profile [optimized + debuginfo] target(s) in 3.28s
target/thumbv7em-none-eabihf/release/task-net -> target/sidecar-b-dev/dist/net.elf
building crate task-control-plane-agent
    Finished `release` profile [optimized + debuginfo] target(s) in 0.22s
target/thumbv7em-none-eabihf/release/task-control-plane-agent -> target/sidecar-b-dev/dist/control_plane_agent.elf
building crate drv-stm32h7-sprot-server
   Compiling drv-stm32xx-sys-api v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32xx-sys-api)
   Compiling drv-stm32h7-spi-server-core v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32h7-spi-server-core)
   Compiling drv-stm32h7-sprot-server v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32h7-sprot-server)
    Finished `release` profile [optimized + debuginfo] target(s) in 3.35s
target/thumbv7em-none-eabihf/release/drv-stm32h7-sprot-server -> target/sidecar-b-dev/dist/sprot.elf
building crate task-udpecho
    Finished `release` profile [optimized + debuginfo] target(s) in 0.19s
target/thumbv7em-none-eabihf/release/task-udpecho -> target/sidecar-b-dev/dist/udpecho.elf
building crate task-udpbroadcast
    Finished `release` profile [optimized + debuginfo] target(s) in 0.20s
target/thumbv7em-none-eabihf/release/task-udpbroadcast -> target/sidecar-b-dev/dist/udpbroadcast.elf
building crate task-monorail-server
   Compiling drv-stm32xx-sys-api v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32xx-sys-api)
   Compiling drv-stm32h7-spi-server-core v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32h7-spi-server-core)
   Compiling task-monorail-server v0.1.0 (/Users/mjk/oxide/hubris/task/monorail-server)
    Finished `release` profile [optimized + debuginfo] target(s) in 4.14s
target/thumbv7em-none-eabihf/release/task-monorail-server -> target/sidecar-b-dev/dist/monorail.elf
building crate drv-stm32xx-i2c-server
   Compiling drv-stm32xx-sys-api v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32xx-sys-api)
   Compiling drv-stm32xx-i2c v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32xx-i2c)
   Compiling drv-stm32xx-i2c-server v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32xx-i2c-server)
    Finished `release` profile [optimized + debuginfo] target(s) in 1.56s
target/thumbv7em-none-eabihf/release/drv-stm32xx-i2c-server -> target/sidecar-b-dev/dist/i2c_driver.elf
building crate task-hiffy
   Compiling drv-stm32xx-sys-api v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32xx-sys-api)
   Compiling drv-stm32xx-i2c v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32xx-i2c)
   Compiling task-hiffy v0.1.0 (/Users/mjk/oxide/hubris/task/hiffy)
    Finished `release` profile [optimized + debuginfo] target(s) in 1.43s
target/thumbv7em-none-eabihf/release/task-hiffy -> target/sidecar-b-dev/dist/hiffy.elf
building crate task-sensor
    Finished `release` profile [optimized + debuginfo] target(s) in 0.19s
target/thumbv7em-none-eabihf/release/task-sensor -> target/sidecar-b-dev/dist/sensor.elf
building crate drv-fpga-server
   Compiling drv-stm32xx-sys-api v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32xx-sys-api)
   Compiling drv-stm32h7-spi-server-core v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32h7-spi-server-core)
   Compiling drv-fpga-devices v0.1.0 (/Users/mjk/oxide/hubris/drv/fpga-devices)
   Compiling drv-fpga-server v0.1.0 (/Users/mjk/oxide/hubris/drv/fpga-server)
    Finished `release` profile [optimized + debuginfo] target(s) in 2.13s
target/thumbv7em-none-eabihf/release/drv-fpga-server -> target/sidecar-b-dev/dist/ecp5_mainboard.elf
building crate drv-fpga-server
   Compiling drv-stm32xx-sys-api v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32xx-sys-api)
   Compiling drv-fpga-devices v0.1.0 (/Users/mjk/oxide/hubris/drv/fpga-devices)
   Compiling drv-stm32h7-spi-server-core v0.1.0 (/Users/mjk/oxide/hubris/drv/stm32h7-spi-server-core)
   Compiling drv-fpga-server v0.1.0 (/Users/mjk/oxide/hubris/drv/fpga-server)
    Finished `release` profile [optimized + debuginfo] target(s) in 2.16s
target/thumbv7em-none-eabihf/release/drv-fpga-server -> target/sidecar-b-dev/dist/ecp5_front_io.elf
building crate drv-transceivers-server
    Finished `release` profile [optimized + debuginfo] target(s) in 0.21s
target/thumbv7em-none-eabihf/release/drv-transceivers-server -> target/sidecar-b-dev/dist/transceivers.elf
building crate task-packrat
    Finished `release` profile [optimized + debuginfo] target(s) in 0.22s
target/thumbv7em-none-eabihf/release/task-packrat -> target/sidecar-b-dev/dist/packrat.elf
building crate drv-sidecar-seq-server
    Finished `release` profile [optimized + debuginfo] target(s) in 0.20s
target/thumbv7em-none-eabihf/release/drv-sidecar-seq-server -> target/sidecar-b-dev/dist/sequencer.elf
building crate task-thermal
    Finished `release` profile [optimized + debuginfo] target(s) in 0.22s
target/thumbv7em-none-eabihf/release/task-thermal -> target/sidecar-b-dev/dist/thermal.elf
building crate task-power
    Finished `release` profile [optimized + debuginfo] target(s) in 0.19s
target/thumbv7em-none-eabihf/release/task-power -> target/sidecar-b-dev/dist/power.elf
building crate task-validate
    Finished `release` profile [optimized + debuginfo] target(s) in 0.20s
target/thumbv7em-none-eabihf/release/task-validate -> target/sidecar-b-dev/dist/validate.elf
building crate drv-ignition-server
    Finished `release` profile [optimized + debuginfo] target(s) in 0.19s
target/thumbv7em-none-eabihf/release/drv-ignition-server -> target/sidecar-b-dev/dist/ignition.elf
building crate task-vpd
    Finished `release` profile [optimized + debuginfo] target(s) in 0.19s
target/thumbv7em-none-eabihf/release/task-vpd -> target/sidecar-b-dev/dist/vpd.elf
building crate task-dump-agent
    Finished `release` profile [optimized + debuginfo] target(s) in 0.20s
target/thumbv7em-none-eabihf/release/task-dump-agent -> target/sidecar-b-dev/dist/dump_agent.elf
building crate task-idle
    Finished `release` profile [optimized + debuginfo] target(s) in 0.17s
target/thumbv7em-none-eabihf/release/task-idle -> target/sidecar-b-dev/dist/idle.elf
building crate task-udprpc
    Finished `release` profile [optimized + debuginfo] target(s) in 0.22s
target/thumbv7em-none-eabihf/release/task-udprpc -> target/sidecar-b-dev/dist/udprpc.elf

This takes ~20 seconds; if all of the tasks were 0.2s (a typical no-op rebuild speed), it would be closer to 5.5 seconds.

I believe this is because the stm32xx-sys build now depends on task_name, so it can't be cached between tasks.

Possible fixes:

hawkw commented 4 months ago

Ah, yup, that's annoying, sorry about that!

I think both of your proposed solutions make sense and should fix the problem. I went with the approach of just generating the pins for an individual task when I wrote this code initially because it seemed conceptually cleaner, but I think changing this to just always generate all the pins for every task would be simpler and avoid triggering rebuilds. On the other hand, we would probably want to generate submodules for each individual task if we went with that approach, because otherwise, we could potentially have namespace collisions (although in practice I don't see any cases where pin names might collide...)

The separate crate approach feels nicer IMO, since it means that every task doesn't see every pin, and it lets us avoid doing the codegen at all when building tasks that don't depend on GPIO IRQs. I'll look into that, and if it turns out to be annoying, I might just do the "always generate all pins" approach instead.

hawkw commented 4 months ago

Ah, I've figured out why I was putting the generated code in drv-stm32xx-sys-api instead of in the tasks that consume it: the generated code depends on drv-stm32xx-gpio-common, which most of the downstream tasks don't have direct dependencies on:

building crate task-nucleo-user-button
   Compiling task-nucleo-user-button v0.1.0 (/home/eliza/Code/oxide/hubris/task/nucleo-user-button)
cargo:rerun-if-env-changed=HUBRIS_TASK_CONFIG
error[E0432]: unresolved import `drv_stm32xx_gpio_common`
 --> /hubris/target/thumbv7em-none-eabihf/release/build/task-nucleo-user-button-14915eea1683731a/out/gpio_irq_pins.rs:1:29
  |
1 | pub mod gpio_irq_pins { use drv_stm32xx_gpio_common :: { PinSet , Port } ; pub const BUTTON : PinSet = Port :: C . pin (13usize) ; }
  |                             ^^^^^^^^^^^^^^^^^^^^^^^ use of undeclared crate or module `drv_stm32xx_gpio_common`

For more information about this error, try `rustc --explain E0432`.
error: could not compile `task-nucleo-user-button` (bin "task-nucleo-user-button") due to 1 previous error
Error: failed to build user_button

Caused by:
    command failed, see output for details

That's annoying but certainly not unsolveable.

hawkw commented 4 months ago

I can just change the codegen to use the stm32xx-sys-api reexports though.