Rebuild fails if `--build` option specified

[SeppoTakalo]

If mbed build command have been issued earlier with --build parameter, rebuilding of application fails.

Various errors happens. Sometimes it tries to build with some random target, ignoring .mbed file. Sometimes it fails to resolve options specified in mbed_app.json

Steps to reproduce

Build failure

$ git clone https://github.com/ARMmbed/example-mbedos-blinky.git
$ cd example-mbedos-blinky
$ mbed deploy --procotol ssh
$ mbed target K64F
$ mbed toolchain GCC_ARM
$ mbed compile --build out/K64F
...
Image: out/K64F/mbed-os-example-blinky.bin
$ mbed compile --build out/K64F
...
/Users/septak01/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3/../../../../arm-none-eabi/bin/ld: section .flash_config loaded at [0000000000000400,000000000000041f] overlaps section .interrupts loaded at [0000000000000000,00000000000007ff]
/Users/septak01/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3/../../../../arm-none-eabi/bin/ld: section .text loaded at [0000000000000410,000000000000c3a7] overlaps section .flash_config loaded at [0000000000000400,000000000000041f]
/Users/septak01/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3/../../../../arm-none-eabi/bin/ld: region `m_interrupts' overflowed by 1024 bytes
/Users/septak01/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3/../../../../arm-none-eabi/bin/ld: region `m_flash_config' overflowed by 16 bytes
collect2: error: ld returned 1 exit status

[mbed] ERROR: "python" returned error code 1.
[mbed] ERROR: Command "python -u /Users/septak01/src/mbed-os-example-blinky/mbed-os/tools/make.py -t GCC_ARM -m K64F --source . --build out/K64F" in "/Users/septak01/src/mbed-os-example-blinky"

$ ls BUILD/
K64F              UBLOX_EVK_ODIN_W2
$ cat .mbed
ROOT=.
TARGET=K64F
TOOLCHAIN=GCC_ARM

Build fails, and looks like it tried to build Ublox, even that I asked K64F

Configuration error

$ git clone https://github.com/ARMmbed/mbed-os-example-mesh-minimal.git
$ cd mbed-os-example-mesh-minimal
$ mbed deploy --procotol ssh
$ mbed target K64F
$ mbed toolchain GCC_ARM
$ mbed compile --build out/K64F
...
Image: out/K64F/mbed-os-example-mesh-minimal.bin
$ mbed compile --build out/K64F
...
Scan: env
[ERROR] Attempt to override undefined parameter 'mbed-mesh-api.6lowpan-nd-channel-page' in 'application[*]'
[mbed] ERROR: "python" returned error code 1.
[mbed] ERROR: Command "python -u /Users/septak01/src/mbed-os-example-mesh-minimal/mbed-os/tools/make.py -t GCC_ARM -m K64F --source . --build out/K64F" in "/Users/septak01/src/mbed-os-example-mesh-minimal"

Possible problem

Looks like mbed compile creates a .mbedignore file into BUILD folder, but does not create it into a folder specified by --build option

$ ls -a BUILD
.           ..          .mbedignore
$ ls -a out/
.    ..   K64F

Therefore build travels into these folders, causing problems.

[SeppoTakalo]

@screamerbg Please check.

[bridadan]

I did a little investigation and this is definitely an issue with the mbed OS tools not mbed CLI. Would you mind opening this on ARMmbed/mbed-os @SeppoTakalo?

[SeppoTakalo]

Cloned https://github.com/ARMmbed/mbed-os/issues/3787

[screamerbg]

@SeppoTakalo, thanks for the report.

When you specify --build output directory, then mbed CLI passes it as it is to the mbed OS tools. If --build is not specified then mbed CLI defaults to <target>/<toolchain> pattern.

In the first scenario (--build specified`), it's up to the user to define a folder that doesn't conflict with the target/toolchain configuration. Since the output folder is the same, but these are different targets, essentially output files get badly mixed. This could be also happen if you change the toolchain and use the same output dir.

In summary and as mentioned above, it is expected that when you use --build then you have to handle the changes in your target and toolchain. E.g. use --clean flag

@SeppoTakalo @bridadan

[screamerbg]

Perhaps this should be indicated better in the documentation.

[bridadan]

@screamerbg From @SeppoTakalo's description it doesn't sound like he's changing the target or toolchain, so I don't think that's the exact problem.

[screamerbg]

cc @theotherjimmy

[theotherjimmy]

I was thinking of adding a .mbedignore file to the root of the build folder (containing *). @screamerbg would you rather see this in mbed-cli or mbed-os tools? maybe not at all? I'm thinking mbed-cli.

[theotherjimmy]

https://github.com/ARMmbed/mbed-os/pull/3852

[screamerbg]

This is now merged and should be fixed. Can you confirm with the latest mbed OS @SeppoTakalo?

[SeppoTakalo]

The issue, as reported, is now fixed. However, only partially. One issue remains: If you specify another output directory, it will start to fail again.

$ mbed compile
Building project mbed-os-example-mesh-minimal (K64F, GCC_ARM)
Scan: .
Scan: FEATURE_BLE
Scan: FEATURE_COMMON_PAL
Scan: FEATURE_LWIP
Scan: FEATURE_UVISOR
Scan: FEATURE_ETHERNET_HOST
Scan: FEATURE_LOWPAN_BORDER_ROUTER
Scan: FEATURE_LOWPAN_HOST
Scan: FEATURE_LOWPAN_ROUTER
Scan: FEATURE_NANOSTACK
Scan: FEATURE_NANOSTACK_FULL
Scan: FEATURE_THREAD_BORDER_ROUTER
Scan: FEATURE_THREAD_END_DEVICE
Scan: FEATURE_THREAD_ROUTER
Scan: FEATURE_STORAGE
Warning: Multiple linker scripts detected: ./mbed-os/targets/TARGET_Freescale/TARGET_MCUXpresso_MCUS/TARGET_MCU_K64F/device/TOOLCHAIN_GCC_ARM/MK64FN1M0xxx12.ld -> ./out/K64F/.link_script.ld
Scan: FEATURE_COMMON_PAL
Scan: FEATURE_LWIP
Scan: FEATURE_NANOSTACK
Scan: FEATURE_STORAGE
Scan: env
Scan: mbed
[ERROR] Attempt to override undefined parameter 'mbed-mesh-api.6lowpan-nd-panid-filter' in 'application[*]'
[mbed] ERROR: "python" returned error code 1.
[mbed] ERROR: Command "python -u /Users/septak01/src/mbed-os-example-mesh-minimal/mbed-os/tools/make.py -t GCC_ARM -m K64F --source . --build ./BUILD/K64F/GCC_ARM" in "/Users/septak01/src/mbed-os-example-mesh-minimal"
---

I would like to see all output directories treaded same way as BUILD so that what ever I specify, it will place .mbedignore there, so that next build with other output directory wont scan it.

Intention is that I can build by application several times, using different mbed_app.json files or compilers and keep all of the build directories in the tree for gathering the results later. This would be used in CI.

Should I open a new bug or keep this open?

[screamerbg]

@SeppoTakalo Placing an .mbedignore file in the output directory will cause mbed CLI to ignore the directory if included in other projects. Here's a very valid (and common) usecase:

1. Compile /sources/uvisor-source/ as a static library and output it to /sources/uvisor-lib, e.g. mbed compile --source /sources/uvisor-sources --build /sources/uvisor-lib --library
2. Compile /sources/uvisor-example as an application and add additional source /sources/uvisor-lib, e.g. mbed compile --source /sources/uvisor-example --source /sources/uvisor-lib

If .mbedignore is used in /sources/uvisor-lib then it will be ignored and not compiled/linked against /sources/uvisor-example

[SeppoTakalo]

@screamerbg OK. If that is the usecase, this should be documented. Sounds like this is a issue that should not be fixed but behaviour should be documented instead.

Current documentation says only: --build <BUILD> to select the build directory. Default: BUILD/ inside your program root. so there are no hints that care should be taken when combining multiple build directories.

I agree the view that only default BUILD/ should be added to .mbedignore but that should be mentioned somewhere. Maybe we should either use BUILD/ in CI or add our own .mbedignore when we do multi-compiler builds.

[screamerbg]

Bump @theotherjimmy @SeppoTakalo what's the latest status on this?

[SeppoTakalo]

I have not seen any documentation added regarding this one, so the status is exactly same as earlier.

[theotherjimmy]

@SeppoTakalo Where would you expect to see this behavior documented?

[screamerbg]

The build directory is now automatically ignored https://github.com/ARMmbed/mbed-cli/blob/master/mbed/mbed.py#L2489. Please reopen if the issue still exists.