Build: simplify use as a library

[elagil]

Hello!

I am in the process of using blackmagic firmware as a library in a project that uses an unsupported platform and software framework (an RTOS). For that, I would like to make the build system of blackmagic a bit more modular, similar to what is done in lwip. More specifically, https://github.com/lwip-tcpip/lwip/blob/master/src/Filelists.mk .

The idea is to create separate makefiles/CMakeLists for each of the components (e.g. core and targets), and also make it an option to build (or not build) with the opencm library. For example, I have no use for the opencm library, because my RTOS ships its own HAL.

Would you accept such a pull request?

[elagil]

Seeing https://github.com/blackmagic-debug/blackmagic/pull/1528, that is probably a good place to include this kind of feature. However, adding the Makefile support first, then rebasing, is also an idea.

[dragonmux]

Hi elagil, now the v2.0 cycle has started we are actually about to embark on a complete build system replacement - see #1528 and also a KConfig-based Makefile system to replace the current Makefiles. If you need to use the project as a library, we strongly recommend you use Meson and the Meson build system.

BMD uses libopencm3 extensively, so your RTOS's HAL is likely not usable in this situation, especially if you're on a platform libopencm3 supports. Failing that you would have to do something like Farpatch does to glue w/e HAL you have to the code base. This also provides a CMake build system to copy but you will have to maintain this yourself. CMake can talk to Meson build systems though via its foreign build system mechanism, and we would peruse this route rather than building your own CMakeLists.txt for the project - less to maintain.

Edit: To make it a little more clear, we are not going to add a CMake build system to the project. The target path at this juncture is a kernel-style build system with KConfig/Make, paired with a Meson build system w/ full subproject support as a modern build system.

We personally use Meson for all our projects, it solves all the cross-platform headaches properly and without lots of work from the project to do, it supports Canadian-cross and other complex build setups, and will allow complex library configurations without much effort while keeping the build system easy to maintain and low barrier to entry for contributors.

[elagil]

Hi @dragonmux, thanks for the answer!

To give some context, I am creating a BMP variant with Ethernet. It is based on the STM32F407, and I am currently waiting for the hardware to arrive: https://github.com/elagil/net-bmp. Thus, I need some MAC/PHY support in my HAL, and I would appreciate an RTOS in general. That is why I don't see a simple way of just adding my board as a platform. Rather, I would like to interface to something like an API in the blackmagic "library", where my core application provides the data exchange (ethernet, USB) with a host.

Concerning the libopencm3 dependency: Is there so much of it? The main thing I see are the gdb_if and rtt_if modules, and of course the platform definitions (especially common).

[dragonmux]

Your use-case makes it a good idea to read how Farpatch does what it does, as that uses the Espressif IDF as a HAL and uses an RTOS very carefully to function. If you do really wish to ignore libopencm3 then you will have to ignore src/platforms/common entirely and write replacements for everything there, and likewise the GDB and RTT interface implementations as you noted. This means supplying your own JTAG and SWD bitbanging routines.

The only thing we can warn if you do embark down that path rather than using your RTOS choices' HAL only for the Ethernet and non-BMD-core parts.. is that they are very sensitive to timing. It is quite possible to use a blend of libopencm3 and your RTOS' HAL for what it's worth - locm3 for the common platform components, and then the RTOS's HAL for things you're adding around BMD. It would be what we would do as trying to get the bitbanging for JTAG and SWD right is quite difficult and time-intensive.

Note that locm3 does not internally depend on the state it sets up not changing or otherwise have control structures, it is intended to be a very thin HAL which is very tolerant to being reached around (something BMD actually takes advantage of!)

[ALTracer]

Thus, I need some MAC/PHY support in my HAL, and I would appreciate an RTOS in general.

I'll chime in and point out that you don't strictly need a RTOS (ChibiOS) to network. I ran some lwIP demos in Standalone mode over 10Base-T (PHY negotiation issue) on another Cortex-M4 and it managed fine.

BMD is a traditional superloop with interrupts. I recommend hooking your lwip heavy thread into there, and hooking the lwip fast polling light thread onto the (now) 1kHz SysTick or another periodic timer. If that's enough for gdb tcp functionality, then consider external uart proxy, traceswo passthrough, rtt proxy, and debugging logs proxy. BMF mixes most of these into a single USB CDC-ACM, but it not need be that way. Using it as a library is complicated, +1 for Farpatch which targets wireless ESP32 and IDF (FreeRTOS?) but replaces half the interfaces code. There is another project targeting esp8266 which enables Non-Stop continuous debugging mode, this upstream BMD is all-stop halting debug only.

Also please consider masking interrupts in jtagtap/swdptap bitbanging routines, which are universal but not protected like this (yet). There was no preemption outside of occasional USB FS CDC-ACM postponed dequeueing and SysTick. Threads and random Ethernet broadcast ARP traffic may interfere.

[elagil]

These are very helpful tips, thanks a lot! I will investigate the existing options.

Since you mention bitbanging: can SWD/JTAG be implemented by means of the SPI peripheral as well? The schematics of the BMP seem to indicate that.

[dragonmux]

In theory, this issue should now be solved at least with Meson as if you use the project as a subproject, it will build libblackmagicdebug as a library that can then be linked to. Please let us know how you get on with that and if it solves your query.

[elagil]

Thanks for the update! I will investigate that.

Currently, I don't build with Meson (my RTOS ships with Makefiles), but I can probably transition to it.