Linking with static libraries

[nashif]

Reported by Flavio Santes:

Provide an example on how to integrate an external static library to Zephyr. This could be very useful if a third-party component needs to be integrated to a Zephyr application.

(Imported from Jira ZEP-366)

[nashif]

by Anas Nashif:

you should have created this as a story, not a requirement

[nashif]

by Flavio Santes:

It seems that it could be useful for the community.

[nashif]

by Paul Sokolovsky:

Based on having a look at https://gerrit.zephyrproject.org/r/#/c/2048/ and https://gerrit.zephyrproject.org/r/#/c/2476/ (which are marked as implementing this user story), I'd like to share my use case. I'm starting with the work https://lists.zephyrproject.org/archives/list/devel@lists.zephyrproject.org/thread/ODCXJZDXZZVRKMO44LVZIUI4ZIL6KLJQ/ of adding Zephyr/Arduino 101 target for JerryScript. And exploring it in the direction of proving that Zephyr is a cross-platform RTOS, so instead of adding Zephyr port for a particular board to a generally hardware-independent project like JerryScript, it's possible to add a generic Zephyr target, and let Zephyr deal with a majority or hardware-specific matters. And immediate aim in that regard is making the project from the link above (https://github.com/sergioamr/jerryscript/tree/wip/sergio/zephyr_arduino101/targets/arduino_101) to build and run for qemu_cortex_m3.

While following that path, I discovered that:

1. There's no easy way to pass an extra library to link a Zephyr application with.
2. There's no easy way to pass search location for an extra library to link with.
3. Not providing an easy way to do the above leads to very convoluted workarounds and extra work required for integration, and it's easy to produce a build script (makefile) which will by a chance work with a particular architecture, but will break in an unobvious way for another architecture.

I spent extensive time to find workarounds for these problems (without changing Zephyr source, that was part of conditions - to see if Zephyr is already ready for integration with 3rd-party components). So, based on these struggles, I would say that this user story is important, and hopefully not just for me, but for an emerging wider community. But I'd especially like to draw attention to point 3 above (I'll cover it in more detail in follow-up message). The port above was done by an Intel engineer. I'm not sure if he's a member of Zephyr team, but arguably he might have better insight into it than a random community guy like me. And yet after spending almost a week on that, I came to conclusion that he did more work than was required to produce JerryScript port, and his port literally works by a chance on x86/iamcu, and breaks on multiple levels when tried with another arch.

Based on the above, I would propose to extend the scope of this user story. Instead of "Allow to link with user static libraries", formulate it "Allow user to include arbitrary compile- and link- time options into a Zephyr project". That will cover static libs usecase, but most importantly will allow to establish once and for all a consistent interface to adjust build settings (up to a usecase of "I'ma compiler writer implementing new optimization option and I want to try it on a real-world Zephyr app by passing it on make command line.")

Based on the above, requirements may be:

1. Keep it simple and reuse general knowledge. Zephyr may use its internal high-level make vars, but CFLAGS and LDFLAGS is what everyone knows, so for quick user adjustment, reusing those names would be the best, and each of them will cover few usecases (e.g. CFLAGS for adding external header location and trying a new optimization option; LDFLAGS for both specifying external libs and their locations).
2. Have these variable be easily distinguishable by having common prefix, suffix. E.g. USER_CFLAGS/USER_LDFLAGS, EXTRA_CFLAGS/EXTRA_LDFLAGS, or CFLAGS_EXTRA/LDFLAGS_EXTRA.

[nashif]

by Paul Sokolovsky:

3. Not providing an easy way to do the above leads to very convoluted workarounds and extra work required for integration, and it's easy to produce a build script (makefile) which will by a chance work with a particular architecture, but will break in an unobvious way for another architecture.

So, a more detailed case study on trying to make https://github.com/sergioamr/jerryscript/tree/wip/sergio/zephyr_arduino101/targets/arduino_101 first build, then run for qemu_cortex_m3.

First thing I noticed that the port contained asm implementation of setjmp/longjmp for x86. I was a bit surprised as teh project used newlib as libc, and it definitely should have had it. I commented it and got missing setjmp/longjmp for qemu_cortex_m3 build. Investigating the issue, turned out that zephyr_arduino101 used TOOLCHAIN_LIBS to pass in JerryScript libs. TOOLCHAIN_LIBS is, well, -lgcc and is tackled at the very end of linker command line, and as a result following line was executed:

-lm -lc -lrelease.external-cp_minimal-entry \ -lrelease-cp_minimal.jerry-core -lrelease-cp_minimal.jerry-libm.lib -lgcc

and of course JS libs didn't pick up stuff from libc. As a result of that, @sergioamr added setjmp/longjmp and string function implementations, which weren't really needed and setjmp/longjmp implementation made the port arch-specific.

It doesn't end here though - after I got qemu_cortex_m3 to build, it didn't boot in qemu. Investigation showed that ARM mode code was generated instead of Thumb2. fast-forward painful debugging, it happened because project makefiles overrode LIB_INCLUDE_DIR, and that variable is set in very subtle (if not to say worse) way, so touching it uncarefully just led to ARM code being generated silently.

The point of this is: if there was (official, documented) support for passing use CFLAGS and LDFLAGS into a Zephyr build, nothing from the above would happen.

P.S. Checking https://github.com/sergioamr/jerryscript/commits/wip/sergio/zephyr_arduino101 now, the branch was rebased and at least first of the issues above (extra setjmp/longjmp impl) was fixed. But TOOLCHAIN_LIBS/LIB_INCLUDE_DIR are still used to communicate with Zephyr build system.

[nashif]

by Anas Nashif:

Agree this is an issue as described in the last 2 comments, however I do not see how the associated gerrit submission fixes the issue. As suggest, we need to open new stories or even bugs with the real issue. The sample submitted can be seen as an example why this needs to be fixed, but not something that solves the issue.

Can you please open new stories with the above detailed issues so we can fix this in the build system?

[nashif]

by Paul Sokolovsky:

Anas: Indeed, the first patch prepared by Flavio, https://gerrit.zephyrproject.org/r/#/c/2048/ , was further along the idea. But it was abandoned, and in the next patch, https://gerrit.zephyrproject.org/r/#/c/2476/ , he explicitly mentions "This commit does not depend on modifications to the main Makefile." As the were no detailed message for abandonment of #2048, I may only imagine that there was internal (e.g. spoken) discussion with Zephyr team, with formal criteria like "Do we really need it, or is it more of wishful thinking now?" and "Can we do it without patching Zephyr?".

So, in my extensive comments above, I'm trying to support Flavio's cause: "Is it wishful thinking?" - no, it's really needed, or otherwise there's bunch of confusion. "Can it be done without patching Zephyr?" - Yes, it can be, but it's better to apply simple 2-3 line patch which would provide consistent user interface, which would allow for even more usecases than passing in external static libs.

Hope that makes sense. I assume the rest of the suggestions are addressed to Flavio, though if I can help with anything, I'll be glad to.

[nashif]

by Anas Nashif:

Paul Sokolovsky This is a known issue but I did not think the approach for building external libraries presented by Flavio Santes was something solid, it was just a stopgap. Stopgaps have cause problems in the past, so I agree this needs to be addressed, but I do not think we should rush into it and instead provide a proper and long term solution, this is being looked at right now.

[nashif]

by Anas Nashif:

btw, another approach to link an external library that we used in the past.

https://github.com/nashif/zephyr_modules

Works the same way with less structure and Makefiles, but still not the ultimate things we need.

[nashif]

by Paul Sokolovsky:

Anas, thanks for the response.

I did not think the approach for building external libraries presented by Flavio Santes was something solid, it was just a stopgap

I can understand and relate to that bit, and Flavio in a README in his patch mentions that too: "If the library's source code is available, perhaps it could be more easier to integrate that source code to your application than creating the static library." Here's one point though - Zephyr is pretty much a newcomer to a crowded RTOS space. It's fair to think it'll find its distinctive place just due to names standing behind it, but velocity of community adoption may largely depend on how it's easy to start playing with it. And the common userstory might be "I have my libs, I already built them - I know how to do that. Now I want to try to integrate them in a (sample) Zephyr app, and if I'll be able to that easily, I'll think that Zephyr is cool, and will look (wait) for right way to do that."

That however the last argument I would have, and if something good is cooking, I'm happy to wait and see, and happy to have given my small +1 towards resolving that.

But in the meantime, zephyr_arduino101 was merged into JerryScript upstream: https://github.com/Samsung/jerryscript/commit/7583b63a6eef61f676cf02c08d52b865f964116a . So, a stopgap or not, it's starting to spread around (and the solution in that patch is "worse" than what's in Flavio's patch, as it's just broken for ARM targets).

[nashif]

by Paul Sokolovsky:

https://github.com/nashif/zephyr_modules

FYI, that didn't build for me ("make[3]: *** No rule to make target `/mnt/hdd/projects-3rdparty/Embedded/Zephyr/zephyr/._modules/Makefile'. Stop."). Not a big deal though, as Flavio's way (independently rediscovered by myself) does work, and I'm happy to wait for an official solution.

[nashif]

by Sakari Poussa:

I have the same build error with Anas Nashif modules so I would be good if it gets fixed.

I think we have multiple issues here. One, the jerryscript port for Zephyr should made to work for ARM as well. We did not focus that initially but would be willing to hep to get it fixed. Two, it would be good to have Zephyr build system to easily support bringing in 3rd party libs, like jerryscript. I think Anas' example is a good start and we are investigating if we can use that for jerryscript integration well. This bug is about the latter.