humenda
commented
3 years ago Hi
I am currently trying to extract the Rust libraries from this repository for making compilation of binaries for l4re-targets a little bit faster.
Right now, for compiling Rust projects, it is required to build everything of l4re. And this takes quite some time.
AFAIK, this is not the case. You need to build everything once. Later, you can go to the corresponding subdirectory and invoke make there and it'll built just this again.
So I extracted the libraries for (maybe) compiling l4re-binaries of rust programs directly. But the linker part of the build seems to hinder this endeavour . When
cargo buildfinishes the build by calling in the linker, there are a few problems.
(I'll paste the l4-bender command, which is invoked by cargo below)
- The first problem seems to be that all of these paths are interpreted as if they were relative paths, so I had to call this command from the root-directory
- The second following problem seems to be that every single file (and folder) which is mentioned in the command, is interpreted as a
.pc-file. Even the last part of the command with-owhich should normally define the output file (but here it is used as an input with a .pc ending?)For compiling I used my "own" rust toolchain:
https://github.com/Steav005/rust/tree/update-l4re-target Which is based upon atopia's rust toolchain https://github.com/atopia/rust/tree/update-l4re-target Which is based upon the recommended toolchain https://github.com/humenda/rust
I wasn't aware that this has been modified in the mantime. I never had the time to properly blog about this, so here's a quick summary about why things are as they are. Rust/Cargo insist on calling the linker (except for work-arounds, see further below). The same goes for BID, the L4Re build system. L4Re relies on quite some linker magic and hence the make logic was at some point all put into a large Perl script called l4-bender. L4-bender calls the linker and bends the linker arguments passed to it further to pass it to ld. The issue with l4-bender is that it behaves almost like ld, but really just almost. E.g. the order of arguments matters, especially the separation of "--". Rustc has only limited support of passing through linker arguments and in particular, the order is hard to influence. In addition, l4-bender needs arguments passed from Make that contains spaces and these cannot be passed to Rustc.
My solution for this has been to add a new Rust linker variant l4-bender. This was actually suggested by some Rust folks. The issue is that the linker backend changes frequently and there has been no automatic testing of it.
L4Re uses PC files to find its libraries in the build tree and l4-bender unwraps the PC files and passes the required arguments to ld. I did not look at the links that you've provided, but it is very likely that the L4Bender struct in the compiler has a bug so that it assumes to make more things a PC-file-argument where it shouldn't be.
The main problem seems to be that the called
l4-bendercommand is wrong.
With this custom rust toolchain and the21.07 l4-bender, basic compilation of this repository works flawlessly for theamd64target. (And I have never seen a.pc-file in this repo's outputs).
Under obj/**/pc?
I know that this repository uses
makeandrustcdirectly for compiling rust libraries/binaries.
But there are to many files to this project for me to grasp the entire compilation process.
Yeah, the relevant bits are in src/l4/mk. The rest are mostly Makefiles including other makefiles or those in obj/ that are auto-generated. It's all a bit arcane.
How should the correct
l4-bender-command be structured ? Is it even possible to build binaries for l4re externally while static-linking the required libraries? Are there documents on how to usel4-bender?
2): yes. 1) 3): L4Re lacks documentation on the build system, apart from the initial README. The only docs that you can get are the undocumented sources themselves. Given the current small interest in maintaining the l4-bender linker variant in Rustc, I would suggest the following:
- Building of libraries stays the same (lib.mk).
- Applications are built as static libraries.
- This needs to be specified in Cargo.toml for each application.
main.rshas to export a C-compatible main function.- in bin.mk, binary.inc and maybe rules.mk, the rules that contain
RSandRUSTneed to be adapted. They would need to be a copy of the C versions, though not linking object files, but the static archive that was produced.
I don't know whether you have time and energy to look into this and whether my description has been clear enough. I have tried to update my Rust toolchain to a recent Rustc, but the L4Bender support is completely broken and I would basically need to rewrite the l4-bender support. I lack the time to do so. This is also the reason why I did not do anything about the other PRs of yours: I cannot test them.
The steps to change to static library building could be done in around 1-3 weeks and I'd be happy to help out in case you would like to look into it. And even though time is a limited resource on my side, I'm definitely around for helping to investigate issues.
Please let me know what you think.
sevenautumns
wucke13
hargoniX
Hello,
I am currently trying to extract the Rust libraries from this repository for making compilation of binaries for l4re-targets a little bit faster.
Right now, for compiling Rust projects, it is required to build everything of l4re. And this takes quite some time.
So I extracted the libraries for (maybe) compiling l4re-binaries of rust programs directly. But the linker part of the build seems to hinder this endeavour . When
cargo buildfinishes the build by calling in the linker, there are a few problems.(I'll paste the l4-bender command, which is invoked by cargo below)
.pc-file. Even the last part of the command with-owhich should normally define the output file (but here it is used as an input with a .pc ending?)For compiling I used my "own" rust toolchain:
https://github.com/Steav005/rust/tree/update-l4re-target Which is based upon atopia's rust toolchain https://github.com/atopia/rust/tree/update-l4re-target Which is based upon the recommended toolchain https://github.com/humenda/rust
For
l4-bender, I used the latest snapshot version 21.07The main problem seems to be that the called
l4-bendercommand is wrong.With this custom rust toolchain and the
21.07 l4-bender, basic compilation of this repository works flawlessly for theamd64target. (And I have never seen a.pc-file in this repo's outputs).I know that this repository uses
makeandrustcdirectly for compiling rust libraries/binaries.But there are to many files to this project for me to grasp the entire compilation process.
How should the correct
l4-bender-command be structured ? Is it even possible to build binaries for l4re externally while static-linking the required libraries? Are there documents on how to usel4-bender?The failing l4-bender invocation