Reproducing rust/cargo packages for sb2

[rubdos]

Hi! This is kind-of a continuation of the derailed discussion of #11.

I've finally been able to build rust.spec (now with Rust 1.51), and I have a few questions and proposals.

First a few observations:

1. The current version of this package can be built for three architectures. However, only on x86/i486 this yields an actual package; on aarch64/armv7hl this produces a stub that says "This is the stub rustc. If you see this, scratchbox2 is not working".
2. The version -git3 (rust-1.44.0+git3-1.2.3.jolla.armv7hl) is what gets shipped in the Application SDK. Until that version (in fact, until git5/31dd484), the rust.spec in here produced an actual rustc/cargo that worked in sb2 for emulated building of applications. That commit changed this behaviour to produce stubs, and changed the paradigm from sb2-building to Platform SDK cross compiling.
3. This repository still contains the 0006-Provide-ENV-controls-to-bypass-some-sb2-calls-betwee.patch patch that makes it possible to compile a rustc that runs in sb2.

Now a few questions:

1. Are my observations correct?
2. Where is Jolla headed with this package? Are you targeting the Application SDK or the Platform SDK, or both?
3. What will become of the Application SDK Rust package, will this be abandoned and be a forever 1.44+git3, or will this eventually get an update?
4. This thing was really painful to build. As in: I have never felt that enraged and emotional about a piece of software since I left MS Windows when I was 15. That's ten years ago. I will open a merge request with build instructions, let's document this thing together, and keep the documentation in sync.
5. I still would like to get rid of the crappy build "system" that I use for Whisperfish, in favour of something mainstream, i.e., I want to build Whisperfish in sfdk, ideally, or in Platform SDK if necessary. The former is preferred (because there's a ton of invoking gcc that I don't want to cross build anymore). How can I help Jolla to get Rust as a tier-1 language for SailfishOS, at least as a compiler for stupid CLI applications?

Best regards,

Ruben

[Thaodan]

Can you please follow/answer to #11? I explained 1-3 there. It is related to how scratchbox2 works. ARM is usually the target so it does not make sense to build it for it but use the x86 build of the ARM compiler from the tooling.

[martyone]

It's not necessarily incorrect that a new issue was opened - regular version bumps are one thing, learning how to build stuff is another thing :)

It may not be immediately clear how sb2 operates, so maybe this could shed some light on it https://sailfishos.org/wiki/Platform_SDK_Installation#Scratchbox2 and this will cover most details should there be any question left https://git.sailfishos.org/mer-core/scratchbox2/blob/master/scratchbox2/docs/SB2_internals_1st_ed_20120425.pdf .

Before, the rust compiler was not used as a cross-compiler. Instead the target-compatible executable was used to compile rust code for a target. Later we enabled cross-compiling, so nowadays just the i486 (host-compatible) rust compiler is needed. It is meant to be installed under the tooling while the target-compatible stubs need(?) to be installed under the targets for proper sb2 operation.

One special thing with rust compared to other compiled languages is that for one of its language features, the Procedural Macros, it produces a temporary host-compatible code which it then executes as a later part of the compilation. It requires a host-compatible linker for that purpose - both running on host and producing host-compatible binaries.

This requirement is something that was not usual to see at the time scratchbox2 was designed, so it does not really support it. Fortunately there is a feature of sb2 designed for the purposes of its own development, that we could use for this purpose. It is the magic "host-cc" compiler. When you install GCC directly under the SDK (not under the tooling), you can then invoke that compiler from a sb2 session using the magic name "host-cc".

Cargo is configured to use it here https://github.com/sailfishos/rust/blob/master/rust.spec#L527 and it does so with the help of an exec wrapper created here https://github.com/sailfishos/rust-cross/blob/master/rust-cross-template.spec#L72

Regarding your questions:

1. Quite correct, hopefully it will be more clear to you now
2. The package is used everywhere: Application SDK, Platfrom SDK and OBS
3. Unless I missed something, the next SDK release will use 1.44.0+git10
4. Cross-compilation is not a trivial problem in general. Enabling rust cross-compilation in a way compatible with the current toolbox was painful. Now you should know. Had you known that before, you would say building it was just a piece of cake I believe :)
5. Unless I missed something, Rust (cross-)compiler should be available regularly since the next SDK release. This magic that you found before will still be needed https://git.sailfishos.org/mer-core/gecko-dev/blob/master/rpm/xulrunner-qt5.spec#L224

[rubdos]

Can you please follow/answer to #11? I explained 1-3 there.

I fail to see where you explained what Jolla is planning to do with the Rust packages in #11, @Thaodan. From @martyone's comment, it is however obvious that my ignorance of sb2's architecture (together with not seeing rust-cross) is a big part of my issue.

---

It's not necessarily incorrect that a new issue was opened - regular version bumps are one thing, learning how to build stuff is another thing :)

11 was becoming a mess, this is my attempt at getting a bit of structure. Since Jolla said that regular version bumps will not at Rust's pace (which we all understand), we might as well close #11 after the build process gets a bit reproducible.

It is meant to be installed under the tooling while the target-compatible stubs need(?) to be installed under the targets for proper sb2 operation.

That starts to make a lot of sense to me now.

here https://github.com/sailfishos/rust-cross/blob/master/rust-cross-template.spec#L72

This is the first I learn about the rust-cross repository... wow. More sense incoming. Not sure how I was able to miss that.

1. Quite correct, hopefully it will be more clear to you now

A lot.

2. The package is used everywhere: Application SDK, Platfrom SDK and OBS

Which is indeed obvious from your text.

3. Unless I missed something, the next SDK release will use 1.44.0+git10

:+1:

4. Cross-compilation is not a trivial problem in general. Enabling rust cross-compilation in a way compatible with the current toolbox was painful. Now you should know. Had you known that before, you would say building it was just a piece of cake I believe :)

Oh from the Git log, I did get that you folks experienced a lot more pain than I did :-)

5. Unless I missed something, Rust (cross-)compiler should be available regularly since the next SDK release. This magic that you found before will still be needed https://git.sailfishos.org/mer-core/gecko-dev/blob/master/rpm/xulrunner-qt5.spec#L224

Which boils down to: I still need to apply patch 0006 on Rust 1.51, then install it on the tooling, then get the cross packages done.

It all makes sense now, I think. I suggest I come back to you with build instructions in a merge request here and at rust-cross when I figure everything out (which should be smooth™ from here), such that nobody has to live through this pain anymore ;-)

[martyone]

[...] I suggest I come back to you with build instructions in a merge request here and at rust-cross when I figure everything out (which should be smooth™ from here), such that nobody has to live through this pain anymore ;-)

Yup, thank you in advance!

[Thaodan]

Can you please follow/answer to #11? I explained 1-3 there.

I fail to see where you explained what Jolla is planning to do with the Rust packages in #11, @Thaodan.

I (tried) to explain your observations - @martyone explained that much better after me.

[rubdos]

I (tried) to explain your observations - @martyone explained that much better after me.

Looks like my main problem was that I didn't understand sb2 too well.

It indeed seems that the -git10 version is in the newer Platform SDK already, I was still enjoying 4.0.

Can either of you comment on one extra thing? I wonder how the sb2 target should resolve a call to cargo. If I now run rpmbuild against sb2, a cargo build statement in my RPM file yields me "This is the stub cargo. If you see this, scratchbox2 is not working. Called as" (which is exactly what I would expect, since that's what in /usr/bin/cargo). So my question is: how should I call the tooling-cargo?

[martyone]

Your scratchbox2 version is likely missing this change https://git.sailfishos.org/mer-core/scratchbox2/merge_requests/47.

[rubdos]

Your scratchbox2 version is likely missing this change https://git.sailfishos.org/mer-core/scratchbox2/merge_requests/47.

Awesome, upgrading Platform SDK to 4.1 via ssu & zypper made it accelerated.