Support heterogeneous build environments

[yanj403]

Is there any plan to add support for heterogeneous build environments? Like icecream did, pack the buildenv to tarball then send to remote peer....

[0x804d8000]

From what I can tell, icecc provides that functionality via chroot (or some equivalent mechanisms). This is mostly useful for supporting multiple versions of compiler.

If this is the feature what you’re asking for, it’s already partly supported by yadcc.

Adding paths to alternative compilers to extra_compiler_dirs when starting daemon on compile server should do the trick, I think. (See the documentation for more details.)

What’s not possible with yadcc but supported by icecc is when you want to allow users to use arbitrary compilers that were not known to the compile cluster beforehand. I’d say icecc is more flexible in this case, but I’m not sure that’s a normal situation in production environment. Besides, that would require root privilege to run the daemon (required by chroot).

If you’re proposing supporting non-x86 compilers on x86 machine, I’m not sure it’s possible without help from qemu. IIRC icecc doesn’t allows this, either.

[yanj403]

chroot

Appreciate your quick response. Yes, what I am looking for is support arbitrary compilers which can NOT be installed beforehand. Is there any plan to support it?

[0x804d8000]

I don’t think it will be implemented any time soon, unfortunately. I failed to imagine a case where it’s desired in production.

Are you using the feature in production environment? If so, do you mind to elaborate your use case?

Although I’m not working on the feature, from a design perspective, I can see some challenges in implementing it (in case someone wants to submit a PR):

• Transferring the compiler to the compile server can be bandwidth-intensive. This is especially true when used with a large compile cluster, where the client is more likely communicating with a “fresh” compile server — In this case even a compile-server-local cache can help little.
• One need to filter syscalls carefully to avoid a “malicious” compiler (e.g. a backdoor or a proxy) being executed. This is even more important if the compile cluster is operated by a different group of people than the users.
• The protocol needs some refactor, both in assigning compile server, and accessing compile cache. This should be relative easy, though.

[0x804d8000]

FWIW, there is a workaround for this. If you can push your compiler to an NAS that’s accessible to the compile cluster, you might want to check the implementation in branch next. There the daemon can re-scan periodically to see if new compilers are provided, and use them as appropriate. See https://github.com/Tencent/yadcc/blob/next/yadcc/daemon/cloud/compiler_registry.cc#L44

[yanj403]

Yes, I want to use the feature in production environment. And the use case is compiling Android(AOSP). For Android build, the toolchain used is from the source tree, that means different source commit may use different toolchain.

As the build time in Android is considerable, so the time used to transferring the compiler is small relatively.

[0x804d8000]

I agree that’s a totally valid use case. This also reminds me about compiling UE4. Personally I had no experience about compiling either of them but both are so popular among developers that they should be better supported.

Unfortunately for the moment I don’t have much time to implement the feature, I’m open to a PR about this.

TBH, however, since I haven’t tried yadcc with something like AOSP, I’m expecting several issues (besides this one) to show up in this case. Therefore I suggest you first try adding prebuilt compilers shipped with AOSP to yadcc manually and see if everything else works as intended.

[yanj403]

ok, thanks for your advise.