Support cross building - Githubissues

dswiecki commented 1 year ago

Fixes #1503 It's a POC how cross building can look like. Assumes buildx available in system. Problem and limitations:

due to docker buildx sytnax pushing must be done at once with docker buildx build and all platforms. Otherwise, only the last arch is pushed, I could be done with docker manifest but would be more complex
no changes in publishLocal - I assume that the user can only have arch version supporting his computer architecture locally
binary compatibility - here I need your advice on whether you'll accept changing publishDocker signature and which version compatibility should be checked for

Example docker repo https://hub.docker.com/r/dswiecki/docker-test/tags

lightbend-cla-validator commented 1 year ago

At least one commit author (dsw@touk.pl, dsw@touk.pl) is not linked to a user. See https://help.github.com/en/articles/why-are-my-commits-linked-to-the-wrong-user#commits-are-not-linked-to-any-user

lightbend-cla-validator commented 1 year ago

Hi @dswiecki,

Thank you for your contribution! We really value the time you've taken to put this together.

Before we proceed with reviewing this pull request, please sign the Lightbend Contributors License Agreement:

https://www.lightbend.com/contribute/cla

lightbend-cla-validator commented 1 year ago

Hi @dswiecki,

Thank you for your contribution! We really value the time you've taken to put this together.

Before we proceed with reviewing this pull request, please sign the Lightbend Contributors License Agreement:

https://www.lightbend.com/contribute/cla

muuki88 commented 1 year ago

Thanks a lot. I will take a look ASAP.

Binary compatibilty shouldn't be too much of a concern

mpro7 commented 1 year ago

Any updates on this PR?

nik-kashi commented 1 year ago

Thanks a lot. I will take a look ASAP.

Binary compatibilty shouldn't be too much of a concern

@muuki88 Hey Muki, Would you take a look at this PR, please?

muuki88 commented 1 year ago

There you go: https://github.com/sbt/sbt-native-packager/releases/tag/v1.9.12

mzuehlke commented 1 year ago

I have the feeling that https://github.com/sbt/sbt-native-packager/pull/1527/files#diff-b866751bc91854625b6a5d821ae6a4df02736ceb3581779d81319caee1d9d2cdR281-R289 broke publishing.

Shouldn't the else branch be dockerExecCommand ?

See https://github.com/scala-steward-org/scala-steward/pull/2928

bphenriques commented 1 year ago

Is this automatic or do we need to add some settings?

mpro7 commented 1 year ago

@bphenriques it looks like you need to provide below line in build.sbt file, otherwise only the architecture you're running on your machine will be published.

dockerBuildxPlatforms := Seq("linux/arm64/v8", "linux/amd64")

manuzhang commented 1 year ago

@muuki88 It looks dockerBuildxPlatforms is not supported for publishLocal task.

muuki88 commented 1 year ago

It should be. Both commands should use the same options.

hedefalk commented 3 weeks ago

It seems publishLocal does not use dockerBuildxPlatforms but instead overrides with host platform only. "Smartness" that was very confusing to me. I thought it would do the same, but just not push.

muuki88 commented 3 weeks ago

@hedefalk thanks for confirming. Would you like to open a PR and fixt that behaviour?

hedefalk commented 3 weeks ago

@hedefalk thanks for confirming. Would you like to open a PR and fixt that behaviour?

Sorry, I'm not really up for it right now, but I'll add a todo when I get a breather… :/

But to understand context - you @muuki88 is the maintainer here and would also want "Docker/publishLocal" to do the same thing as "Docker/publish" minus the push?

That was what I had expected and I kinda lost a few hours on believing that was the case working on x-building for cloud, but I can also see the rationale why one would want publishLocal to be streamlined for quick local usage only building for the host platform. Maybe one could support both in a nice way and possibly add some docs on it…?

muuki88 commented 3 weeks ago

Hi @hedefalk

No worries :relaxed: If you don't ask, you don't know. We all do this in our spare time :wink:

But to understand context - you @muuki88 is the maintainer here and would also want "Docker/publishLocal" to do the same thing as "Docker/publish" minus the push?

IMHO this makes sense. I have no experience with other docker build platforms, but from a "don't do weird shit" perspective, the behaviours should match by default. If this is not doable, because of contstraints, then at least a big warning should indicate "hey - what you build locally is probably not what you build in production because xyz"

sbt / sbt-native-packager

Support cross building #1527