search

scala / scala3

The Scala 3 compiler, also known as Dotty.
https://dotty.epfl.ch
Apache License 2.0
5.83k stars 1.05k forks source link

Distribute releases as .deb and .msi #12502

Open bjornregnell opened 3 years ago

bjornregnell commented 3 years ago

Expectation

There are not yet any links to packages under "Other resources" on https://www.scala-lang.org/download/ for deb and msi to the new awesome Scala 3.0.0

It would be good to know how the naming standard of the links will be when it is eventually up; direct links are published from elsewhere, e.g. in coming course material etc.

Will it be something like this? https://downloads.lightbend.com/scala/3.0.0/scala-3.0.0.deb

bjornregnell commented 3 years ago

A reason I'm asking is that I need to give links to deb packages to our sysadmin of our university computers that will be upgraded soon.

SethTisue commented 3 years ago

(It won't be lightbend.com, but I don't know if it's been otherwise discussed.)

smarter commented 3 years ago

There's no process in place currently for generating .deb, the released artifacts are at https://github.com/lampepfl/dotty/releases and only includes tarballs currently. I assume our sbt-assembly settings could be tweaked to also generate .debs but we need someone to investigate that.

bjornregnell commented 3 years ago

I guess deb and msi would be good as probably quite a few have gotten used to rely on them over the years and it might be good to reduce the gap between scala2 and scala3. I know our sysadmins prefer .deb over .tar.gz, but if deb don't exist they'll have to manually place the stuff in the file system, which is also doable but more work. (We have a lot of strange stuff installed on our student lab computer's linux installations...)

So, the download link is currently https://github.com/lampepfl/dotty/releases/download/3.0.0/scala3-3.0.0.tar.gz @smarter Can we rely on that the link will have the same form for 3.x.y for the foreseable future as in https://github.com/lampepfl/dotty/releases/download/3.x.y/scala3-3.x.y.tar.gz with scala3 (and not just scala). And will the dotty part of the url be kept or will it be called scala (or scala3) instead?

smarter commented 3 years ago

I think this link format should keep working yeah, even if we rename the repo it should become a redirect.

bjornregnell commented 3 years ago

ok thanks @smarter good to now.

It seems as sbt-native-packager can do deb-packaging with some settings ceremony and installing some linux stuff like dpkg-deb etc: https://github.com/sbt/sbt-native-packager https://www.scala-sbt.org/sbt-native-packager/formats/debian.html

bjornregnell commented 3 years ago

@SethTisue How is Scala 2 packages deb msi build currently? Is that process easy to more or less copy-paste for Scala 3?

SethTisue commented 3 years ago

How is Scala 2 packages deb msi build currently? Is that process easy to more or less copy-paste for Scala 3?

all that stuff is over in https://github.com/scala/scala-dist (and https://github.com/scala/scala-dist-smoketest). it uses (a now probably rather outdated) version of sbt-native-packager

scala-dist has been largely untouched for some years now, and I was never super familiar with it in the first place (since we've barely needed to touch it since Scala 2.11 days), so I'm not really sure what the difficulty level would be of adapting it to Scala 3, but offhand I wouldn't expect there to be any fundamental barrier. most of the actual functionality comes from sbt-native-packager.

bjornregnell commented 3 years ago

Thanks for pointers. I had a look and there is some complexity in it above my head, but I guess much of that is due to docker and CI and platform specific stuff. Perhaps the actual packages are easy to generate, but it seems as if one VM is needed per platform etc.

bjornregnell commented 3 years ago

@smarter Since 22 days ago winget has support for 2.13.5. I guess msi-packaging is needed for winget to support also Scala 3. So solving this issue would also help windows user that opt in for the new windows package manager (I guess quite a few of my students on windows will want that). Is there any progress or plans for this? https://github.com/microsoft/winget-pkgs/tree/master/manifests/s/Scala/scala https://github.com/microsoft/winget-pkgs/blob/master/manifests/s/Scala/scala/2.13.5/Scala.scala.yaml https://docs.microsoft.com/en-us/windows/package-manager/package/manifest?tabs=minschema%2Cversion-example

smarter commented 3 years ago

Is there any progress or plans for this?

No one is assigned on this which means no one is working or planning to work on this and it's open for taking by anyone interested.

bjornregnell commented 3 years ago

@smarter (or who is responsible for the packaging of scala 3? perhaps @anatoliykmetyuk who is writing the release blogs) I was able to build a deb package after doing these preliminary additions (should probably go elsewhere in the dotty build?): in build.sbt:

//added to try native packaging: should perhaps go elsewhere (?)
//https://www.scala-sbt.org/sbt-native-packager/formats/debian.html#requirements
enablePlugins(JavaAppPackaging)
enablePlugins(DebianPlugin)
debianPackageDependencies := Seq("java8-runtime-headless")
maintainer := "anatoliykmetyuk@gmail.com"  // Who should be the person? must be a person?
packageSummary := "scala3"  // should include the right version etc
packageDescription := """More information here: https://www.scala-lang.org/"""

in project/plugins.sbt

//to enable native packaging: 
//https://github.com/sbt/sbt-native-packager
addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.7.6")

The changes are available here: https://github.com/bjornregnell/dotty/commit/c8e9d6dccf31c695330f8fa328b1d5bef78645da The above changes made me able to do:

sbt:scala3> debian:packageBin
[warn] sbt 0.13 shell syntax is deprecated; use slash syntax instead: Debian / packageBin
[info] Wrote /home/bjornr/git/hub/bjornregnell/dotty/target/scala-3.0.0/scala3_3-3.0.1-RC1-bin-SNAPSHOT-nonbootstrapped.pom
[info] Wrote /home/bjornr/git/hub/bjornregnell/dotty/library/target/scala-3.0.0/scala3-library_3-3.0.1-RC1-bin-SNAPSHOT-nonbootstrapped.pom
[info] Wrote /home/bjornr/git/hub/bjornregnell/dotty/tasty/target/scala-3.0.0/tasty-core_3-3.0.1-RC1-bin-SNAPSHOT-nonbootstrapped.pom
[info] Wrote /home/bjornr/git/hub/bjornregnell/dotty/interfaces/target/scala3-interfaces-3.0.1-RC1-bin-SNAPSHOT.pom
[info] Wrote /home/bjornr/git/hub/bjornregnell/dotty/compiler/target/scala-3.0.0/scala3-compiler_3-3.0.1-RC1-bin-SNAPSHOT-nonbootstrapped.pom
[warn] You have no main class in your project. No start script will be generated.
[warn] You have no main class in your project. No start script will be generated.
[info] Building debian package with native implementation
[info] dpkg-deb: building package 'scala3' in '../scala3_3.0.1-RC1-bin-SNAPSHOT-nonbootstrapped_all.deb'.
[success] Total time: 2 s, completed May 30, 2021, 3:48:35 PM
sbt:scala3>

What is left (at least...) if this is a viable route forward:

More inspiration for settings of sbt-native-packager can be found here: https://github.com/scala/scala-dist/tree/2.13.x/project in the Wix.scala Unix.scala and ScalaDist.scala files.

smarter commented 3 years ago

should probably go elsewhere in the dotty build?

Probably in the dist project since that's what's used to generate tarball (dist/packArchive): https://github.com/lampepfl/dotty/blob/master/project/Build.scala#L1717-L1718 /cc @liufengyun

integrate this in the CI/CD perhaps dockerize to enable package testing etc (I have no clue on what is needed there)

We run dist/packArchive and upload it to github in the publish_release job, so that's where we'd have to run whatever extra commands are needed to generate the .deb and .msi: https://github.com/lampepfl/dotty/blob/56abade12dadc0519eb149d53fb88e97fcd31c67/.github/workflows/ci.yaml#L541-L588

bjornregnell commented 3 years ago

@liufengyun @smarter I have pushed some more settings here but the deb-package is still produced in target and not in dist: https://github.com/bjornregnell/dotty/commit/d5d450351743b02c6c8f11829c3aa62a40b4a460 Let me know if I can help more (e.g. I'm happy to test a deb archive and an msi file on my machines); I tried to fiddle with the build architecture but I'm still in the dark on how the build is structured and how to get settings and plugins to be working with the dist sub project. And I have no clue about wix. Anyway, the CI-stuff is better done by someone with admin rights etc.

liufengyun commented 3 years ago

We are currently using sbt-pack. It was a choice several years ago. I think it makes sense to switch to sbt-native-packager, as it supports more features.

Anyway, the CI-stuff is better done by someone with admin rights etc.

@anatoliykmetyuk Could you please extend a hand?

bjornregnell commented 3 years ago

See also https://github.com/coursier/coursier/issues/2058 An .msi installer and a .deb package would be really helpful to regain the scala 2 user experience in terms of installation. Esp. since cs does not work for runing scala 3 in terminal (currently).

bjornregnell commented 3 years ago

The current situation on scala-land.org/download is more problematic than it seems; it actually does not say how to run a compiled app. I think this i pretty important and a severe regression from Scala 2 getting-started-instructions. @smarter Should I open a new issue on that matter, as packaging and absence of "how to run" are separate things?

smarter commented 3 years ago

On https://github.com/scala/scala-lang/ yes

bjornregnell commented 3 years ago

Ok thanks, I'll discuss there if the existing issue https://github.com/scala/scala-lang/issues/1248 should be a more general/specific/split/rephrased to make the importance of a runner for Scala 3 more explicit. I don't think this is "just" a coursier issue, but a degradation from Scala 2 and a "hygiene factor" so to speak.

bjornregnell commented 3 years ago

See also https://github.com/lampepfl/dotty/issues/12551

bjornregnell commented 3 years ago

Any progress on this? My windows-bound students that can't get WSL going run into the brick-wall, so they need an .msi... Let me know if I can contribute somehow.

michelou commented 3 years ago

Any progress on this? My windows-bound students that can't get WSL going run into the brick-wall, so they need an .msi... Let me know if I can contribute somehow.

Starting with version 3.0.2 of Scala 3 (see pull #13006) a Windows user can just unzip the distribution archive and use the batch files scala.bat, scalac.bat, etc.. (just as sbt.bat, mvn.cmd, gradle.bat, ant.bat, etc..). Your students then have to take care of the two environment variables %JAVA_HOME% and %PATH%.

bjornregnell commented 3 years ago

Thanks @michelou for the workaround and for contributing the bat-magic !! :heart: It still would be very good with a .msi install that sets the path and java home for pure beginners, as with Scala 2. Any progress on that or anything there that I can do to contribute?

michelou commented 3 years ago

@bjornregnell In the past I've developed several Windows installers using NSIS (e.g. RS MINERVE) but WiX 3 looks more adequate to your/our needs since

I'm experimenting with small examples and will eventually submit a PR if things go well.

bjornregnell commented 3 years ago

and will eventually submit a PR if things go well.

That would be really great! @anatoliykmetyuk @SethTisue It would be cool if this can be integrated with the CI-pipeline that produces the releases? It seems like the sbt native packager delegates to WIX, so perhaps that's the way to go? https://www.scala-sbt.org/sbt-native-packager/formats/windows.html

bjornregnell commented 2 years ago

FYI, @Husenap has made a custom bat-script just to help us survive the current situation in my scala course with a majority of students on windows, available here: https://github.com/lunduniversity/introprog/blob/master/tools/install-scala.bat @michelou

michelou commented 2 years ago

@bjornregnell I saw it; do you expect any feedback from me ?! PS. Using the GitHub link directly would be better : install-scala.bat.

bjornregnell commented 2 years ago

Yes thanks, I should have used that link. :) I just changed my comment above now to that explicit link. Sorry for not being explicit about expected feedback while tagging you -- I just wanted to show our work-around if you or others can benefit from that. Perhaps if you can share info on how it goes with .msi-installation-file for Scala 3 and if it is slated for 3.1.0 or if there are any showstoppers so far etc?

bjornregnell commented 2 years ago

@anatoliykmetyuk I have tested this .msi from @michelou 's repo https://github.com/michelou/wix-examples/releases/tag/scala3-3.1.0.msi and it works like a charm :)
Could this work be used as a basis for an official msi for the scala-lang.org release page for windows, so that there is an easy alternative for those on windows, esp beginners who don't know what a path is and just wants a single click official msi install?

bjornregnell commented 2 years ago

It could e.g. for now be linked under "Other ways to install Scala" on https://www.scala-lang.org/download/scala3.html

anatoliykmetyuk commented 2 years ago

/cc @prolativ

bjornregnell commented 2 years ago

Now also API docs included by @michelou here: https://github.com/michelou/wix-examples/issues/2