Open vsoch opened 5 years ago
The question is how to create .dsc package?
I think the "dsc" file is just a metadata file that has package information (checksums, etc.) looking at this guide https://wiki.debian.org/SimplePackagingTutorial
And you can see an example in this repository:
https://github.com/trevorsandy/lpub3d/blob/master/builds/linux/obs/debian/lpub3d.dsc
The instructions for the simple packaging tutorial seem pretty straight forward, but I'll let @yarikoptic weigh in.
dpkg-buildpackage
and (gbp buildpacke if you use it) would do that as a part of the overall procedure. You could also use dpkg-source -b
directly iirc but that one might not create .changes which you need to use for upload to Debian (mentors) using eg dput
.
If you haven't already, you should contact the people from https://wiki.debian.org/Teams/RustPackaging
okay I posted to the list, here https://alioth-lists.debian.net/pipermail/pkg-rust-maintainers/2019-October/007819.html and got a response quickly! https://alioth-lists.debian.net/pipermail/pkg-rust-maintainers/2019-October/007820.html
You can read the response in detail, but in summary:
https://gist.github.com/vsoch/af0d7533775e3c21418e6b8a6a3825bd
Also note that I ran the command cargo-debstatus debstatus
from the root of the repo, after commenting out #default-run = "nu"
otherwise I got an error message.
And directly from the response - the other challenges we'd run into:
As a spoiler, the output for nu is quite long unfortunately. Other issues you'd run into:
So - thinking / looking this over, I think we could try to tackle this now, and it would be huge amounts of work to first add all these other projects, but more realistically if we just wait some time for the ecosystem to develop, the dependencies will be available as packages, and perhaps a stable version will exist that doesn't require a beta release.
Thoughts? This isn't blocking anyone from developing or using rust - there are containers available along with the ability to hack around locally. My thinking is that the best strategy is to wait, and then come back when nu doesn't need a beta rust, and cargo-debstatus doesn't return (mostly missing) packages.
nu currently requires a beta release of rust that isn't in debian (I ran into this when I was trying to package - it said it was missing a default-run) it has pre-release versions of futures 0.3.0 in its dependency tree that Debian currently can't support with their tooling
As is noted in the e-mail, both issues are bound to resolve. 1.39.0 release will happen on Nov 7, 2019, so in precisely three weeks, while futures 0.3.0 will likely have a release around that time as well: https://github.com/rust-lang-nursery/futures-rs/issues/1893
Great! So @est31 should we put a TODO to take a look on November 8th?
Hi :)
How are we doing here? @vsoch @yarikoptic
nushell/nushell#900
@est31 said that a release will be in debian on November 7th, so I suggested looking on November 8th. It's still November 2nd @andrasio ! :)
Also please read - even with the rust release it may be there are too many dependencies not packaged yet to do. https://github.com/nushell/integrations/issues/2, and time might be the best asset here. Note that I'm not an experienced debian maintainer (I know pretty much nothing) but I'm feeling out as I go!
@est31 said that a release will be in debian on November 7th
When I said that there will be a release on November 7th, I meant the upstream release, not the release in Debian. Not sure when it'll be in Debian. Debian unstable still has 1.37.0 and 1.38.0 is needed in Debian before it can upgrade to 1.39.0.
Debian unstable now has 1.39.0: https://packages.debian.org/unstable/rustc.
okay I'm running the command again, the steps I took are:
cargo-debstatus debstatus
from the root of the repository, and the updated output is here:
There are still quite a few outdated - is this something we can take any kind of action on? @yarikoptic @sanxiyn and @est31 you seem to have expertise in debian packaging - your comments would be greatly appreciated!
Of course there are some possible actions. For example,
$ cargo debstatus -p subprocess
subprocess v0.1.18
├── crossbeam-utils v0.5.0 (outdated)
├── libc v0.2.66 (in debian)
└── winapi v0.3.8 (in debian)
shows subprocess needs to update crossbeam-utils dependency. Looking at subprocess repository, this is already done, just not released. When it is released, nu can update subprocess dependency, etc.
It is somewhat daunting to go through the long list, but I think there is no way around it. Good news is that it is finite.
@sanxiyn could you explicitly state what you are suggesting that we can do? To go through the list, one by one, check the output and then package.. and then what?
@yarikoptic @sanxiyn and @est31 I'd like to help as much as I am able - can we choose one of the libraries that isn't on debian, and together walk through how we would package? If I am able to help with a single external dependency, then I can slowly work through some of the others, and at least get us closer.
Hi @vsoch - has this progressed since? It's been almost 1 year and a half, hopefully by now the adoption of the Rust ecosystem within Debian is much better.
I'm not working on it actively, so no. I just tried to install cargo-debstatus to generate another tree to look at, but it no longer compiles on my system. Maybe someone else can give it a shot? I opened the issue to express the next but I am not a debian developer nor do I have the expertise with rust to be able to really take charge of it.
cargo-debstatus still builds on my system (rust 1.52.1 on Arch Linux) but I had trouble keeping up with cargo development and it has never been properly ported to the new Cargo.lock
format (https://github.com/kpcyrd/cargo-debstatus/issues/7), the codebase was mostly cargo-tree, cargo-outdated and some debian specfic code pasted into one codebase. I vaguely remember support for the new Cargo.lock
file has been added to the master branch, but has never been released because the "cargo-outdated" feature hasn't been ported yet and I'm currently not interested in actively developing the project anymore. Pull requests are still very welcome though!
Packaging the rust ecosystem in debian is quite labor-intensive (both getting the packages in and maintaining them afterwards) and there are requests by the debian ftp-master team that haven't been implemented due to lack of volunteer time. Uploads are being processed again though and there are still people working on rust packaging in #debian-rust on oftc. Merge requests can be sent to this repository https://salsa.debian.org/rust-team/debcargo-conf/, the readme contains instructions on how to do that (though, granted, there's no intro text but running the commands ./new-package.sh
and ./update.sh
as described in step 2 prints further instructions). You can always ask in the irc channel if you're stuck.
Debian is currently in freeze for the next release, if you're planning to do non-trivial uploads you might want to wait until debian bullseye has been released.
This issue is being marked stale because it has been open for 30 days without activity. If you feel that this is in error, please comment below and we will keep it marked as active.
If someone is interested in working on this, please let us know. We would be happy to coordinate to work on an official package.
This issue is being marked stale because it has been open for 90 days without activity. If you feel that this is in error, please comment below and we will keep it marked as active.
This issue has been marked stale for more than 10 days without activity. Closing this issue, but if you find that the issue is still valid, please reopen.
Should this be reopened? Just to advertise that help packaging is still wanted
I took a little time to play around with this here https://github.com/Inveracity/nushell-debian-package
I'd be happy to collaborate on this
@Inveracity Thanks for investigation. PRs are welcomed. and we can test it in nushell/nightly repo
If you get something completed, you should reach out to the Debian Rust maintainer group. They have picked up a few things recently, https://qa.debian.org/developer.php?email=pkg-rust-maintainers%40alioth-lists.debian.net
(I did not check if there is already an open issue, there might be already)
If you get something completed, you should reach out to the Debian Rust maintainer group. They have picked up a few things recently, https://qa.debian.org/developer.php?email=pkg-rust-maintainers%40alioth-lists.debian.net
(I did not check if there is already an open issue, there might be already)
I've focused my attention on packaging the prebuilt binary, is that still a good fit for the Debian Rust maintainers? From a quick glance It seems more related to crates rather than shipping binaries.
I've focused my attention on packaging the prebuilt binary, is that still a good fit for the Debian Rust maintainers? From a quick glance It seems more related to crates rather than shipping binaries.
It is definitely for shipping binary crates - but all dependencies also need to be in the repo so there are a lot more library crates than final binaries. (I believe dependency resolution is what makes the upstream packaging so difficult, since sometimes overrides need to be done by hand) . But binaries like ripgrep
, just
, b3sum
, bat
, alacritty
, bindgen
, fd
, and even rustc
and cargo
are all maintained by this team.
Config for the debian repositories happens via debcargo
, with configs at https://salsa.debian.org/rust-team/debcargo-conf. That readme has better instructions for how to package something intended for upstream, as opposed to just building a .deb
to self-distribute (which is, of course, still worth it!).
I see, thanks for the information!
For the time being my personal motivation is being able to install nushell with apt, either through a PPA hosted via nushell.sh or via package-cloud or otherwise.
I hope my PR is enough to at least get the wheels turning for an eventual apt install nushell
installation experience.
I will leave the crate packaging up to someone who is more experienced in the rust eco-system 😄
As already pointed out by @tgross35, all crates nu depends on need to be uploaded to Debian first, before you can upload a nu
package that builds from all this uploaded source code. Downloading sourcecode from crates.io during build (or uploading pre-built binaries) is not allowed and can't be used for official Debian packages.
To get a status report you can run cargo-debstatus on nu and get something like this (I can not post the full output because it's about 2 MB of text and I could not find a pastebin that allows me to paste this much text):
Some crates that do not have (in debian)
behind them can be patched (for example, Debian sid has nix 0.26 instead of nix 0.27, but the crates that depend on 0.27 can likely be patched downstream to work with 0.26 until 0.27 is ready).
Nu is in itself split in many small crates, merging this back into fewer crates (like tokio did) would help with packaging. The amount of crates nu itself is split into is also the reason the cargo debstatus
report is 2 MB big.
If you're interested in helping out you can pick any crate from this tree that has all its dependencies already marked with (in debian)
and send a merge request to https://salsa.debian.org/rust-team/debcargo-conf. The work required is mostly filling out the metadata paperwork needed for Debian and if you're stuck you're very welcome to ask in #debian-rust on otfc irc.
maybe this issue can be moved to https://github.com/nushell/integrations?
I reached out to @yarikoptic about proper Debian packaging (I've never done it!) and he has shared his wisdom! Here is his proposal for going about this:
This doesn't really afford a complete GitHub or general CI release cycle, and for this reason we can definitely provide the Docker build, but maybe we should also try to do this proper? Thoughts?