`make native-deb` should be removed, since it's never worked in a 2.2 release tarball

[rincebrain]

Attempting to make native-deb or any of the children of it, including make native-deb-utils, will not work in any released tarball, because the tarballs are prepared incorrectly and missing most of contrib/debian/.

They will error out with

config.status: creating contrib/debian/rules
config.status: creating Makefile
config.status: creating include/Makefile
config.status: creating lib/libzfs/libzfs.pc
config.status: creating lib/libzfs_core/libzfs_core.pc
config.status: creating lib/libzfsbootenv/libzfsbootenv.pc
config.status: creating module/Kbuild
config.status: creating module/Makefile
config.status: creating rpm/generic/zfs-dkms.spec
config.status: creating rpm/generic/zfs-kmod.spec
config.status: creating rpm/generic/zfs.spec
config.status: creating rpm/redhat/zfs-dkms.spec
config.status: creating rpm/redhat/zfs-kmod.spec
config.status: creating rpm/redhat/zfs.spec
config.status: creating tests/zfs-tests/tests/Makefile
config.status: creating zfs.release
config.status: creating zfs_config.h
config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing po-directories commands
cp -r contrib/debian debian; chmod +x debian/rules;
cp contrib/debian/control debian/control; \
dpkg-buildpackage -b -rfakeroot -us -uc;
cp: cannot stat 'contrib/debian/control': No such file or directory
dpkg-buildpackage: error: cannot open file debian/changelog: No such file or directory
make: *** [Makefile:14290: native-deb-utils] Error 255

On any of 2.2.0 through 2.2.6.

[Gendra13]

Wouldn't it make more sense to fix the tarball instead of removing the option? Because if you just do a git clone from master or the 2.2-release branch instead of using the tarballs, make native-deb-utils is working perfecly.

[rincebrain]

You would think so.

But since it's been broken for a year, and it's been reported several times to the people generating releases without any change, I've given up on that happening.

[wmmur]

@rincebrain interesting? do you want to destroy something that works (make native-deb-utils) and leave make deb that doesn't work? #16478

[rincebrain]

I'd rather we stopped claiming that building packages worked if we don't test it and don't fix it when it's broken.

I'm tired of having to explain to people for a year that it's known to be broken.

Honestly, I'd remove the alien'd deb target while we're at it, because those would require a lot of patches to alien to be able to actually work reliably.

[wmmur]

@rincebrain I use make native-deb-utils compilation. tested for >=zfs-2.2.4, make deb does not work. I'm sorry you've had such a negative experience.

[rincebrain]

You should probably read the report before commenting any more.

The entire point is that the release tarballs do not work, for any version. I tried this on all of the release tarballs from 2.2.0 to 2.2.6. They do not contain most of the files needed for this to ever work, on any release, ever.

This has been reported, and not fixed, just like "make -j native-deb" has been broken since it was merged, and not fixed, other than saying "don't do that".

If we're not going to fix it, we should stop pretending to support it.

[robn]

All other things being equal, I agree that nothing is better than a broken thing. However, the fact that there is a broken thing is at least a statement of intent, so if its still wanted and not hard to fix, then I'm more on the side of fixing it, as long as that happens.

It sounds like its more of a symptom, than a cause, and that's more about tarballs being broken. I've seen that myself on the FreeBSD side (in a customer project where git wasn't readily available); that one was easily fixed though.

@tonyhutter with your release manager hat on, how "working" do you expect the tarballs to be? Would you prefer that broken options like this are fixed for all cases, removed for all cases, or gated off to tarball builds? Or something else?

[nabijaczleweli]

Given opinion from 2023-04 and opinion on original PR from 2022-05, shall I say... vindicated?

[nabijaczleweli]

Someone should probably do a more thorough review of the tracker but #15638 and #16289 indicate, to me, that the package also doesn't work from a checkout.

(Also this is a dupe of #15404.)

[rnickle]

Someone should probably do a more thorough review of the tracker but #15638 and #16289 indicate, to me, that the package also doesn't work from a checkout.

(Also this is a dupe of #15404.)

I agree that there are issues with building from a tar-ball, but I have been successful in building native-deb, native-deb-kmod and native-deb-utils from a checkout on Ubuntu 24.04 and Debian 12.

[nabijaczleweli]

just building, or actually using? #15638 at least certainly indicates to me that it builds, but "actually using" it doesn't really happen

let's see what the original PR says should happen here:

Proper Debian style packaging can:

1. Make things easier for developers to have an actively managed packaging for Debian based systems.
2. Ease the testing and development of ZFS packages on Debian based systems.
3. It may also serve as a starting point for downstream projects to base their packaging on and add their own customization.
4. Current .deb packages do not include debug symbols and information to facilitate debugging process.

1 seems farcical, 2 has certainly not come to pass (and is nonsensical on its face, to me), 3 is weird given that there's no reason to... use this packaging? instead of the one from debian? if you're building debian (debian derivative) packages?, 4 is an outright lie for real Debian packages, and just fix the alien ones if they don't have dbgsyms? lol

the original goals seem, to me, clearly unmet, this has increased support load, it doesn't work at all in any released version, and barely? works? or is subtly broken? on trunk

[nabijaczleweli]

If you don't consider "support load increased" self-evident, a cursory glance says #14736 #15404 #15586 #15638 #16289 are all "contrib/debian is broken" AFAICT.

[rnickle]

If you don't consider "support load increased" self-evident, a cursory glance says #14736 #15404 #15586 #15638 #16289 are all "contrib/debian is broken" AFAICT.

Thanks for summarizing the issues.

[behlendorf]

Thanks for fully cataloging the issues and opening a PR to get feedback. I agree, this hasn't exactly been as trouble free as I was originally hoping. However, before we go ahead and remove this functionality we should do an honest assessment of what it will take to fix the open issues and properly maintain it. @usaleem-ix can you investigate the open issues.

[nabijaczleweli]

(ftr this is just what i got from searching "native deb" and "contrib/debian" and a cursory ^F inside, so i don't expect this to actually be exhaustive)

[m-ueberall]

I agree that there are issues with building from a tar-ball, but I have been successful in building native-deb, native-deb-kmod and native-deb-utils from a checkout on Ubuntu 24.04 and Debian 12.

just building, or actually using? #15638 at least certainly indicates to me that it builds, but "actually using" it doesn't really happen

FWIW, I switched to make native-deb a while ago on Ubuntu 20.04/22.04 (arm64/amd64, 2<n<10 machines) and everything works as expected – including the automatic dkms-based rebuilds after a kernel upgrade – once you reboot from a rescue image once and thoroughly replace all zfs* packages by their openzfs-* "counterparts".

[usaleem-ix]

@usaleem-ix can you investigate the open issues.

I am working on that.

[wmmur]

I use make native-deb on Devuan Ceres and Debian Sid. compilation works, installation of openzfs-* packages, including dkms compilation of kernel modules. Edit However, I understand that it is "support load increased".

[usaleem-ix]

@rincebrain it is really sad and disappointing coming all this from you, since you were the very first person who advocated for adding native Debian packages on OpenZFS channel on Slack, when it was discussed over there before any of the work was done on this topic.

You have pushed for native Debian packages and advocated for it, once more, when the PR was closed after it faced severe backlash from downstream Debian maintainers.

Let me be very clear here:

other than saying "don't do that".

These are your words, not mine. I have never said that. If you care to read, I have given you ample reasons why I don't "think" this is a good idea. If something is not implemented the way you want it to be, that is a difference of opinion. If you really think that should be the way, work on it and post a PR for review and see what community and maintainers think about. Since it is evident you cannot understand and handle other's opinion, let me re-iterate what you said couple days ago, on a completely different issue, but I think it is more fitting here:

It's nobody's responsibility to fix it.

There's not like, assigned areas of the project where it's so-and-so's job to fix this if it breaks, and the original company that contributed the project A) doesn't seem to hit these and B) appears to have stopped contributing some time ago.

The rest of the companies that use OpenZFS, to my knowledge, mostly don't use native encryption, so that leaves random volunteers or any exceptions, to fix these.

That's not intended as an indictment of anyone involved, it's just a statement of "nobody's responsible for making sure it gets done, so it doesn't get done if nobody is responsible for it and not enough people are in the set of {able, motivated by encountering it} to have it happen organically"

As far as broken release tarballs are concerned. There was no clear standing on how release tarballs are prepared. People have asked about it on the issue from maintainers. There was no response. Moreover, as mentioned over here previously, it is clearly stated that out of 3 different file formats/release tarballs, only one is broken. Other two work, git checkout also works. Other people have also mentioned this time and again.

There have been efforts to fix other issues like fixing the DKMS package. The intent to fix issue/s is clearly evident there, unlike here.

[usaleem-ix]

To the person who once said "Don't @ me here again". I guess the nightmares must have kept you up at nights ;)

Your intentions have been pretty clear from day one. And since you have now become self-proclaimed judge, jury and executioner on this, I don't need to argue with you about usefulness of this or anything that you have said above. Multiple community members have come forward and described how they use it in their workflow, without hitting open issues.

However, to set the record straight, and to clear accusations have been made above. Let me clarify, we at iXsystems, often merge upstream ZFS sources very often. There have been scenarios where upstream release has not been finalized yet, but we have had to merge whichever state the release is in, with pending patches merged afterwards. Furthermore, the current master branch of ZFS (to be 2.3) is already included in TrueNAS SCALE 24.10-RC1 release. The reason being that we want to include new ZFS features and fixes sooner rather than later in our product, test them early, fix the issues and release them in the wild.

Downstream Debian release cycle is not sufficient. For instance, upstream zfs-2.2.6 was released on August 22nd, but downstream Debian changelog says it was merged on September 5th, after 15 days. Make no mistake, this is not a "complain" of any sorts.

Since the day native Debian packaging is merged in upstream, it is being used actively in our production, without any issues. TrueNAS SCALE is shipped with native Debian packages.

And effectively what is being called a lie:

4 is an outright lie for real Debian packages, and just fix the alien ones if they don't have dbgsyms? lol

The Debian packages that were being talked about in the original statement were not downstream Debian ones. The statement was about upstream Debian packages, which at the time were only being generated with Alien, without debug symbols. Let me repeat, there were no debug symbols in upstream Alien generated Debian packages at the time. That issue was also fixed by me here.

Who in their right mind would ever need debug symbols for upstream Debian packages, right? This is certainly not useful for anyone building upstream ZFS packages.

And for the packaging bits being broken and support load increased, these native packages are being tested by upstream CI and as mentioned above, being used in TrueNAS SCALE production without any of these issues being hit. If there were issues found in the main workflow, those would have been fixed. The fix for the issues reported were pretty straight forward and anybody with the intention to "fix" it would have easily done it.

All factually incorrect statements with zero understanding and intention to fix issues, have been made by @nabijaczleweli and maybe @rincebrain . I rest my case.

[rincebrain]

Umer, you might want to reconsider posting a two page personal attack screed.

I understand that you're upset, but I wasn't trying to attack you with this, personally or professionally. I was reporting that this is still broken, and I kept encountering people reporting it broken every release, so we should stop claiming it works, if we're not going to fix it. I don't have the time to fix everything I encounter, which is why I report bugs, so that it's more discoverable by other people who encounter it, and can hopefully build on it until it's fixed.

I'm sorry that you think I was misstating your remarks, but that was not deliberate, and I don't think "make -j shouldn't break the build" is just a difference of opinion.

[behlendorf]

I'd prefer to try and work through the reported issues before we consider dropping this support. A first round of fixes was merged in #16604 which we'll back port for 2.2.7.