/etc/os-release info - Githubissues

bennuttall commented 4 years ago

Do you plan to update the OS name in /etc/os-release? There will be scripts out there expecting raspbian and importantly distinguishing it from debian so if there's a new name it should be reflected here.

piwheels logs downloads and we're able to show trends of OS usage. As of today, downloads form "raspbian" and "debian" are being merged. It would be better if we could distinguish "raspbian" (pre-today images) from "pios" (post-today images) and "debian" (actual debian).

ghollingworth commented 4 years ago

In the image name we're using raspios... Currently it's raspbian because it's another oversight!

I think raspios would be a good replacement for the shortened version

XECDesign commented 4 years ago

I wouldn't update /etc/os-release. It tells you what the underlying OS is.

MichaIng commented 4 years ago

The main repository is the standard Debian, from where the base-files is pulled which ships /etc/os-release and hence shows you correctly that it is indeed Debian. Not sure why it became common across manufacturers to give ones Debian-image a different name like Rabian (Raxda), TinkerOS (ASUS), now RaspiOS and whatnot instead of just saying what it is? Raspbian is the only one that actually deserves (😛) it as it is truly an own main distro repository 😉.

I don't know about future plans, but I don't see any reason to not use standard Debian and again go with the overhead to clone-compile a complete new distro that has nearly no difference to Debian. Remember that Raspbian was necessary due to this special armv6hf that RPi1 (+Zero) is, which is not Debian-compatible. For x64-capable models (and the only real ARMv7 model: RPi2 PCB1.1) Debian works pretty fine. All RPi-specific packages and builds (kernel/firmware/GPU-accelerated media players/special-licensed software) can and AFAIK are already shipped via archive.raspberrypi.org apart from the main repo.

XECDesign commented 4 years ago

We couldn't use raspbian because we'd end up with "An operating system for raspberry pi built on top of Rasbian with out desktop, patches and optimisations on top" and "An operating system for raspberry pi built on top of Debian arm64 with out desktop, patches and optimisations on top" and so on. Much easier to trim all of that down to Raspberry Pi OS and specify the architecture.

MichaIng commented 4 years ago

"Debian image", "Debian image with Raspberry Pi desktop" etc would have been the alternative. Ah I see now the Raspbian images have been renamed as well, so the name "Raspbian" is history? I hope the contained /etc/os-release will keep it at least for some time as this is often used for identification 😉.

Hmm not sure now Raspberry Pi OS (32 bit) and Raspberry Pi OS (64-bit) will be the future names. "Raspbian (32 bit)" and "Debian (64 bit)" would have made it much clearer (IMO, to clarify that its two different main APT repos) and probably avoided confusion. However we'll adapt and the step to finally offer an official x64 image is absolutely awesome in any way 👍.

XECDesign commented 4 years ago

Raspbian itself is a volunteer project (raspbian.org) and still remains Raspbian. We're just not referring to the images we build as Raspbian. /etc/os-release will stay the same because it makes sense.

bennuttall commented 4 years ago

How do you tell apart Raspberry Pi OS from Debian, in say the user agent?

XECDesign commented 4 years ago

Some ideas:

Check for /etc/rpi-issue or /boot/issue.txt, which will tell you the image was built with pi-gen.
Check available repos. If archive.raspberrypi.org is there, it's probably one of our images.
Is raspberrypi-kernel installed? Debian is likely to use a different package name.

Nothing is really going to be 100% accurate.

Out of curiosity, what are the two branches of that decision? Why would you need to do something different based on Debian vs Raspberry Pi OS?

It's just Debian with an extra repo on top. The name is just shorthand and, at least in theory, shouldn't make a difference to how applications choose to behave.

I feel like whatever that difference you're looking for is something other than what we happen to call a particular collection of software, but I have no idea what this is for, so I'm probably wrong.

bennuttall commented 4 years ago

Two things: scripts like this one which prepares the piwheels builders check for the OS name. In this case we're not differentiating between Debian and Raspbian, so it's not a problem here, but I'll have to change it (assuming Mythic beasts images follow suit) and I'm just saying there will be other examples where maybe that goes unnoticed?

Second thing is the piwheels stats: when a pip request hits piwheels.org, we log the user agent and keep track of download stats, including OS/distro/version. Previously you could compare Raspbian usage with Debian, now the PiOS usage is merged with Debian. It would be nice to be able to distinguish PiOS users.

https://blog.piwheels.org/piwheels-stats-for-2019/

XECDesign commented 4 years ago

Since we already patch pip, we could add something to set PIP_USER_AGENT_USER_DATA or similar. Is there something that could be added to the conf file where we tell it to use piwheels?

MichaIng commented 4 years ago

@bennuttall

It would be nice to be able to distinguish PiOS users.

Of course one could put some effort into this, but I do not get the point why this is especially relevant for RaspiOS? You also do not distinguish between TinkerOS users and Debian, or "Rabian" users and Debian or DietPi RPi users and Raspbian, or FriendlyCore and Ubuntu, since each pair uses the same main repository and only put different board-specific extras on top. Not sure why one would make a special case for Raspberry Pi OS (64 bit) here, because of expected quantity? A different topic for statistical interest would be to distinguish between hardware models, but that should not depend on /etc/os-release but e.g. /proc/cpuinfo or such, probably in combination with uname -m (or dpkg --print-architecture) architecture.

Two things: scripts like this one which prepares the piwheels builders check for the OS name. In this case we're not differentiating between Debian and Raspbian

Since RaspiOS (64 bit) IS Debian (+ some extras like every other SBC image), AFAIK doing some checks to override it with "it is Raspbian" would break the script? I mean if you check the OS name/ID, all that is relevant should be the package manager and expected available packages + their versions (especially library versions). And then it is perfectly correct to use "Debian" as derived from /etc/os-release, isn't it? For the "real" Raspbian, now RPiOS (32 bit) it is relevant since it is an own repo with own special architecture (armv6hf) and probably differences in available packages and library versions.

bennuttall commented 4 years ago

Looking at man os-release I'd say that /etc/os-release should be updated. I think it should have ID as something unique like rpios, ID_LIKE as debian or debian raspbian and PRETTY_NAME something like Raspberry Pi OS GNU/Linux 10 (buster). Also would be a chance to set HOME_URL, SUPPORT_URL and BUG_REPORT_URL to something that goes to RPTL not Debian or Raspbian.

XECDesign commented 4 years ago

Imagine if every single script that uses os-release had to have a case for each single debian and ubuntu derivative. What you'd see is a lot of "Sorry, this script doesn't support your OS", as you already sometimes do with Raspbian.

It doesn't make sense for all bugs to be reported to us when 99.99% of packages aren't ours.

I'll go with no on this one.

MichaIng commented 4 years ago

This would require an own base-files package and would break many scripts that expect "ID=debian" (and in case "ID=raspbian") even that they could run perfectly fine.

I guess it would cause much less issues the other way round: If there is anything specific found broken because Raspberry Pi OS (64 bit) has "ID=debian" in its /etc/os-release, those cases can be handled individually.

waveform80 commented 4 years ago

I wouldn't update /etc/os-release. It tells you what the underlying OS is.

No, it tells you what this OS is, and optionally what it derives from (in ID_LIKE); "man os-release" is fairly clear about that, as was its release announcement. Now, whether Raspi OS is Debian or not is another question and not one I can answer directly (but obviously I'll stick my nose in later on... :).

Imagine if every single script that uses os-release had to have a case for each single debian and ubuntu derivative. What you'd see is a lot of "Sorry, this script doesn't support your OS", as you already sometimes do with Raspbian.

I'm probably missing something, but I'm not understanding the logic here. Most scripts that I'm aware of that use /etc/os-release usually use it for reporting or labelling purposes. For example, pip (via a vendorized version of the distro library), incorporates portions of /etc/os-release into its user-agent. It doesn't really care what the id is, it just wants an identifier for the label. Likewise for GUI software installation applications that can then fire up and present "Your Distro Here Software Center" instead of "Debian Software Center".

When scripts do discriminate, they can check ID in os-release and, should they find it something they don't recognize (say, "raspios"), they can fallback to ID_LIKE which should usually provide a reasonably well-known distro. In this case it'd be perfectly reasonable for ID_LIKE to be "debian". That said, sometimes when scripts discriminate, it's often more complex anyway.

On Ben's original point: people serving stuff to pip (or insert-your-script-here), may well care about determining whether their users are coming from debian / raspbian / raspios / etc. In the case of piwheels it matters quite a bit what packages our wheels are built on top of (there is a certain amount of divergence between debian and raspbian that matters in this area; I'd anticipate something similar in the case of raspios).

Then, there's Debian's own guidelines on branding. Does Raspberry Pi OS diverge that far? That's not my call to make, but to my eye it certainly presents itself as something that isn't Debian (the boot screen, the desktop appearance, the pi software center, etc. etc.)

XECDesign commented 4 years ago

I'm probably missing something, but I'm not understanding the logic here. Most scripts that I'm aware of that use /etc/os-release usually use it for reporting or labelling purposes.

I've had to modify scripts many times to add 'Raspbian' to a case statement because it would error out when it encountered something it wasn't expecting. You can search github for "/etc/os-release" and "unsupported" and you'll see plenty examples of scripts that would stop working.

I understand all the logic and I've read all those links.

To avoid an argument, I'll bow out of this one and leave it up to the powers that be.

MichaIng commented 4 years ago

Most scripts that I'm aware of that use /etc/os-release usually use it for reporting or labelling purposes.

Just two installer examples which would be broken, and such use of /etc/os-release is pretty common:

(EDIT: Whoopsie, parallel post, so basically a prove of what has been said above 😉)

As can be seen there, Raspbian is the only addition to the main distributions, Debian and Ubuntu. No other SBC manufacturer changes this, even that they give their images as well different OS names on their download page etc. Why should Raspberry Pi do and create a special case with extra effort for installer coders?

should they find it something they don't recognize (say, "raspios"), they can fallback to ID_LIKE

Tell them, they will be happy to be loaded with extra work from a single manufacturer 😉.

may well care about determining whether their users are coming from debian / raspbian / raspios / etc.

Nice to know indeed, but not for the cost of breaking certain installers and creating extra load to their devs. And as said above, no-one is or was ever complaining this for TinkerOS or all the others, why now here?

In the case of piwheels it matters quite a bit what packages our wheels are built on top of (there is a certain amount of divergence between debian and raspbian that matters in this area;

And this is what hits the nail on the head: Yes there is divergence between Debian and Raspbian, because Raspbian is an own main repository with own architecture, which is the reason it ships an own /etc/os-release. But there is no difference between Debian and "Raspberry Pi OS (64 bits)" in this regards. The available packages are the same as it's the same repository. So it is perfectly fine and correct to handle it the same. So when piwheels build tools read "Debian", simply proceed as with every other Debian + based on arch of course, and you are fine, as well for "Raspberry Pi OS (64 bits)". Doing it different would probably create incompatible wheels. When you read "Raspbian" go on as before since nothing will change with it and the need for RPi build tools for armv6hf.

bennuttall commented 4 years ago

And as said above, no-one is or was ever complaining this for TinkerOS or all the others, why now here?

I'd rather Raspberry Pi followed examples of good quality distro maintenance standards, like Ubuntu flavours, than look to Tinkerboard and other Pi clones for inspiration.

MichaIng commented 4 years ago

I'd rather Raspberry Pi followed examples of good quality distro maintenance standards, like Ubuntu flavours, than look to Tinkerboard and other Pi clones for inspiration.

It is simply a fundamental difference if you ship an own main repository or use the official Debian repo. But yeah I made my point, finally it's up to the RPi devs to decide what is most resonable.

bennuttall commented 4 years ago

Raspberry Pi does ship its own main repository: https://archive.raspberrypi.org/debian/dists/buster/main/

MichaIng commented 4 years ago

This is the kernel/bootloader/firmware repo with a few special software packages that are hardware accelerated by the RPi or have a special license (RealVPN). Such repo ist shipped by most other SBC manufacturers as well, most importantly to ship kernel+dtb+bootloader packages.

I don't mean the "main" component that most repos have, with "main" repo I mean the one in /etc/apt/sources.list (on apt-based distros) that contain all the GNU/Linux libraries, utils and end user software. Probably wrong wording from my side, but I hope you get what I mean.

Compare the package.gz size, to get an idea what defines the distro: https://archive.raspberrypi.org/debian/dists/buster/main/binary-armhf/ https://deb.debian.org/debian/dists/buster/main/binary-armhf/

ghollingworth commented 4 years ago

Having looked into this in detail there is no reason to change the os-release contents at the moment, it's a difficult decision because it would cause us immediate problems with creating the build, having to go through many scripts to fix-up this support.

Ubuntu is different to Raspberry Pi OS, they do host everything themselves and therefore having ID as ubuntu is fair enough.

It seems that it should be possible to change ID and set a ID_LIKE to debian because the man os-release says scripts should drop back to ID_LIKE if they don't see anything they recognise. But in fact this is just not the case, most scripts do not fallback to ID_LIKE. Therefore, this will just break more things.

I think for this Ben, Serge has offered to patch pip to change it to provide something you can use instead. I think that'll have to suffice here

bennuttall commented 4 years ago

Thanks. I don't think it's worth patching pip, because people commonly update to a newer pip than the one available in apt (pip3 install pip --upgrade installs a newer version into /usr/local/) and that wouldn't maintain any patch applied in the apt version. The pip config file currently added by Serge is maintained, so that works either way.

In future, piwheels will just have to report Raspbian/Debian usage stats combined.

xnox commented 4 years ago

https://www.debian.org/derivatives/

Derivatives can use parts of Debian's infrastructure if needed (like repositories). Derivatives should change references to Debian (like the logo, name, etc.) and to Debian services (like the website and BTS).

https://wiki.debian.org/Derivatives/Guidelines#De-.2FRe-branding

Depending on the degree of divergence from the vanilla Debian, it might be necessary to introduce non-functional modifications in the deployed system to eliminate user confusion of the derivative distribution with vanilla Debian

base-files ... /etc/os-release

Clearly Raspberry Pi OS is not vanilla Debian, and thus it should customise base-files. This does not preclude from reusing Debian infrastructure and/or other packages. All as per Debian Derivatives guidelines.

You can seek more help and guidance from the Debian Derivatives Front Desk https://wiki.debian.org/DerivativesFrontDesk that have email, irc, mailing list contacts who can help with making Debian Derivative.

MichaIng commented 4 years ago

IMO "Raspberry Pi OS (64 bit)" is not really a derivative. It is Debian with an additional hardware-specific repository to ship kernel and firmware and such, so that Debian can run on a 64 bit capable RPi models. That is also the reason why I would not have given it an own name.

Also one comes into the complicated situation that "Raspberry Pi OS (32 bit)" indeed is an own derivative with a complete own major repository and own architecture, incompatible with Debian, which is why it ships correctly a custom /etc/os-release and name "Raspbian". Now the name "Raspberry Pi OS" mixes Raspbian (for 32 bit) and Debian (for 64 bit). This might be easier/cleaner for new users, which simply want to download the officially provided RPi OS image, but when going into detail it causes the confusion that one according to /etc/os-release and MOTD (login prompt) is named "Debian" while the other is named "Raspbian". And while for the first it makes sense to report bugs with packages to the Debian bug tracker, for the second it makes sense to first have the Raspbian maintainers a look at, in case it is a Raspbian-only package issue.

Clearly Raspberry Pi OS is not vanilla Debian

What "vanilla" Debian is should then be defined. There are hundreds of different ways and images and installers to start with Debian, each with multiple options again, resulting in significantly different setups, regarding both, the degree of preinstalled software and the deeper system setup (partitioning, network, kernel, bootloader, ...).

Please have a look at the list of "highlighted" derivatives at their wiki. These use all their very own major repository (meaning the repo in /etc/apt/sources.list where all core system packages, libraries GNU tools etc are coming from), with in case fundamental differences, like PureOS being a rolling release distro, and Ubuntu of course. The full list contains as well derivatives which are much closer to Debian, like Armbian, but also Armbian does not ship a custom /etc/os-release, but instead add an own /etc/armbian-release on which one can identify it, if interested.

I'm btw just seeing things from the software developers perspective (when aiming to ship/install pre-compiled binaries, place init scripts and such), where it is easier to identify and handle the underlying OS the same. When /etc/os-release says "Debian", and "Buster", we know exactly which packages are available, which library versions, what and how we need to compile, e.g. automated builds on a Debian Buster VM, etc. If it says "ID=RaspiOS" and "ID_LIKE=Debian", we cannot be sure

if we can trust the Debian repository being in place or not
if it is the armv6hf architecture for 32-bit RPi models or regular arm(v7)hf
if we need some workarounds or can support the OS at all
if we can place 3rd party repos for Debian or need to use the one for Raspbian or if they would cause package dependency havoc
if we can build on a Debian VM or if we need a new build system for this, which can become crazy if this would be required for every single SBC manufacturer...

The major distribution package repositories really gives us most relevant information we need to ship/install software correctly, place the correct pre-compiled binary, writing clear guides/instructions for users etc. And the distro name should clearly identify which major repository can be expected. It is not about which packages are installed exactly and how they are configured or customised, such can never be trusted, it is about which packages are available, how new ones can be installed, which binary architecture is used etc. And from that perspective, "Raspberry Pi OS (64 bit)" is Debian 🙂.

xnox commented 4 years ago

Custom kernel, not authored by Debian kernel team, is not vanilla. Or the fact that it doesn't work with Debian's kernel.

Which packages are available, is subject to which repositories are enabled during the build, and have no correlation to /etc/os-release file whats-so-ever.

And clearly unless all repositories for Raspberry Pi OS are available (ie. certain suite of debian + raspbian os suites), one cannot build packages targeting Raspberry Pi OS correctly. For example, custom prebuilt binary dkms .deb packages.

This has nothing to do with how any other derivative does things, each derivative is unique and make their own choices. It's all to do with what Raspberry Pi OS wants to do. The most correct way is to be explicit, rather than implicit.

All of debian packages build in chroots, and have no connection to underlying OS. I.e. one can use mk-sbuild / debootstrap on fedora and build correct packages for Debian, Ubuntu, Raspberry Pi OS, etc. And setting up correct build-environment doesn't depend on values in /etc/os-release. Nor does your VM has to run any particular flavour or release of Debian, if you choose to build inside the VM. And every single SBC manufacturer uses the same build-VMs for building rpms/debs targetting pleora of different distributions.

xnox commented 4 years ago

And from that perspective, "Raspberry Pi OS (64 bit)" is Debian .

That's quite a stretch of imagination. As to call oneself that, one needs to be a pure Debian blend and build out of the Debian Archive only.

MichaIng commented 4 years ago

Custom kernel, not authored by Debian kernel team, is not vanilla. Or the fact that it doesn't work with Debian's kernel.

If that is the reason to ship a custom /etc/os-release and custom name, then Debian is a x86 distro only, since it cannot boot itself on any ARM device. I know there is a generic ARM kernel, but that does either not boot at all or only with very limited hardware features. Custom bootloader + kernel + dtb is practically required for ARMs/SBCs.

Which packages are available, is subject to which repositories are enabled during the build, and have no correlation to /etc/os-release file whats-so-ever.

For practical reasons it is pretty common to derive the enabled repos from /etc/os-release. Since most repos have mirrors with by times very custom URLs, it is impossible to scan /etc/apt/sources.list for this purpose. Hmm the repo keys could be used indeed, however then one does not know if the Debian repo has been only added additionally (e.g. to provide backports on Raspbian for ARMv7+ RPi models) or is really the major repo.

And clearly unless all repositories for Raspberry Pi OS are available (ie. certain suite of debian + raspbian os suites), one cannot build packages targeting Raspberry Pi OS correctly. For example, custom prebuilt binary dkms .deb packages.

Taking aside kernel modules and such things, that is exactly the point. Raspbian + Debian repo mix in general does not work anyway, aside for single specific frontend packages that you want from Debian backports, if your model is RPi2+ (ARMv7+), e.g. So practical is:

grep -q '^ID=debian' /etc/os-release => Debian build stack (based on dpkg arch)
grep -q '^ID=raspbian' /etc/os-release => Raspbian build stack (always armv6hf)

Currently this is working 99,99% failsafe, so creating the need to further checks should have a very good reason.

This has nothing to do with how any other derivative does things, each derivative is unique and make their own choices.

Yes of course, however following common practice is also an argument, i.e. handling things like all other SBC manufacturers do. It is not a killer argument but it is always one to not create too much special-cases. If you create an own derivative with own ideas, market model, philosophy, target user space, there is certain reason so do own branding, including /etc/os-release. However RPi foundation is a SBC manufacturer which wants to sell it's SBCs and simply needs a functional OS to easily get it running. If an existing OS, after adding required hardware-specific packages, does, there is likely little motivation to start creating an own brand around it. For RPi1 it was as mentioned only required due to the special Debian-incompatible ARM arch, and Raspbian is a volunteer project, just to mention.

All of debian packages build in chroots, and have no connection to underlying OS. I.e. one can use mk-sbuild / debootstrap on fedora and build correct packages for Debian, Ubuntu, Raspberry Pi OS

Ah yes, the question is hence which archive to load with debootstrap, which architecture, build stack etc. Finally you have certain binaries and only need a single arm64 build for stock Debian and Raspberry Pi OS (64 bit) (so no need to differentiate here) while you need a different ones for Debian armhf and Raspbian armhf (so there is a way required to differentiate), and the question simply is how one can derive the required binary correctly without a too large coding effort.

However depending on which part one highlights, there are no strict definitions and practices on this topic but things are fluent and vary with the perspective. So such discussion can never come to an end. The important question is whom/where/how a custom /etc/os-release helps and whom/where/how it harms (including additional effort in any way). Currently the only practical argument mentioned was that piwheels uses the /etc/os-release ID for statistics and now the 64 bit RPi OS will be mixed with Debian. Against it stands the need to update likely hundreds of installers/setup scripts to support the 64 bit RPi OS when using a custom /etc/os-release ID (Regardless if using the ID in installers is good practice or not, it is in fact used, with success!).

xnox commented 4 years ago

Custom kernel, not authored by Debian kernel team, is not vanilla. Or the fact that it doesn't work with Debian's kernel.

If that is the reason to ship a custom /etc/os-release and custom name, then Debian is a x86 distro only, since it cannot boot itself on any ARM device. I know there is a generic ARM kernel, but that does either not boot at all or only with very limited hardware features. Custom bootloader + kernel + dtb is practically required for ARMs/SBCs.

Debian ships hundreds of dtbs & flash-kernel and u-boot builds to boot the stock Debian installer on hundreds of different boards both armhf and arm64.

See for example http://ftp.debian.org/debian/dists/sid/main/installer-armhf/current/images/device-tree/ and http://ftp.debian.org/debian/dists/sid/main/installer-armhf/current/images/u-boot/

debian-arm porting team has many contributors from many vendors, ARM, and Linaro to enable as many boards as possible. They are happy to take patches / dtbs / firmware builds to enable more boards too!

Which packages are available, is subject to which repositories are enabled during the build, and have no correlation to /etc/os-release file whats-so-ever.

For practical reasons it is pretty common to derive the enabled repos from /etc/os-release.

That is just not true. debootstrap & sbuild defaults to deriving packaging list from the requested suite name & variant. Plus extra repositories. Which /etc/os-release is on the host, or in the target chroot is irrelevant. None of that is based on anything in /etc/os-release. I am curious how you setup your repositories, cause it sounds like the way things are done are fragile and unlike how everyone else builds .deb packages. Because nobody relies on os-release file for that.

And clearly unless all repositories for Raspberry Pi OS are available (ie. certain suite of debian + raspbian os suites), one cannot build packages targeting Raspberry Pi OS correctly. For example, custom prebuilt binary dkms .deb packages.

Normally, this is achieved by having a custom suite name in debootstrap, and contributing that to upstream debootstrap project. That way, anybody who has access to debootstrap package on any host, can setup a correct chroot with correct source / binary archives. None of that uses os-release as input.

Taking aside kernel modules and such things, that is exactly the point. Raspbian + Debian repo mix in general does not work anyway, aside for single specific frontend packages that you want from Debian backports, if your model is RPi2+ (ARMv7+), e.g. So practical is:

grep -q '^ID=debian' /etc/os-release => Debian build stack (based on dpkg arch)

grep -q '^ID=raspbian' /etc/os-release => Raspbian build stack (always armv6hf)

Currently this is working 99,99% failsafe, so creating the need to further checks should have a very good reason.

/etc/os-release is primary a user facing file, along with documentation. It's not meant to be used to decide how to build packages, which repos to use etc. It's meant as a universal file to identify a particular distribution and point of contact for bug reports, support, and community.

Along with os-release file, other base-files should be customize to advertise Raspberry Pi OS resources.

This has nothing to do with how any other derivative does things, each derivative is unique and make their own choices.

Yes of course, however following common practice is also an argument, i.e. handling things like all other SBC manufacturers do. It is not a killer argument but it is always one to not create too much special-cases. If you create an own derivative with own ideas, market model, philosophy, target user space, there is certain reason so do own branding, including /etc/os-release. However RPi foundation is a SBC manufacturer which wants to sell it's SBCs and simply needs a functional OS to easily get it running. If an existing OS, after adding required hardware-specific packages, does, there is likely little motivation to start creating an own brand around it. For RPi1 it was as mentioned only required due to the special Debian-incompatible ARM arch, and Raspbian is a volunteer project, just to mention.

For the ARM64 port, there is no reason for Raspberry Pi OS to become a pure blend of Debian or Ubuntu for example. It is simple enough to get your pi-specific kernel flavour into Debian, or have everything enable there, to avoid building Raspberry Pi OS at all for ARM64.

I am aware that RPi1 / Raspbian are not possible to bootstrap in the Debian Project itself due to lack of armv6l server grade hardware.

But even then, I'm simply asking you to customize base-files & debootstrap in addition to the kernel packages, to ensure there are enough breadcrumbs and pointers to correctly find & rebuild sources of Raspberry Pi OS and for end user to access documentation, community, and where to file issues.

Cause clearly Raspberry Pi OS users should not contact Debian BTS about kernel bugs, but should contact Raspberry Pi OS instead.

The important question is whom/where/how a custom /etc/os-release helps and whom/where/how it harms (including additional effort in any way). Currently the only practical argument mentioned was that piwheels uses the /etc/os-release ID for statistics and now the 64 bit RPi OS will be mixed with Debian. Against it stands the need to update likely hundreds of installers/setup scripts to support the 64 bit RPi OS when using a custom /etc/os-release ID (Regardless if using the ID in installers is good practice or not, it is in fact used, with success!).

/etc/os-release is primarily end-user facing documentation, together with lsb-release, issue.net and the rest of the files. It matters to end user to know what they are running, what it is called, and what the URLs are for support, documentation, issues.

It is possible to customize them without breaking compatibility with any existing tools that try to detect "deb based OS" whilst still providing customzied values, and without breaking ability to automatically build packages.

For example, you will notice that whilst Ubuntu customizes /etc/os-release /etc/lsb-release it maintains compaiblity with dpkg/deb suite discovery by keeping /etc/debian_version intact which points at bullsey/sdi, and by ensuring that /etc/dpkg/origins/* files are valid and correct derife from debian.

Debian is explicitely setup to handle derivatives and their names. As long as the dpkg vendor, lsb-release, deboostrap have correct matching values, everything identifies derivatives "like any other .deb / debian-like distribution" whilst having custom /etc/os-release file.

It seems there is lack of understanding of how all the files in base-files should be customized to correctly advertize a custom name, without breaking compat.

MichaIng commented 4 years ago

Debian ships hundreds of dtbs & flash-kernel and u-boot builds to boot the stock Debian installer on hundreds of different boards both armhf and arm64.

Interesting, I didn't know those lists (the "buster" links btw look very similar), although the dtbs are missing quite important boards (RK3399 most importantly) and the bootloaders are only available for a very few very rarely used boards. I guess those are to be used by the generic ARM kernel?

Never tested it to be true, but I bet the provided functionality by these is very limited. However indeed a guess only, based on some other attempts to get things working with generic or mainline kernel.

That is just not true. debootstrap & sbuild defaults to deriving packaging list from the requested suite name & variant.

I was not talking about building deb packages here but about deriving which build and how to install when coding installers, sorry for confusion.

For the ARM64 port, there is no reason for Raspberry Pi OS to become a pure blend of Debian or Ubuntu for example. It is simple enough to get your pi-specific kernel flavour into Debian, or have everything enable there, to avoid building Raspberry Pi OS at all for ARM64.

And that is the argument and reason why they do it exactly like this, put RPi-specific kernel flavour into Debian for ARM64, merging all into a handy ready-to-run image file, just it. The fact that it says "Raspberry Pi OS (64 bit)" above the download button does not make it something different.

I think a major point of what you write is about kernel vs userland indeed:

Of course kernel or hardware-related issues must be reported to RPi devs, but I guess this is obvious enough?
On the other hand when you face issues with MariaDB or the webserver, or anything else that is pulled from the Debian repository, it doesn't make sense to report this to RPi devs.

Not sure what you mean with the OS build sources, as this has neither something to do with /etc/os-release, nor the repository. If one needs to know how the downloaded and flashed OS image was produced, then one should start searching for it where it was downloaded from? When it's about the repository packages, 99% are from Debian, otherwise the package meta data contain links to sources and who is maintainer.

And yes of course it is "possible" to make everything work with a custom "/etc/os-release", but this means coding effort for hundreds of installers. This is what I tried to say: It is not only important what is possible and what would be the "correct" way and what would suit an ideal situation or guidelines etc, but sometimes it matters more how things are currently practically done in fact. Currently the change of /etc/os-release WILL break many things.

However finally it is neither mine nor your decision, probably things will change, and the RPi dev/maintainer team will surely discuss and decide based on deeper knowledge then both of us (I guess) 😃.

raspberrypi / Raspberry-Pi-OS-64bit

/etc/os-release info #6