Create Linux Abstractions

[rmartin16]

What is the problem or limitation you are having?

Unlike Windows and macOS, Linux offers a robust diversity of operation. The impact of differences among Linux distros is most pronounced for Linux System packaging. While all the distros mostly provide the same services, they all implement them differently. This includes how they self identify, package installers, package naming, filesystem layouts, etc.

These differences are currently handled in-line without much re-usability. For instance, providing the user with the command line to install a specific package; I would like to use that logic in Linux System packaging for error handling with X11 passthrough for Docker....but that isn't really possible currently.

Describe the solution you'd like

Introduce abstractions for Linux implementations and their tools.

The Linux abstraction would provide a uniform interface to details each distro provides but each differently.

The Linux tool abstraction would encapsulate how a tool is represented in any Linux distro. For instance, the pkg-config tool on RHEL systems is in the pkgconf-pkg-config package while Arch provides it in the pkgconf package (although...and this may be even trickier...we install the base-devel "virtual" package to get it).

Describe alternatives you've considered

Continue to piecemeal Linux's lovely embrace of diversity.

Additional context

Criticism welcome since I would like to implement something that addresses this.

[freakboy3742]

I'd say I'm broadly in favor of what you're suggesting - with the caveat that there's some fuzzy detail around how far the border around "Linux abstractions" stretches.

I definitely have no objection to a neat wrapper around identifying Linux distros, identifying important file system locations that we need to know (e.g., the lib/lib64 eccentricities), identifying if a specific system package is installed, and installing a package that isn't available (or an idempotent "ensure package is available"-style command).

I could probably also get behind wrappers for specific functionality like the X11 wrappers - "ensure X11 tunnelling can be done" type validation.

I'm not sure how far down this rabbit hole you're planning to go, though; or how you're planning to wrap this up in a way that is usable both inside and outside docker.

So, I guess this is a "show me what you've got in mind".

[rmartin16]

My approach so far has been to implement this abstraction as a Tool; in that way, a tool named linux would be verified for the local machine or for an arbitrary Docker image and provide a unified interface for details about the linux install via the ToolCache. Since different apps in a project could technically use different Linux installs, this tool would be implemented as an app-specific tool.

However, this is creating a chicken and egg problem. A lot of this Linux information is necessary to finalize app configuration for Linux System builds....but that step happens before app tool verification runs. So, app configuration finalization cannot use app-specific tools. I was going to try to rearrange this....but if we consider the Create command, app configuration finalization must run before the command starts and app tool verification must run after the actual app has been laid out in the filesystem.

So, creating an app-specific "Linux tool" that represents a specific Linux installation doesn't look like it's going to work.

Thinking out loud...sticking with the "Linux tool" approach, since the tool could not be defined as an app-specific tool (without a lot of other refactoring of Briefcase), the tool would need to be a project-wide tool. With that constraint, perhaps this tool could simply return an object that provides the Linux abstraction which the individual Command implementations can use and track. So, for Linux System builds, this "Linux tool" could be invoked for the targeted Linux distro and the returned object could simply be stored in the app configuration....similar to how all minutia of the targeted Linux is being stored now.

[freakboy3742]

Is the problem maybe that "Linux" is a too big of a target for a single "Tool"? Does the problem get any better if it's broken down into smaller pieces? I'm thinking something like a Meta-tool for "command line utility" so you independently verify "socat", "which" etc, and a "LinuxPackaging" tool to verify if rpm-build et al are installed.

[freakboy3742]

A mental note for a potential related feature: #1137.

One of the issues we hit in tutorial situations is that Linux users don't have the system packages for GTK etc installed. When the get to briefcase dev for the first time, the app fails - but the failure mode is an obscure compilation error in the middle of installing PyGobject.

Given that an app declares the system package requirements, it would be desirable to verify as a precursor to installing Python packages (both in briefcase dev, briefcase update -r, and the inferred equivalent for briefcase run) that the declared packages (both system_requires and system_runtime_requires) are installed. We can't actually install them, as we don't want any part of Briefcase to be running sudo - but we can give the exact command line that is required to install those packages.

[rmartin16]

Is the problem maybe that "Linux" is a too big of a target for a single "Tool"? Does the problem get any better if it's broken down into smaller pieces? I'm thinking something like a Meta-tool for "command line utility" so you independently verify "socat", "which" etc, and a "LinuxPackaging" tool to verify if rpm-build et al are installed.

It may be reasonable to break it up that way....but the crux of the problem still stands: which Linux are we talking about?

Because whether I need a specific command line, or a package name, or a file system reference, I need to know the distribution of Linux.

I think one thing I will need to do is make this "linux tool" passive; that is, it will not invoke subprocess or Docker commands. It will just return known static information about a distro or ingest data read from the distro installation by the platform format build logic (whether its the host or a container).

I've also been considering an interesting interface for the tool. To avoid having specific platform format logic needing to manage effective instances of the tool for different distros, the tool could expose information about the host machine via methods/properties on tools.linux while information about arbitrary distros could be reached via methods/properties on tools.linux[<distro name/tag>]. This avoids tucking these references in obscure places on the AppConfig or the Command....haven't decided the merit of this, though.

One of the issues we hit in tutorial situations is that Linux users don't have the system packages for GTK etc installed.

👍🏼 Hit this every time I set up a new VM; I'll definitely keep it in mind.