feature: "re-package" Goa repos to allow local recompilation

[mewmew]

It has been said before, and deserves to be stated again, Goa has really been a game changer for the developer experience when interacting with the Genode ecosystem.

There is one common task that I am currently lacking in Goa, and that I imagine others may too.

That is the task of "re-packaging" a Goa repo, to allow for local recompilation.

I do this manually, by copying the downloaded source code to the source directory (src), populating the package directory (pkg) with the needed archives and runtime files, and the root directory with used_apis and a LICENSE file, also downloaded from the online depot.

Being able to automate this process would be great, and would also facilitate contributing to the Goa repositories of other developers, as getting a local compilabe copy up and running would be as simple as running the command:

goa clone genodelabs/black_hole

Just for context, this script is an example of what I run "manually" to make a Goa repo locally compilable: (Note: I didn't try compiling this black_hole repo with Goa, but have used similar methods to "manually" clone Goa repos into the directory structure expected by Goa when compiling projects)

#!/bin/bash

wget -O src.tar.xz "https://depot.genode.org/nfeske/src/black_hole/2024-01-04.tar.xz"
wget -O pkg.tar.xz "https://depot.genode.org/nfeske/pkg/black_hole/2024-01-04.tar.xz"

mkdir -p black_hole

tar -xJvf src.tar.xz
mv 2024-01-04 src_depot

cp src_depot/LICENSE black_hole/
cp src_depot/used_apis black_hole/
cp -r src_depot/src black_hole/

tar -xJvf pkg.tar.xz
mv 2024-01-04 pkg_depot

mkdir -p black_hole/pkg/black_hole
cp pkg_depot/* black_hole/pkg/black_hole/

Also, of course let me know if I am missing something and that there already exist an easy way to handle this process?

Basically, I want to clone a remote Goa project, so I get the same directory structure, as the original developer have when they hit goa publish.

Joyous regards, Robin

[jschlatow]

Wow, that's an intriguing idea! Goa actually captures all the used utility files in the src archive. That means it should be possible to use the extracted src archive as a Goa project without any modifications (if the archive has been packaged with Goa). It should be pretty straightforward to implement such a clone command since Goa already incorporates the mechanisms for downloading and extracting depot archives. Would you like to have a go at implementing this?

Note that depot archives that have not been created with Goa do not use a commodity build system, i.e. they are lacking a Makefile, CMakeLists.txt, etc., but rather come with a target.mk file that is not (yet) supported by Goa. Those archives will also be lacking any import and artifacts files.

[mewmew]

Wow, that's an intriguing idea! Goa actually captures all the used utility files in the src archive. That means it should be possible to use the extracted src archive as a Goa project without any modifications (if the archive has been packaged with Goa).

Happy you like it Johannes. It feels like a natural extension to the already amazing Goa tool, and it would help facilitate further collaboration between developers of the broader Genode ecosystem.

It should be pretty straightforward to implement such a clone command since Goa already incorporates the mechanisms for downloading and extracting depot archives.

That's great! And for sure, the clone command should be able to make use of the same mechanisms for downloading and extracting depot archives.

Would you like to have a go at implementing this?

I have skimmed though the source of Goa, and while it looks rather self-explanatory, I myself don't feel I have a deep insight into best practice for how to write expect scripts.

On a first glance, it seems like there are some parts of the code that could become exposed to command injection vulnerabilities unless dealt with great care.

As such, I'll leave it to someone who already has more experience with writing expect scripts.

Discussion for security considerations of Goa development platform

(Note, this is off-topic, but may be the basis of discussions of security considerations of the Goa development platform.)

On a related note, have there been consideration into how to limit the damage of potential threat actors as the Genode ecosystem grows? For instance, having the Goa tool use a chroot on Linux? And, how to limit the damage of a potential command injection vulnerability in the Goa script?

I know a lot of the security may be based on trust. As in, I trust this person that I add a depot download link to, thus their code will not perform malicious actions. And, to begin with, such a trust model works well, but with larger and deeper dependency chains, becomes more laborous to review.

I know that running the compiled artifact in a Genode system would limit what the component is given capabilities to perform. So, the scenario I'm considering is that of trying to take control of a developer machine, running Linux, by exploiting (a potential) vulnerability in Goa.

Just to give an example of the kind of exploit that would be used to target developers, consider the CVE-2018-6574 vulnerability in the go get command of the toolchain for the Go programming language. (A proof of concept exploit for the aforementioned vulnerability: https://github.com/j4k0m/CVE-2018-6574)

I realize the discussion went of on a tangent, so perhaps there are better forums to discuss this as well.

Cheerful regards, Robin

[nfeske]

Thanks Robin for sharing your concerns, which definitely deserve a dedicated issue. I have taken your text to issue #99.