Intended workflow with sys-git

[wassp-ds]

Commitment

I confirm that I have read the following resources:

• How to ask questions The Smart Way
• Writing the perfect question
• Question checklist

Question

Question on interacting with sys-git and remote repositories. What is the intended workflow between the dev, let's say GitHub and sys-git? I am relating that question to the instruction on from the main README on how to keep the salt formulas up to date as well. Can you provide a basic workflow example?

Thanks!

.

[ben-grande]

wassp-ds wrote:

Question on interacting with sys-git and remote repositories. What is the intended workflow between the dev, let's say GitHub and sys-git? I am relating that question to the instruction on from the main README on how to keep the salt formulas up to date as well. Can you provide a basic workflow example?

sys-git is your "GitHub" (remote repository) but without an issue tracker, purely a git server.

You clone software on the "dev" qube from GitHub, Gitlab, Codeberg, Sourcehut and any other repository you fancy (if qusal.ConnectTCP policy allows).

Them you may store repositories in "sys-git", consider it as a backup of your repositories. Clients can fetch and pull from the server if the policy allows. You may use "qubes-builder" to access only the "qubes-builderv2" repository in "sys-git", while "dev" may only access "qusal" and "dotfiles". You may also allow a qube to fetch from "sys-git" while not being able to push to it, just as if it was a read-only repository.

Workflow:

1. Clone repository from a Git server;
2. Verify integrity and authenticity (signatures);
3. Configure policy to allow the "dev" qube to push and fetch repository "dotfiles" to "sys-git"; and
4. Configure qube "disp-dev" to fetch repository "dotfiles" from "sys-git".

The workflow makes more sense when you have multiple client qubes, "dev-companyA" accessing only repos of "companyA", "dev-companyB" accessing only repos of "companyB".

As "sys-git" is not networked, it cannot pull a repository by itself, you need to push from another qube first. It is not intended to keep fetching "qusal salt formulas", but to store any repository you'd like.

As how to keep the formulas up to date... as I don't provide an RPM repository yet due to server and maintenance costs, you should fetch the repository everytime before applying new changes. This way, you can have most up to date configuration. For already applied formulas, you'd need to reapply them if something changed. A lot of the times you don't have to apply the whole formula again, just specific states, but this is just for advanced users, as it can lead to mistakes that applying the whole formula won't lead.

-- Benjamin Grande

[wassp-ds]

So let me replay that with another use case, to make sure I understand it: Let's say I have two qubes: dev-companyA qube that has access to the github page through qusal.ConnectTCP, with 3 repos and dev-companyB that has access to another 3 repos, just to set focus. I will then set my policies on the sys-git per repo for the qubes and repos:

qusal.GitFetch +companyA-repo1 dev-companyA sys-git ask target=sys-git default_target=sys-git

...rinse and repeat for the other 2 repos and all the other permutations between the qubes and repositories (total of 6 rules for qusal.GitFetch, respectively adding others as and when required). Probably the next step is then to go through the

cd ~/path/to/repo-from-companyA-github
git init-qrexec

on dev-companyA. This will create a bare repo on sys-git for each of the repos initialised / pushed from the fetched and pulled ones from GitHub. (I can probably use sys-pgp for split-signing and verifying the integrity and authenticity). Again - repeat for dev-CompanyB accordingly. (I can probably create subdirectories on ~/src/companyA, ~/src/companyB etc. to make sure I don't have everything in one big src directory).

git remote add sg qrexec://@default/repo-from-companyA-github to add the remote on sys-git.

I develop as normal on the dev-companyA - create branches, sign-commit and push to GitHub, but I want to also push to sys-git to store a local copy of the changes.

Is that more or less correct? I am starting to get blurry about how the git push will be differentiated when pushing to GitHub, and when to sys-git :).

[ben-grande]

So let me replay that with another use case, to make sure I understand it: Let's say I have two qubes: dev-companyA qube that has access to the github page through qusal.ConnectTCP, with 3 repos and dev-companyB that has access to another 3 repos, just to set focus. I will then set my policies on the sys-git per repo for the qubes and repos:

qusal.GitFetch +companyA-repo1 dev-companyA sys-git ask target=sys-git default_target=sys-git

...rinse and repeat for the other 2 repos and all the other permutations between the qubes and repositories (total of 6 rules for qusal.GitFetch, respectively adding others as and when required).

Yes, currently you have to either allow all repos for a qube or give per repo access. The other method is using multiple sys-git: sys-git-companyA, sys-git-companyB, and client qubes with tag companyA-code are allowed to all repos of sys-git-companyA qube and qubes with tag companyB-code, are allowed to all repos of sys-git-companyB qube.

But now that I am thinking about it, I can make the usage better, of allowing per "master" directory access and them per repo.

With sys-git with the following file hierarchy:

.
├── companyA
│   ├── testA1
│   └── testA2
└── companyB
    ├── testB1
    └── testB2

And see the following Qrexec policy arguments:

+companyA ## related to all repos under the master dir "companyA"
+companyB ## related to all repos under the master dir "companyB"
+companyB+testB1 ## related to repo "testB1" under the master dir "companyB"
+companyB+testB2 ## related to repo "testB2" under the master dir "companyB"

With this method, you could allow per parent directory access. I will prohibit the creation of repositories with the + sign in them.

Probably the next step is then to go through the

cd ~/path/to/repo-from-companyA-github
git init-qrexec

on dev-companyA. This will create a bare repo on sys-git for each of the repos initialised / pushed from the fetched and pulled ones from GitHub. (I can probably use sys-pgp for split-signing and verifying the integrity and authenticity). Again - repeat for dev-CompanyB accordingly. (I can probably create subdirectories on ~/src/companyA, ~/src/companyB etc. to make sure I don't have everything in one big src directory).

Currently not possible to use subdirectories. The character / is prohibited and also can't be used in argument of the Qrexec policy.

git remote add sg qrexec://@default/repo-from-companyA-github to add the remote on sys-git.

I develop as normal on the dev-companyA - create branches, sign-commit and push to GitHub, but I want to also push to sys-git to store a local copy of the changes.

Is that more or less correct? I am starting to get blurry about how the git push will be differentiated when pushing to GitHub, and when to sys-git :).

It is more or less correct except the use of subdirectories which are not currently supported.

Pushing to GitHub or sys-git will not get blurry, it is just like pushing to any other repository when you have multiple ones:

% grl ## git remote -v
origin  gh:ben-grande/qubes-core-admin-linux (fetch)
origin  gh:ben-grande/qubes-core-admin-linux (push)
upstream        gh:qubesos/qubes-core-admin-linux (fetch)
upstream        gh:qubesos/qubes-core-admin-linux (push)

I can do git push origin BRANCHNAME and git push upstream BRANCHNAME. Use if git push -u REMOTE BRANCH will make it default to that remote for the current branch, being the remote sys-git, origin, upstream, whatever you have configured.

Don't overthink it, just do it. Use a test repository if you are afraid, do some commits, push to both github and sys-git, experiment.

[wassp-ds]

Cool - thanks for the explanation :)

[ben-grande]

If you want the subdirectory to work, please open a new feature issue.