Feature/384-generate-checksums-for-component-files

reglim commented 1 year ago

Fixes #384

This is basically a rebase of #473, which was really out of date, so refer back to that PR for more information.

Requirements

All new code should be covered with tests, documentation should be updated. CI should pass.

Description of the Change

Adds a skeleton (skel) folder at .aptly/skel that can store files you want to be in the repository
On publish, aptly will look in that folder and write the files to the repository, and release files
I did not add any API or CLI tags to change the skel directory, as that would be bad security
All files will be copied AS IS. No compressing or signing

The skel folder mirrors the public folder, so on publish .aptly/skel/dists/maverick/main/dep11/Components-i386.yml.gz will create .aptly/public/dists/maverick/main/dep11/Components-i386.yml.gz or equivalent.

Checklist

[ ] unit-test added (if change is algorithm)
[x] functional test added/updated (if change is functional)
[ ] man page updated (if applicable)
[ ] bash completion updated (if applicable)
[x] documentation updated
[x] author name in AUTHORS

codecov[bot] commented 1 year ago

Codecov Report

Attention: Patch coverage is 70.83333% with 21 lines in your changes missing coverage. Please review.

Project coverage is 74.78%. Comparing base (8d09c20) to head (632c8e7). Report is 39 commits behind head on master.

Files	Patch %	Lines
deb/publish.go	55.31%	15 Missing and 6 partials :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #1186 +/- ## ========================================== - Coverage 74.79% 74.78% -0.01% ========================================== Files 144 144 Lines 16256 16314 +58 ========================================== + Hits 12158 12201 +43 - Misses 3156 3167 +11 - Partials 942 946 +4 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

randombenj commented 1 year ago

@jhonny-oliveira would you mind testing the changes before we merge?

jhonny-oliveira commented 1 year ago

Given this:

xd@dep11:~/.aptly/public/appstream$ tree -d -I pool -I appstream ~/.aptly/addon
/home/xd/.aptly/addon
└── ubuntu
    └── dists
        └── lunar
            ├── apps
            │   └── dep11 -> ../../../../../public/appstream/ubuntu/export/data/lunar/apps
            └── play
                └── dep11 -> ../../../../../public/appstream/ubuntu/export/data/lunar/play

After running:

aptly publish snapshot -component=apps,play xtradeb_apps_20230526 xtradeb_play_20230526 ubuntu

I got this:

xd@dep11:~/.aptly/public/appstream$ tree -d -I pool -I appstream ~/.aptly/public/
/home/xd/.aptly/public/
└── ubuntu
    └── dists
        └── lunar
            ├── apps
            │   └── binary-amd64
            └── play
                └── binary-amd64

So, the dep11 directory is still missing (target should be similar to: http://nl.archive.ubuntu.com/ubuntu/dists/lunar/main/). Any ideas? Did I miss something?

I'm also having trouble building the latest code with the previous debian recipe: https://launchpad.net/~xtradeb/+archive/ubuntu/apps/+sourcefiles/aptly/1.5.0-1~xtradeb1/aptly_1.5.0-1~xtradeb1.debian.tar.xz. But, I will ask you about it in a different thread.

jhonny-oliveira commented 1 year ago

Any update on this?

reglim commented 1 year ago

Not really. I hope @btkostner can take a look at this, since I don't actually understand the underlying issue.

btkostner commented 1 year ago

@jhonny-oliveira try using “.aptly/skel” instead of “.aptly/addon”.

PR looks good to me.

jhonny-oliveira commented 1 year ago

Dear @btkostner ,

I have tried your recommendation without success. Is there anyway we can troubleshoot this in a more interactive way? Discord or any other IM?

Thank you!

r4co0n commented 1 year ago

Is there some merit in always looking for the relevant publication's skeleton files in a path aquired by joining skelDir, p.Prefix, "dists", p.Distribution, component?

I would prefer if this was configurable, even if I won't ever touch it. It could be an option you pass to publish repo|snapshot|switch, or am I mistaken?

aptly-dev / aptly