Handle signature files

[maximbaz]

For example, repoctl remove should also remove signature files together with actual package cache files (unless backup-dir is not set to empty string, in which case both files should be ignored).

Today presence of signature files causes repoctl remove to output error: unknown file format. I believe repoctl update also suffers from the presence of signature files.

P.S. In the example below you will see that there are also .sig~ files, handling them is optional (but desired), because they only appear when someone is re-signing the same file twice.

P.P.S. For me personally it's a low priority, because I don't expect repoctl to actually manage cache files (I use backup-dir="") I'm a little annoyed by the false errors error: unknown file format, but I'm actually not affected negatively by this.

---

$ ll /var/cache/pacman/maximbaz-aur/yay*
.rw-r--r-- 4.8M maximbaz  4 Mar 14:09 /var/cache/pacman/maximbaz-aur/yay-3.440-1-x86_64.pkg.tar
.rw-r--r--  566 maximbaz  4 Mar 14:09 /var/cache/pacman/maximbaz-aur/yay-3.440-1-x86_64.pkg.tar.sig
.rw-r--r--  566 maximbaz  4 Mar 13:56 /var/cache/pacman/maximbaz-aur/yay-3.440-1-x86_64.pkg.tar.sig~

$ repoctl remove --debug --backup=false yay
error: unknown file format.
error: unknown file format.
removing package from database: yay
deleting: /var/cache/pacman/maximbaz-aur/yay-3.440-1-x86_64.pkg.tar

$ ll /var/cache/pacman/maximbaz-aur/yay*
.rw-r--r-- 566 maximbaz  4 Mar 14:09 /var/cache/pacman/maximbaz-aur/yay-3.440-1-x86_64.pkg.tar.sig
.rw-r--r-- 566 maximbaz  4 Mar 13:56 /var/cache/pacman/maximbaz-aur/yay-3.440-1-x86_64.pkg.tar.sig~

$ repoctl remove --debug --backup=false yay
error: unknown file format.
error: unknown file format.

$ rm /var/cache/pacman/maximbaz-aur/yay-3.440-1-x86_64.pkg.tar.sig~

$ repoctl remove --debug --backup=false yay
error: unknown file format.

$ rm /var/cache/pacman/maximbaz-aur/yay-3.440-1-x86_64.pkg.tar.sig

$ repoctl remove --debug --backup=false yay

[cassava]

The most recent commit 58f5438 should resolve this issue.

However, all it does is stop complaining. This is by no means support for signatures. I wonder what this would consitute, in particular for the following situations:

• Checking validity of packages: ?
• Adding packages: ?
• Removing packages: just remove them

[maximbaz]

That commit only bumps the version, just double check that you didn't forget to push something 🙂

• Checking validity of packages:
  • is this a repoctl command that I don't know of? otherwise just skip the signature files
• Adding packages:
  • just skip the signature files
• Removing packages:
  • respect the backup option, remove the signatures only if you are removing the actual files, otherwise skip

[cassava]

Yes, but the previous commit updates the dependencies, in particular https://github.com/goulash/pacman which contains the fix.

[cassava]

I asked about validity checking, because if I have a package that I force update, but don't change the signature, then I will have signatures of packages that don't match, right?

But this could come later, if someone feels they would benefit from it.

[sudoBash418]

I, for one, would definitely like having signature management builtin. It would simplify my scripts and be much more robust than my current solution.

[PedroHLC]

I would prefer an option for adding packages only if signature exists (at least for 'repoctl update'), no need to validate it though. Sometimes I call 'repoctl update' and it adds an incomplete .tar.xz (cause they're being written while the commands run), this generates corrupted entries in the database. Signatures are created after the package, so this would prevent this problem. Checking if a signature exists before adding it also is the correct behavior, so any unsigned file would not be mistakenly added, downloaded, etc.

[cassava]

These are definitely good ideas! Currently I have a lot less time than I'd like to, which is why this issue is still open. Sorry guys! :-(

[doronbehar]

I've delved into the code base of Repoctl a little bit today and I think this feature should be incorporated first in https://github.com/goulash/pacman. I couldn't find evidence it supports / reads signature files. What do you think @cassava ?

[cassava]

Hey @doronbehar, it doesn't support signature files (I know, because I wrote it). I would probably add something to repoctl itself first, and then if that fits well in the pacman library, move it there after some testing. Originally, I thought it would be useful to have a "pacman" library to support other applications. So far, I'm not aware of anyone else using it though.

[cassava]

Alright, I implemented some preliminary support of signature files. Since I only just now started working with them, I'd appreciate it if those intereseted (i.e. you) would test it. If I get enough positive feedback, I'll then release a new version of repoctl with this change.

[doronbehar]

Thanks! As a start, I've just tested with an existing repository using branch devel:

I'm not sure if you intended this or not but when a pkg.tar.xz file is manually removed from the repo's directory, it would be nice if it's corresponding .sig file will be removed as well on repoctl update. I've tested this case with and without --require-signature.

Besides that, on repoctl remove and repoctl update, the old signature file is removed as well so that's good :smile_cat:.

When is there a need for using --require-signature? Signing is usually done by makepkg when the package is created.

[cassava]

Hm, should perhaps document the flag a bit better. --require-signature prevents the package being added (with the add command) if there is no accompanying signature. This is useful if you always want signatures to be available for all packages. In the update case, it will prevent the package being added to the database if there is no signature. Suggestion came from @PedroHLC.

As for the orphan signature files... does this really occur that often? :-D

[doronbehar]

Well TBH that was just my initial test case for the new version.. Usually that should never happen in a normal work flow.

As for the --require-signature this could be very helpful! Is there a chance it could be set by default using the configuration file?

[cassava]

Yes! This can be set by default in the configuration. The key is require_signature, such as:

require_signature = true

In addition, I found a related bug in the update command routine, so you might want to try this on the latest devel.

[doronbehar]

Seems fine to me.