OMV_EXTRAS_APT_REPOSITORY_URL does not play nice with apt-cacher-ng

[deatharse]

Background

ACNG Manual SSL outlines how to cache requests when using TLS.

there are three (and a half) methods to use SSL.

First, the "half method" - not using the proxy at all, configuring each client to not use the HTTP proxy for HTTPS urls. This will obviously disable central caching and requires the client has separated configuration options to set this. For Debian based distros, this can be done by adding a configuration like this: Acquire::https::proxy "DIRECT"; to apt.conf or one of the apt.conf.d files. See section 8.5 for further information. The "backend configuration method": if the clients access the remote sites through URLs remapped on the server, the cacher admin can add https URLs to backend lists instead of http urls. Data will be cached just like usual. The "laissez-faire method": in acng.conf (or related) configure the PassThroughPattern option to contain a regex like . and configure the clients to use apt-cacher-ng as HTTP proxy and let the clients connect to https URLs "as usual". Some limited access control can be achieved through adjustment of the regular expression (. permits access to any host and any port, including 443 for https). Data is not cached on the server. The "tell-me-what-you-need method": on the client side, modify the access URLs and change https to http and put the string "HTTPS///" between http:// and the host name. With this trick, the user client will access the proxy like going for a usual HTTP download and the proxy will access the remote URL with the https protocol. Caching (and file merging to repositories) will work and there is still enough flexibility for the users. The disadvantages of this method are basically the same as with the access URLs rewriting (see Section 3.2) but the method is still the preferred one by the apt-cacher-ng author. For apt's sources.list, the modification may look like the following example.

deb http://HTTPS///get.docker.com/ubuntu docker main
# If apt-cacher-ng is configured as proxy in APT, this makes it
# switch internally to https://get.docker.com/ubuntu
deb http://acnghost:3142/HTTPS///get.docker.com/ubuntu docker main
# Basically the same just with access to apt-cacher-ng through
# URL rewritting instead of setting http proxy.

Issue

Changing OMV_EXTRAS_APT_REPOSITORY_URL via:

  omv-env set OMV_EXTRAS_APT_REPOSITORY_URL http://HTTPS///openmediavault-plugin-developers.github.io/packages/debian

breaks retrieving the key as 10omvextras.sls#L35 uses the repo_url to pull the key.

relevant salt output:

          ID: omvextrasbaserepo
    Function: pkgrepo.managed
        Name: deb http://HTTPS///openmediavault-plugin-developers.github.io/packages/debian shaitan main
      Result: False
     Comment: Failed to configure repo 'deb http://HTTPS///openmediavault-plugin-developers.github.io/packages/debian shaitan main': Error: [Errno -2] Name or service not known reading http://HTTPS///openmediavault-plugin-developers.github.io/packages/debian/omvextras2026.asc
     Started: 16:43:49.480465
    Duration: 1100.803 ms
     Changes:
----------

Proposed fix

Adding the following (keeping with the naming convention in 20dockerrepo.sls)):

{% set repo_key = salt['pillar.get']('default:OMV_EXTRAS_KEY_URL', 'https://openmediavault-plugin-developers.github.io/packages/debian') -%}

using the new repo_key instead of repo_url in download_omvextras_key.

[ryecoaaron]

You are filing an issue on unreleased code? There are many broken things in the current state especially the keys. This code may not be released until the OMV repos have saltstack 3005 or newer.

[deatharse]

I was testing out the InstallScript repo, and this was something I experienced. I had not realised this was as yet unreleased.

[ryecoaaron]

Here is the code for the version that the install script would install - https://github.com/OpenMediaVault-Plugin-Developers/openmediavault-omvextrasorg/blob/9a54f3cfca7377e667dbc5e55be0dcc6902da81a/srv/salt/omv/deploy/omvextras/default.sls

[deatharse]

Both variants exhibit the behaviour originally posted.

[ryecoaaron]

Not surprising. This variable was added to allow some people to use another mirror in a country where github is blocked. It wasn't really meant for apt-cacher. I will try to make the future code in flight work with it.

[deatharse]

No problem separating key urls from repo urls would be useful in future

[ryecoaaron]

These are separated now and omv-extras is back to using bash instead of saltstack. https://github.com/OpenMediaVault-Plugin-Developers/openmediavault-omvextrasorg/blob/master/usr/sbin/omv-aptclean#L60