Debian repository

[p3lim]

Re-opening this issue from the old repository, as it's still something that's very much wanted.

Any chance you could set up a debian repository for this package? It was mentioned in the original feature request (link and link), and I've seen other packages manage this (e.g. blacklist).

This would make upgrading a breeze (link), and can make the installation persistent between upgrades (link).

It would also potentially help solve https://github.com/WireGuard/wireguard-vyatta-ubnt/issues/28.

[p3lim]

Played around with a little, if hosting this somewhere with full control (e.g. not launchpad or some other pre-made service, GitHub pages or a separate repo is possible), reprepro can be used, with a small caveat: debian_revision has to be used to differentiate the packages, since they have the same version for every distribution and target hardware. Adding +DISTRIBUTION~MODEL is a suggestion, e.g. +stretch~e300.

Anyways, the recipe:

• /opt/debs contains all the existing debs found on the repo releases, using the same names
• Install reprepro package

• /opt/apt/conf created, containing the following files:

  • distributions:

    Codename: stretch
    Architectures: mips mipsel
    Components: e50 e100 e200 e300 e1000 ugw3 ugw4 ugwxg
    Description: Wireguard packages for Ubiquiti routers
    
    Codename: wheezy
    Architectures: mips mipsel
    Components: e50 e100 e200 e300 e1000 ugw3 ugw4 ugwxg
    Description: Wireguard packages for Ubiquiti routers

  • options:

    verbose
    basedir /opt/apt

• utility script for reprepro so it can use the existing version names
  • /opt/apt/include.sh:

    #!/bin/bash
    for file in /opt/debs/*.deb; do
        component=$(cut -d- -f1 <<< "$(basename $file)")
        codename=$(cut -d- -f2 <<< "$(basename $file)" | sed 's/v1/wheezy/' | sed 's/v2/stretch/')
        reprepro -C $component -P 500 includedeb $codename $file
    done

• host /opt/apt with Apache (or anything else that can serve files), hiding/ignoring /opt/apt/include.sh and /opt/apt/conf

Adjusting to the existing CI ofcourse, this will maintain the repo by itself.

The repository has multiple parts, it's separated by distribution (stretch and wheezy currently), and utilizing components to separate the repo for the hardware.

The installation instructions for the user would be as follows:

Installing

Check which Debian version is running:

grep PRETTY /etc/os-release

The version name is the last word, either stretch or wheezy.
This will be referred to as <codename> below.

Check what hardware you have:

ubnt-hal getBoardId

The model is in this string, e.g. e300.
This will be referred to as <model> below.

Add repository:

configure
set system package repository wireguard components <model>
set system package repository wireguard distribution <codename>
set system package repository wireguard url '[trusted=yes] http://some.location.of.apt.repo'
commit
save
exit

Update system repos and install package:

sudo apt update
sudo apt install -t wireguard wireguard

The -t wireguard option a filter for the configured repo, which is also used when upgrading.

Upgrading

sudo apt update
sudo apt upgrade -t wireguard

[p3lim]

A note on the [trusted=yes] part of the url, this is to avoid having to set up GPG keys, which is of course highly recommended.

[anthr76]

If I can contribute in any way here I would be happy to!

[jinnko]

One option for hosting the packages would be JFrog's bintray for OSS. If I can find the time I'd be happy to implement it, but can't promise anything in the immediate future.

[anthr76]

I wouldn't mind implementing @jinnko idea or @p3lim either.

Can we get a blessing from the repo maintainers possibly?

Also would it be possible to handle the upgrade process as noted here

[anthr76]

https://bintray.com/anthr76/wireguard-vyatta-ubnt

So I created a test repository with bintray. These seems much of a manual process to upload these debs through the web gui. I'm going to investigate this a bit further tonight along with automating the upload. Bintray or not

[jinnko]

@anthr76 there is the REST API for bintray.

I would expect the automated builds to perform the upload directly as this would give the highest confidence of package integrity and reduces maintenance overhead.

[anthr76]

Wondering would it be possible to use GitHub's actions based on the releases of this repo..

[FossoresLP]

Hi, thanks for the effort.

The reason why I'm not 100% sure about this is that the package is not really tested after being built. When updating using a repo the users might not be ready to toubleshoot issues. What do you think?

Adding the packages to bintray could be integrated in the CI pipeline pretty easily but a testing step would be nice. I was thinking about integrating my spare ER into CI to run some quick tests but haven't gotten around to it, yet.

[jinnko]

Agreed that pushing untested packages would not be great.

Perhaps there could be a testing component and a stable component in the repository, then the CI pipeline could upload to the testing component automatically. After the package has been verified, whether automatically or manually, some mechanism of promoting from testing to stable could be implemented. I'm not sure if bintray has such a feature, so it might be another CI job that could be kicked off, probably manually at first, then at some future time when the testing is automated it could all fall into place without manual intervention.

I would suggest that if there's a way to test the package in a virtual environment (qemu?) it would be more robust and future proof, but I have no idea what the effort is for that, so at this point I guess the easiest approach will be good enough so there's at least something that works.

[anthr76]

FWIW VyOs has docker builds available. Completely unsure how these builds for EdgeOS would work out, but I'd imagine it being impossible to get EdgeOS in any type of enviorment.

@FossoresLP If you need to a ER donation we can work something out.

[FossoresLP]

Hi everyone, I've now created a repo for testing purposes. Thanks @p3lim for your awesome work on this. Most of what I did is based on your comment.

The release is not yet automated using CI and the files differ from GitHub releases. That's because I'm currently modifying the package version until I'll make the necessary changes directly to the packaging process.

The repo URL is https://s3.vorwerk.dev/wg-ubnt/

There is no GPG signing, yet. Please refer to the comment from @p3lin for instructions on adding the repo.

Let me know if it works well for you. If everything works out I'll integrate the repo with the CI and create two versions, one for immediate updates and one that is updated with a bit of a delay to ensure stability.

Privacy notice: The server is hosted in Germany and currently does not keep access logs. That may change in the future and I cannot promise I'll remember to update this comment.

[p3lim]

This might be useful: https://assafmo.github.io/2019/05/02/ppa-repo-hosted-on-github.html

You can set up workflows on this repo to automatically build the deb package, push it to the other repo/branch and update the archive. There are probably actions that exist for this exact purpose already, if not, the workflow runs on ubuntu already so it should be easy to set up manually if necessary.

[FossoresLP]

@p3lim Thanks for the recommendation, hosting is not an issue right now, but I'll keep it in mind in case that changes. The CI process is mostly worked out, I've just not added it to the repository, yet. As soon as I receive some feedback about whether the repo works, I'll modify the CI accordingly.

[jinnko]

I had a go at getting this going on my USG-3 but it failed.

After some debugging I've found that the model, e120, doesn't appear to have a package available.

$ curl https://s3.vorwerk.dev/wg-ubnt/dists/squeeze/e120/binary-mips/Packages
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>dists/squeeze/e120/binary-mips/Packages</Key><BucketName>wg-ubnt</BucketName><Resource>/wg-ubnt/dists/squeeze/e120/binary-mips/Packages</Resource><RequestId>167E2152F1D58B53</RequestId><HostId>cbd21be1-60ce-427b-b5e3-7ecff64a0360</HostId></Error>

Going back to the repo server setup instructions I see that the component is listed as usg3, however when using that in place of the model the curl request is empty (note the Content-Length of 0):

$ curl -v https://s3.vorwerk.dev/wg-ubnt/dists/wheezy/ugw3/binary-mips/Packages
* About to connect() to s3.vorwerk.dev port 443 (#0)
*   Trying 2a01:4f8:252:1494::1...
* connected
* Connected to s3.vorwerk.dev (2a01:4f8:252:1494::1) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using ECDHE-ECDSA-AES256-GCM-SHA384
* Server certificate:
*   subject: CN=s3.vorwerk.dev
*   start date: 2021-04-04 08:27:03 GMT
*   expire date: 2021-07-03 08:27:03 GMT
*   subjectAltName: s3.vorwerk.dev matched
*   issuer: C=US; O=Let's Encrypt; CN=R3
*   SSL certificate verify ok.
> GET /wg-ubnt/dists/wheezy/ugw3/binary-mips/Packages HTTP/1.1
> User-Agent: curl/7.26.0
> Host: s3.vorwerk.dev
> Accept: */*
> 
* additional stuff not fine transfer.c:1037: 0 0
* HTTP 1.1 or later with persistent connection, pipelining supported
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Alt-Svc: h3-29=":443"; ma=2592000,h3-34=":443"; ma=2592000,h3-32=":443"; ma=2592000
< Content-Length: 0
< Content-Security-Policy: block-all-mixed-content
< Content-Type: application/octet-stream
< Date: Tue, 11 May 2021 21:51:03 GMT
< Etag: "d41d8cd98f00b204e9800998ecf8427e"
< Last-Modified: Tue, 27 Apr 2021 17:52:05 GMT
< Referrer-Policy: strict-origin-when-cross-origin
< Server: Caddy
< Server: MinIO
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< Vary: Origin
< X-Amz-Request-Id: 167E219A939E117B
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< X-Xss-Protection: 1; mode=block
< 
* Connection #0 to host s3.vorwerk.dev left intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):

And the apt-get update returns an apparent TLS error. I've checked that apt-transport-https is installed, and found it's linked to libgnutls26, which appears to support TLSv1.2 just fine as the curl commands are also linked to libgnutls26 and are working.

$ sudo apt-get update
Ign https://s3.vorwerk.dev wheezy Release.gpg
Ign https://s3.vorwerk.dev wheezy Release
Err https://s3.vorwerk.dev wheezy/ugw3 mips Packages
  gnutls_handshake() failed: A TLS fatal alert has been received.
Ign https://s3.vorwerk.dev wheezy/ugw3 Translation-en
W: Failed to fetch https://s3.vorwerk.dev/wg-ubnt/dists/wheezy/ugw3/binary-mips/Packages  gnutls_handshake() failed: A TLS fatal alert has been received.

E: Some index files failed to download. They have been ignored, or old ones used instead.

[FossoresLP]

Hi @jinnko, thanks for reporting this issue. I completely missed that Ubiquiti uses different model names internally. I'll modify the repository accordingly in the next days and report back when that's done.