Publish to snap store

[hongbo-miao]

First thank you so much @vergoh detail answer at https://github.com/vergoh/vnstat/issues/252 !

I am wondering if possible to publish vnstat to snap store even manually first because seems this library does not release frequently. 😃

It will help people install latest version easily. Also, it would help automate easily to install latest version on every machine by Ansible. Thanks! 🚀

[vergoh]

After quickly browsing through the snap / snapcraft documentation, I'd estimate most of the effort would go to creating the package definition file (snapcraft.yaml) while the publishing itself shouldn't be that much time consuming (unless the process is gated in some way from the store point of view).

One of the problems I could already identify is figuring out how to define the interfaces (permissions) for the application. https://snapcraft.io/docs/supported-interfaces lists the available interfaces but doesn't directly tell you which one(s) would be needed for vnStat for the /sys/class/net and /sys/devices/**/net/ read-only access (or minimally /proc/net/dev). The network-observe interface sounds about right based on the name alone but the documention really doesn't tell much additional details what's included and what's not: https://snapcraft.io/docs/network-observe-interface

Additionally, as soon as some interface having "no" in the "auto-connect" column is used, that will also require some additional steps to be executed by the user in addition to just installing the snap. That would require some extra testing if there's some method of providing the necessary instructions to users automatically after install or is the expectation with snap that users figure out these sort of details from the snap store package description.

[vergoh]

The snap name vnstat was already reserved since a package of the same name exists via apt. However, as there wasn't any other developer working on packaging it, I got it transferred. Getting the build and runtime dependencies sorted out didn't take too much time after setting up a suitable VM running for testing things without risk of breaking anything that's in actual use. However, some of the sandbox restrictions and the publishing setup aren't either that well documented or working as expected.

I've created https://github.com/vergoh/vnstat-snap for the snap packaging content and I'll try to keep the limitations section at least up to date. Essentially, this is what I found from usability perspective (when a user gets the snap installed):

• network-observe interface needs to be connected and the service restarted, otherwise vnstatd doesn't have access to interface speed details necessary for internal failsafes
  • a little bit surprisingly accessing network interface lists and traffic details didn't require any extra permissions even with the strict sandbox mode being used
• configuration file doesn't get exposed, haven't found any good solution for this yet
• vnstati command is vnstat.image instead (enforced by the snap sandbox)
• vnstat.image can't write files due to snap sandbox restriction
  • use -o - >file.png as a workaround
• man pages aren't available, this appears to be a snap "feature" currently
  • https://forum.snapcraft.io/t/support-for-man-pages/2299
  • https://bugs.launchpad.net/snapd/+bug/1575593
• I have no idea where the database ends up being stored if one would want to take a backup but at least it does survive upgrades

The snap, vnstat, is currently unlisted and available only from the edge channel to make it clear it should still be considered somewhat experimental. sudo snap install --edge vnstat can be used to get it installed but those two other documented commands need to also follow after that. https://snapcraft.io/vnstat is also accessible with the direct link.

Some other issues that I've yet to solve from the build perspective:

• local builds (and that also include GitHub actions) are limited to the architecture available so essentially only amd64 is possible, cross compiling may technically be possible but would also require cross compiling sqlite3 first so that didn't feel worth the effort
• snapcraft.io integration with GitHub doesn't appear to work well for a reason that the documentation at least doesn't explain, changes to the repository do trigger new builds for all architectures but something blocks the release of the build products without explaining why
• remote builds via Launchpad allow building for all architectures but the tooling doesn't support releasing directly and all parts of the tooling (Launchpad build launch and snap store upload) require authentication that appears to assume a desktop system is being used
• once some way of automating the builds is figured out, there would also need to be some additional logic done for getting releases and development versions separated

[hongbo-miao]

Thanks @vergoh ! I tested, it works well! 😃

I installed by Ansible:

- name: Install packages
  hosts: hm-ubuntu
  tasks:
    - name: Install packages from Snap (edge)
      become: true
      become_user: root
      community.general.snap:
        name:
          - vnstat
        state: present
        channel: edge

And it works well even on this aarch64 Ubuntu:

parallels@ubuntu-linux-22-04-desktop:~$ uname --all
Linux ubuntu-linux-22-04-desktop 5.15.0-41-generic #44-Ubuntu SMP Thu Jun 23 11:20:13 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux

parallels@ubuntu-linux-22-04-desktop:~$ which vnstat
/snap/bin/vnstat

parallels@ubuntu-linux-22-04-desktop:~$ vnstat -v
vnStat 2.11 beta1 by Teemu Toivola <tst at iki dot fi> (SQLite 3.37.2)

parallels@ubuntu-linux-22-04-desktop:~$ vnstat --live 1 --style 4 --rateunit 0
Monitoring enp0s5...    (press CTRL-C to stop)

   rx:          33 B/s   66 B          tx:          89 B/s   178 B
   rx:          33 B/s   132 B          tx:         105 B/s   388 B
   rx:          33 B/s   198 B          tx:          97 B/s   582 B
   rx:          33 B/s   264 B          tx:          97 B/s   776 B
   rx:         168 B/s   600 B          tx:          97 B/s   970 B
^C

 enp0s5  /  traffic statistics

                           rx         |       tx
--------------------------------------+------------------
  bytes                        600 B  |           970 B
--------------------------------------+------------------
          max                168 B/s  |         105 B/s
      average                 60 B/s  |          97 B/s
          min                 33 B/s  |          89 B/s
--------------------------------------+------------------
  packets                          6  |               5
--------------------------------------+------------------
          max                  1 p/s  |           0 p/s
      average                  0 p/s  |           0 p/s
          min                  0 p/s  |           0 p/s
--------------------------------------+------------------
  time                    10 seconds

Feel free to close this issue! ☺️

[vergoh]

I'm not planning on closing this issue until I have all the things I feel necessary sorted out.

I now think I did get the automated builds working so one thing less to be solved. It turned out the integration directly from Snap store to GitHub isn't really that ready as the status messages (and errors) aren't that well documented and the build triggers support only using directly the master branch. That would result in only the edge builds being there and no automatic triggers if the actual implementation in this repository is changed -> doesn't fit my requirements.

I didn't find any practical examples of doing cross compiling using the GitHub provided build runners (for docker builds there are qemu solutions) but instead there's a somewhat longer "path" using Launchpad for the builds which however requires also setting up a repository on the Launchpad server. This wasn't again that well documented but (for future reference) https://www.youtube.com/watch?v=X_U-pcvBFrU + https://github.com/nextcloud-snap/nextcloud-snap provided enough examples to get things going. As a results, there's now automated builds setup for development versions on the edge channel and releases on the stable channel. Both will also trigger once per month (like the docker container builds) even if there's no other activity to get possible dependency updates included.

As for the database location (from backup perspective) I was wondering, it ends up getting located in the /var/snap/vnstat/current/var/lib/vnstat/ directory (current being a symlink to the used build number).

What's remains missing is the configurability part. While the default configuration is likely to be ok for many uses, there are at least few settings that I find myself changing depending on the purpose of the system so configurability needs to be somehow support. Either I'll needs to figure out if there's some way to have the configuration file visible and editable (I can see from debug output that it's not visible at all although it's clearly there in the snap package) or the other option appears to write some sort of wrapper for snap get and snap set commands and pushing those settings to the scope of vnStat.

Regarding that ansible setup you did, you appear to still be missing the following steps I mentioned earlier:

sudo snap connect vnstat:network-observe
sudo snap restart vnstat

Without those, the daemon isn't going to have visibility on the interface speeds and it will be using the fallback value instead from the configuration file you can't edit. Depending on the used environment, interfaces and traffic rates, this will either not matter at all or cause false results.

[vergoh]

https://github.com/vergoh/vnstat-snap/ now has the configurability using snap get / snap set documented. Everything that can be changed in the configuration file gets automatically populated in Snap, including new options if more get added later. I've also added build and install tests to the GitHub actions of that repository. https://snapcraft.io/vnstat has been enhanced with few screenshots and made public. That should now cover everything necessary for closing this.

[hongbo-miao]

Wow, thank you so much @vergoh ! 🚀🚀🚀

[vergoh]

network-observe is now automatically connected on install (https://forum.snapcraft.io/t/auto-connect-request-of-network-observe-for-vnstat/36941). As a result, the previously mentioned snap connect and snap restart commands are no longer necessary after install.