Initial work on OPKG updates and package installs

[johntoomey]

• [x] Figure out how to provide packages for the two reference x86 switches together in one place
  • i think the best way to do this is to provide one dir per arch and then eliminate duplicate files with sha and symlinks
  • UPDATE: this appears to be possible by just running the bitbake command to update the package index before rsyncing the files to the output dir - need to test if this actually works.

mion

• [x] Add package management do default images
  • done https://github.com/NetworkGradeLinux/meta-mion/pull/102
• [ ] ~add a bbappend for the opkg recipe to modify etc/opkg.conf~
  • this is generated by yocto with a few config settings
• [x] Check if opkg has the upgrade functionality to upgarade all packages or just one at a time
  • i think the version of opkg installed by yocto can upgrade all packages with one command - need to test
• [ ] ~Figure out how to add a curated list of useful packages~

CI

• [x] Figure out where to store packages on the server
  • /var/www/html/{src,pkg}/
• [x] Create rsync job - both packages and src (d/l dir) - once per week on a monday night or manual trigger?
  • done: this will be done with every build - maybe it should just be nightly..

misc

• [ ] ~Create the subdomains and figure out how this might work - emailed @benvantende about this~
• [ ] ~Define how mirrors might work~

TODO: create a task in the backlog to track whatever needs to happen next

[johntoomey]

https://github.com/NetworkGradeLinux/mion-docs/wiki/Mion-update-strategy

the questions that were here have been moved to the wiki

to start - the question is - what is the minimum we can do to set the groundwork for this foundation during this release cycle? @toganlabs @kprosise @CommitThis @trishan @N-Mc-Nally

The questions above need to be split out into individual tasks and we need to decide which ones get added to the Todo for this release and which ones need to go onto the Backlog.

[johntoomey]

from discussion with @toganlabs and @CommitThis:

• should all systems have a package manager installed and should it be enabled / configured by default

  • Yes - package manager installed by default
  • Build only packages that are part of the default images for the start - maybe add individual packages in the future (no world build to start)

• do we want to maintain a package repo or provide instructions for users to do this themselves

  • maintain a package repo ourselves -
  • need to look into mirroring

• are we ok with the binary distribution nature of packages

  • need to have mirror of dl dir - excluding IP restricted source

• do we maintain the repos on our infrastructure or do we need mirrors - can we rope anyone into helping

  • yes - need more infrastructure
  • not yet on the mirrors
  • pkg.mion.io - src.mion.io -

• do we have to have one repo per machine or could we get to one repo per arch

  • its complicated - we need to do some more testing

• what needs to be done with ci to get this working

  • what are the world build commands? dont care at the moment
  • do we really want to provide world builds considering that we cannot validate the packages that are being built or distributed or do we want to limit the scope to updates of default packages + curated list of extra packages - answered above

• are there any genuine reasons not to use opkg/ipk - seriously needs to be a very good reason

  • it is more difficult to update using IPK because it doesnt figure out the deps but since we are on the dunfell stable branch for all submodules it is possible that updates to packages will actually be miniman - how do we measure update frequency of base packages?

• whatever solution we come up with - what is the overhead in maintaining it all, week to week, and do we have the resources to do it?

  • need to quantify this if possible

[johntoomey]

• https://wiki.yoctoproject.org/wiki/TipsAndTricks/EnablingAPackageFeed
• https://raymii.org/s/tutorials/Building_IPK_packages_by_hand.html#toc_2
• https://openwrt.org/docs/guide-user/additional-software/opkg

[johntoomey]

this is linked: https://github.com/NetworkGradeLinux/mion-ci/issues/19

[johntoomey]

ok - calling this done for now. what has been done:

• enable package manager by default in the base system: https://github.com/NetworkGradeLinux/meta-mion/commit/f3561c06110b3e1ead2c3279b400f6cbcee1ef11
• set the default URL for the package feed: https://github.com/NetworkGradeLinux/mion/commit/11480cc3fc356bedcafc4a52a1090ad56ac4a081
• update the build CI to recreate the packages every night during the build