Simplify the SDK sources

[t-lo]

Issue

• SDK Contains largely unused Chromium OS components like repo.
• Initialising SDK sources uses custom logic to keep sources and release versions aligned

Impact

• Package maintenance process is unnecessary complex

Ideal future state

• All packages use plain Gentoo approach, no need for repo
• Dependencies like coreos-overlay, portage-stable are pulled in via git submodules

Design Proposal: Simplify Flatcar versioning and build process

This design document proposes simplifications to Flatcar's versioning / build process. Motivation for the simplifications proposed is to remove unnecessary complexity from the build process.

Build process complexity

• repo Because of its Chromium OS legacy, the Flatcar build process employs the repo tool to manage project-related repositories[1]. However, real-life Flatcar development does not make use of the repo git project management features. The tool is only used at SDK initialisation time by cork - which itself will be deprecated by the SDK container. While some ebuilds in coreos-overlay make use of CROS_WORKON_*, there is no hard dependency to repo - package builds work fine just using the cros-workon eclass (which can be deprecated at a later point).
• Manifest repositories and XML build info repo uses build manifests (example) to pin all repo-managed repos to specific git revisions. However, this information is not used except for two repos: portage-stable and coreos-overlay. All other repos' entries - specific to a single package - always point to master.

Proposed simplification

• Stop using repo - and cork altogether - for SDK initialisation. The SDK container PR implements this - instead of cork initialising a chroot, an SDK container is used. repo is not used at all.
• Replace manifest repositories with a single authoritative repo. That repo - proposed to be flatcar-scripts - defines information on Flatcar versions (SDK and OS image) as well as versioned references to the ebuild repositories (coreos-overlay and portage-stable). There is an ongoing discussion on how to implement the versioned references.
  • The proposed future workflow for working with sources of a given Flatcar release is:
    • clone the authoritative repository
    • check out a specific release tag, optionally run a git hook / helper script
    • Flatcar build tooling and ebuild repos are now on the version used for building the release specified by tag
  • Accordingly, the proposed workflow for creating a release is:
    • clone the authoritative repository
    • define release information (release version), tag
    • update versioned references to ebuild repos to match the state we want to release
    • push changes to authoritative repository, triggering release build and tests

---

[1] - The following repos are cloned at SDK initialisation time:

• afterburn
• appc-acbuild
• appc-spec
• baselayout
• bootengine
• coreos-cloudinit
• coreos-overlay
• docker
• fero
• grub
• ignition
• init
• locksmith
• mantle
• mayday
• nss-altfiles
• portage-stable
• rkt (yes, really)
• sdnotify-proxy
• seismograph
• shim
• systemd
• toolbox
• torcx
• update_engine
• updateservicectl
• update-ssh-keys

[dongsupark]

Another side issue is to delete coreos-devel/sdk-extras from coreos-overlay. Doing that, we can delete at least ~20 other packages from the source tree.

Apparently the sdk-extras was originally meant to work as a meta package to pull in all the optional packages in the SDK at once. AFAIK it has been unmaintained since 3~4 years, so an attempt of emerge coreos-devel/sdk-extras will give you a huge list of conflicts to resolve. ;-) So I believe it is hard to resurrect sdk-extras at the moment.

[dongsupark]

PRs for deleting sdk-extras and its dependencies:

• https://github.com/kinvolk/portage-stable/pull/200
• https://github.com/kinvolk/coreos-overlay/pull/1207

[jepio]

For the most part done, we'll create smaller follow-up issues when we tackle the remaining items.