zos bootstrap rewrite

[xmonader]

should be for later

[muhamadazmy]

If we gonna switch to use new fl format this can be a good chance to rewrite the entire bootstrap to embed and use the new fl library.

This also will depend on the new flist server

[muhamadazmy]

Here you can find a basic idea why this issue was created

Why a rewrite is needed

Current bootstrap process is strongly coupled with the HUB. when built in bootstrap process starts it first update itself from latest build from the hub. Then it downloads necessary packages from the hub. This works fine for now but it has the following drawbacks:

• All nodes on boot are using a "central" service to download all packages
• Bootstrap is using external binary (rfs v1) to download the packages. this can't be embedded in the bootstrap binary because rfs is using an async runtime while bootstrap is not.

Suggestion for the process:

• Bootstrap binary package still need to be hosted on a known location so nodes can update it while booting.
• Once bootstrap is updated it needs to find best location to download the .fl files for the boot. Right now it uses the hub, but if we going to move completely to rfs v2 we can't do this anymore.
  • A public mirror list can be hosted on CDN
  • A well known mycelium addresses (do we need to run mycelium on boot?)
• Once the fl files are found, the rfs code can take care of downloading the binaries
• use embedded rfs v2

NOTE: this is not an urgent issue, it requires a lot of planning. since we need to know what stores are we gonna use, plan package and geo replication of stores