Manually tested against minio (s3 compatible), gcs (thanks dylan) and a temporary directory.
Setup
Build subo 5.4, start compute creates some functions in the repl shut it down - storage root "./old"
Install subo 6, start compute, create some function in the repl shut it down - storage root "./new"
Build PR branch exec se2 migrate [storage root]
Verify contents
In SCN I wrote some adhoc tests using the storage instances and pointed them at my test storage to confirm i could list tenatnts, find tenants, read the system value etc.
How it works:
Walks storage and marks all assets of interest in a trie
a. each asset is placed under a category root (tenant, runnable, bin, src) and points to the underlying resource
Process each module binary so we can obtain the new ref (sha256 digest), create the new directory (ref) and copy the binary to its home. During this time we record the ref and place it in the tree in a new category (digest).
Next we process the drafts, this requires looking up it's associative ref from the tree then creating a new directory, and finally copying the draft contents into that directory.
Next we process all runnables which are translated into "modules" and recorded at the module rot directory.
Next we need to collect all modules we process and record them in a tenant overview which is then placed at the tenant root directory.
Finally we create a somewhat superficial system version number and place it in the system storage root. This mostly just needs to be greater than 0 so e2core can see that there are modules there for it to discover; subsequent deploys will increment this appropriately from there.
I decided to leave clean up as an exercise for the user to ensure they can gracefully recover in the event something terrible happens which didn't reproduce in my tests.
Testing:
Manually tested against minio (s3 compatible), gcs (thanks dylan) and a temporary directory.
Setup
se2 migrate [storage root]In SCN I wrote some adhoc tests using the storage instances and pointed them at my test storage to confirm i could list tenatnts, find tenants, read the system value etc.
How it works:
Walks storage and marks all assets of interest in a trie a. each asset is placed under a category root (tenant, runnable, bin, src) and points to the underlying resource
Process each module binary so we can obtain the new ref (sha256 digest), create the new directory (ref) and copy the binary to its home. During this time we record the ref and place it in the tree in a new category (digest).
Next we process the drafts, this requires looking up it's associative ref from the tree then creating a new directory, and finally copying the draft contents into that directory.
Next we process all runnables which are translated into "modules" and recorded at the module rot directory.
Next we need to collect all modules we process and record them in a tenant overview which is then placed at the tenant root directory.
Finally we create a somewhat superficial system version number and place it in the system storage root. This mostly just needs to be greater than 0 so e2core can see that there are modules there for it to discover; subsequent deploys will increment this appropriately from there.
I decided to leave clean up as an exercise for the user to ensure they can gracefully recover in the event something terrible happens which didn't reproduce in my tests.