search

filecoin-project / venus

Filecoin Full Node Implementation in Go
https://venus.filecoin.io
Other
2.06k stars 459 forks source link

Handle Repo Migrations when upgrading from go-cidv0.0.1 to go-cidv0.0.3 #3133

Closed frrist closed 5 years ago

frrist commented 5 years ago

Description

3125 upgraded go-filecoin to use go-libp2pv0.2.1 this caused go-cidv0.0.1 to be upgraded to go-cidv0.0.3. This change has broken the repo migration test TestMigrateSomeRepo#Happy path: valid migration passes validation for repo migration repo-1-2 since go-cidv0.0.3 uses base32 encoding instead of base58. Addressing this issue is outside of scope for #3125 and since #3125 is blocking graphsync work, we (@shannonwells and @frrist) have agreed that it will be easiest to skip this test until a better solution can be thought of.

Acceptance criteria

anorth commented 5 years ago

This is a nice example of the potential difficulty involved in maintaining migrations across certain types of changes. In this instance, if the problem boils down to the fixture repo being no longer readable, I think we can delete this test since we don't actually need the migration to be functional: it's serving as a demonstration. Vendoring the CID or low level datastore library isn't worth it here.

cc @ZenGround0

ZenGround0 commented 5 years ago

If the fixture repo is no longer readable doesn't that mean we should remove the migration entirely because our migration tool won't work on it anymore? For sure maintaining this migration is very low value in our current reset-the-devnet-release environment.

For those reading along we're going to eventually need to handle the complexity by either (a) having some process for deleting old migrations and managing communication of deprecating migrations with people running our software.

or

(b) vendoring our migrations.

(a) is good because our current assumption is most people are going to stay fairly up to date and so migrations can die without disrupting users. This way we can avoid the upfront costs of solving the engineering challenges in (b) and focus on other things. It will also make writing a migration a bit easier.

(b) is good because we don't need any process for handling any communication or development snags (like the dev snag that raised this issue). All complexity will be handled by the writer of the migration. Migrations won't fail test due to upgrades and will always be runnable so that the extent of our communication with users will be "look at migration docs and run a migration".

shannonwells commented 5 years ago

If the fixture repo is no longer readable doesn't that mean we should remove the migration entirely because our migration tool won't work on it anymore?

If we went with plan a) then, if this were anything but a "demonstration" migration (despite it having actually done something real), then yes it would mean that, right?

If I understand what you mean by vendoring a migration, then every migration is a separate project?

ZenGround0 commented 5 years ago

If I understand what you mean by vendoring a migration, then every migration is a separate project?

I'm thinking of something smaller scale that looks a lot like the ipfs migration tool. If we did vendoring then I imagine that each migration would just be subdirectory/subpackage of the migration tool which would become its own toplevel project outside of go-filecoin . Because gomod is no use for importing two arbitrary versions of the same project we would have to use something else to write even a single migration (gx seems like a reasonable tool for this job).

anorth commented 5 years ago

If the fixture repo is no longer readable doesn't that mean we should remove the migration entirely because our migration tool won't work on it anymore?

It's still a useful demonstration. It's for humans to read, not computers. I really don't think we need to do anything here.