Dependencies for the integration of FVM with HC into Eudico

adlrocha commented 2 years ago

This issue is an attempt to give some structure and clarity on the different repos and dependencies involved in the integration of HC with FVM.

ref-fvm is the reference implementation of FVM. For HC, our current fork includes Rust implementations for the hierarchical address, subnet IDs, the inclusion of a new type for SCA and some modifications to the test_vm.
builtin-actors is our custom builtin-actors bundle including the implementation of the SCA.
hc-subnet-actor is the reference implementation of the user-defined subnet actor. To be deployed and integrated in Eudico it'll need FVM-M2. So far, some basic tests and integration experiments can be done using the experimental/fvm-m2 branch.
filecoin-ffi is the core dependency to integrate ref-fvm into Eudico. We need this package to point to the right fork of the ref-fvm to work properly with our customizations to the FVM.
eudico, where all the HC code lives. For the integration of FVM we'll need to adapt to the modifications introduced in the Rust implementation of the actors, and to load our custom builtin-actors bundle.
go-address holds the Go implementation of the hierarchical address and SubnetID.

Updates on the state of the integration and changes in the dependency tree will be posted to this issue. Additional info about the integration can be found here

Why trying to make progress in the integration using `experimental/fvm-m2`has shown to be a nightmare

In order to integrate the subnet actor as a user-defined actor we need FVM support for the deployment of user-defined actors (FVM-M2). The experimental/fvm-m2 branch in lotus gives experimental support for the deployment of user-defined actors. We managed to merge it into eudico and test the deployment of a subnet actor, but performing the code changes required for the full integration of FVM with HC and Eudico has been hard due to the dependency dance required:

experimental/fvm-m2 in Lotus leverage a branch with the same name in ref-fvm, filecoin-ffiand builtin-actors.
In the meantime, we keep a forks of ref-fvm, builtin-actors and filecoin-ffi which are periodically rebased with master were we are adding all the HC-specific code required in repos.
On the other hand, other teams in CL and introducing new changes and features over eudico (which is a fork of lotus:master).
This means that in order to be able to maintain an up-to-date experimental/fvm-m2 branch in eudico we need to:
- Keep experimental/fvm-m2 branches of builtin-actors, ref-fvm and filecoin-ffi that we periodically rebase with its experimental/fvm-m2 counterparts and the adlrocha/* forks to include the latest features.
- Periodically rebase with lotus:experimental/fvm-m2 while in parallel we rebase eudico:eudico with new features.

All of this is imposing a "fork-maintenance" overhead really hard to handle. Consequently, we should probably freeze the integration until we have a release candidate in master to make the "dependency-dance" more tractable.

aakoshh commented 2 years ago

It looks like the arrows are reversed, at least in some cases:

filecoin-ffi ... need this package to point to the right fork of the ref-fvm

I would also expect builtin-actors and hc-subnet-actor to depend on ref-fvm, right? Shouldn't they point upwards? Similarly I thought eudico would point at go-address.

aakoshh commented 2 years ago

I'm not entirely clear on why hc-subnet-actor can't be a built-in actor, they seem like two sides of the same coin. In what sense is this a reference, how would another implementation be different from it?

adlrocha commented 2 years ago

It looks like the arrows are reversed, at least in some cases: I am using the arrows to describe what repos use what (maybe the description in-text is a bit misleading).

ref-fvm is used by filecoin-ffi, builtin-actors and hc-subnet-actor.

eudicouses filecoin-ffi(and thus, implicitly ref-fvm) and go-address. Let me know if this makes sense (I can edit the diagram or the text to make it more clear).

I'm not entirely clear on why hc-subnet-actor can't be a built-in actor, they seem like two sides of the same coin. In what sense is this a reference, how would another implementation be different from it?

A builtin-actor should be an exception, everything that we can should be a user-defined actor (actually, I am afraid we may even need to make a really strong case to justify why SCA needs to be a builtin-actor). Regarding ways in which the subnet-actor could differ: you could support different checkpointing signatures for a subnet, prevent its registration until a minimum number of validations has joined, etc. The idea is for the subnet actor to be an interface (analogous to ERC20) that users can implement with their custom logic.

adlrocha commented 2 years ago

Update: We may not need a specific for for FVM-M2 any more once https://github.com/filecoin-project/ref-fvm/pull/658 and https://github.com/filecoin-project/builtin-actors/pull/491 are merged.

filecoin-project / eudico