Do not expose node builder internals to the consumer

[schomatis]

This basically means not allowing (or at least disincentivizing) direct manipulation of the DI graph through node.Override() as we currently do:

https://github.com/filecoin-project/lotus/blob/476df99179157c9d0019d2bcc9a9a4fb6b31d876/cmd/lotus/daemon.go#L280-L305

In the previous example this would mean (among others):

• Avoiding boilerplate node.Online()/node.Repo(r) configuration calls (and their strict calling order that the consumer shouldn't know or care about) which should happen by default and just provide a repo argument (or functional option).
• Not needing to explicitly provide the api.FullNode to "extract" the created artifact of interest but having the API returning it directly.
• Replacing generic Override calls by explicit configuration options for the bootstrapper, genesis and other usually modified dependencies (i.e., white-listing what the consumer is allowed to change).

I'm fine with:

• Retaining the functional options paradigm, but those options should have an intermediary abstraction, say, BuildConfiguration instead of providing direct access to the DI graph.
• Retaining the current node.New() API for internal tests in the node package which needs more advance configurations, but if some external consumer wants to access this low-level API it should be under the explicit assumption that it is manipulating the logic internals and that it is subject to change at any time.

The objective is to decouple the node/ logic from the rest of the code to start cleaning it up and provide a more fine-grained but standardized interface to initialize only the subsystems needed during testing.

[raulk]

I'm onboard with not leaking internals via the API, and obsoleting the need to directly manpulate the DI graph for simple configuration changes.

In practice, I think we do need an escape hatch so that code that needs "privileged" access to the node construction can achieve so.

Passing in an optional callback function to perform Overrides and such, should be good enough for that. An examle of such a privileged user would be testground test plans.

[raulk]

Replacing generic Override calls by explicit configuration options for the bootstrapper, genesis and other usually modified dependencies (i.e., white-listing what the consumer is allowed to change).

I'm a bit +1 on this, as long as we preserve the escape hatch I mentioned.

[raulk]

Not needing to explicitly provide the api.FullNode to "extract" the created artifact of interest but having the API returning it directly.

One nice feature of the current model is that you can extract components that are not necessarily exposed through the public API. We need to preserve that capability, either by exposing those components through getters, or by supplying an extractor callback.

[schomatis]

One nice feature of the current model is that you can extract components that are not necessarily exposed through the public API. We need to preserve that capability, either by exposing those components through getters, or by supplying an extractor callback.

I think we are aiming at the same thing. Just to clarify, I'm not sure we can freely extract stuff at the moment, not the way the current DI model is architectured. I don't see people being able to instantiate for example an isolated state manager or chain storage for a test (that is exactly my motivation for this issue in the first place, https://github.com/filecoin-project/lotus/pull/5313/). If there is something like what you are describing please link it, I might be able to use it as a model and expand on that.