Code reorganization for readability and reusability

[gammazero]

This issue records proposed changes to make the reference provider more easily consumable by parties wishing to implement their own provider based on the reference provider, and by reusing packages from the reference provider.

• Remove provider subdirectory in cmd/provider
• Move engine/linksystem into a separate package
• Consider moving engine/engine.go to top-level directory.
• Expose some of the packages in internal
• Move engine/dscache.go into separate pacakge. Possibly move it into a package under go-datastore.

[masih]

Remove provider subdirectory in cmd/provider

This is a convention followed elsewhere. Examples include car, Ipfs, k8s.

I am curious why we should remove provider subdirectory here?

[masih]

One item I had in mind was to refactor daemon.go command and move the graphsync setup/start out to a package, probably under server/provider/graphsync.

[gammazero]

• The reason to remove provider subdir was because I did not see any reason for it as there are not other subdirs there.
• Putting graphsync into server/provider/graphsync is fine, but consider if this is a package that is useful to outside applications, and maybe not strictly as part of building a server. If it is not specific to server functionality, then maybe it should be a top level package, or maybe should be a package inside a network directory.

[mvdan]

Expose some of the packages in internal

Now done in https://github.com/filecoin-project/index-provider/pull/93.

[mvdan]

Remove provider subdirectory in cmd/provider

I lean against doing this one; see https://github.com/filecoin-project/index-provider/pull/93#issuecomment-970100889

[willscott]

is there more to do here?

[gammazero]

This is done for now.