Explicit dependencies

[metanomial]

Is Your Feature Request Related to a Problem? Please Describe Mounted dependency drives—frontends, web components, modules, databases, etc.—don't scale well. If common users aren't typically visiting a mounted dependency and deciding to host it, there is a good risk that the dependency will be significantly less available than the parent drive requiring it.

Solution You'd Like Mounted dependencies may be explicitly listed in the drive index to adjust hosting behavior:

index.json

{
  "dependencies": [
    {
      "key": "...",
      "auto": true,
      "recursive": false
    }
  ]
}

Automatic dependencies will be automatically hosted along with the depending drive. This is ideal for files like frontends and modules.

Manual dependencies will not be automatically hosted, but an extra slider for each will be added to the hosting drop down panel. This is ideal for dependencies only used by a subset of users, or really large dependencies that most users will not want to host.

Dependencies flagged as recursive will be allowed to add additional dependencies from their own index.json. This is ideal for long module import trees or syndication drives.

Alternatives You've Considered A Hosting API to prompt the user to host an external drive or set of drives. It would be a more flexible system, but would be a little clunky.

[matthewauld]

I might be wrong about this - but I think the solution is to mount the dependency in your project. It also has the advantage of preventing duplication - if 100 projects you are hosting mount drive y, you only actually have one copy of drive y.

If you are sharing a drive, you will want to share the dependencies too, because, an application without the dependencies isn't very useful as you say. Also, I seem to remember reading somewhere that the mounted drive can be transmitted as part of the same connection as the parent drive, without a separate request, making it speedier (beaker/hyperdrive people would have to confirm).

[metanomial]

Oops, I'll edit the post to clarify. I'm referring to mounted drives here. This feature request just allows you to modify the hosting behavior of mounts when hosting the parent drive.

[rhythnic]

On topic, but not something that would go in the beaker code base, it would be nice to have common hyperdrives for developing modules. Right now I've been working off of @pfrazee 's module examples. I love the overall setup, but given that mocha.js and the .ui dependencies are included and not mounted, there will be a lot of redundancy on a user's machine, unless mounted hyperdrive's files are loaded lazily. Does anyone know of hyperdrives to help with module development and publishing?

[metanomial]

@rhythnic mounted drives are only pointers, and they are loaded lazily and not hosted along with the parent drive. I'm hoping there will be a way in the future to pin mounted drives to the parent drive so both are hosted together.

Also, see Paul's UI kit: hyper://8db7ce989b9b11072646c39f8fdf50cfb2dc357a27a4bc03ebd8cf5056e446c9. Is that like what you are asking about?

[rhythnic]

@metanomial I was asking more about hyperdrive's that would be considered development dependencies. I'm using the .ui folder inside of Paul's Worker RPC module, which provides a UI for running tests and a demo, along with a well formatted index.md beneath the file list. It would be good to have that as a hyperdrive that could be mounted into modules. The mocha/chai code in the test folder would also a good candidate for a a development-dependency hyperdrive. This isn't very on-topic with your issue, so please excuse the tangent.

Paul's Worker RPC Module hyper://20852326dcc4eb10267dda3f7606e54ce120a7c2d73a976d58bd8f3fd117e8c2/

Getting back to the issue. I agree with you that when hosting a parent drive, it seems reasonable to host the files in mounted drives that have been loaded. That, in addition to lazily loading mounted files, seems like it would cover most use cases, including code dependencies. I think dependencies large enough for anyone to care whether or not they host are likely to be data sets. Data sets needed for all users of an application should be mounted, and like code, implicitly hosted. Most data sets will probably be in a separate drive from the application.

So to summarize, I agree that mounted drives, lazily loaded, should be implicitly hosted. Where I differ in opinion is that I don't see manual configuration as necessary, unless it's the only way to get the dependencies hosted without user interaction. The idea of a user selecting which mounts to host seems targeted to applications mounting data that isn't necessary for all users. In that case, I believe that data shouldn't be mounted, but instead intentionally hosted by the user, and acted on by the application.

If mounts are lazily loaded and implicitly hosted, and in the event that there's an application mounting optional data, I think it's reasonable to expect a user of that application to delete the parent drive and reload it if they've accessed optional data that they don't want to host. I say that because I believe it's an edge case, but it's very possible I'm not seeing the use case you have in mind.