SPEC: Non-indexed Shards

[caspiano]

Hi! Could we incorporate a top-level key indicating that the repository is private/not intended for use as a shard? When a shards crawler encounters the shard.yml, it would only index the shard iff the file declared private: false or lacked the key.

This change would prevent apps/projects that are not intended for use as a library cluttering up registries while also remaining open source.

P.S. I am aware of the discussion in #283 but I feel like my request is more specific :^)

[straight-shoota]

I support the general idea, but I don't think a private property is a good approach. For example, for shardbox.org it doesn't make any sense. The shard won't be listed unless it's added to the catalog (which is equivalent to private: false) or it's a dependency of another shard. In that case, private: true doesn't make senes and would be ignored because when used as a dependency of registered shard, it can't be considered private.

Maybe a better solution could be a more versatile property to determine the type of the project. Shards currently doesn't differentiate much in this regard. The existence of shard.lock can be used to tell if it's an application. But that's somewhat unreliable because there's nothing stopping libraries from having a lock file and there might even be good reasons for that. So the idea could be to have a project type property with values like library, application, private for example.

[waj]

@straight-shoota defining the type of project is a good idea, but I don't think it necessarily have to express the willing to opt out of indexers. Those are ortogonal properties.

I'd prefer to have a setting that express the intention more clearly. Adhering to something standard like robots.txt could be an option. And it could even allow a shard to be indexed by some crawler and not by others. If that's too complicated, we could adopt a similar term and use robots: false as the setting within shard.yml.

[straight-shoota]

I know this is a different angle. But it seems to solve the actual use case @caspiano described: "indicating that the repository is private/not intended for use as a [dependency]".

And I honestly don't see a valid use case for specifying discovery restrictions in shard.yml. If you don't want your code to be publicly visible, don't put it in a public repo on GitHub.

How shard repos are interpreted by crawlers is an entirely different story. If the shard.yml states that it's not intended to be used as a dependency, it should not be listed as such. But application-only repos can for example still be valuable for collecting usage metrics on its dependencies. Private/testing repos on the other hand should probably be just completely ignored.