Structure for standalone components

[jeffmcaffer]

Many of the UI components we are building (e.g., license picker, source location picker, ...) are likely interesting to others. Certainly people looking to embed ClearlyDefined flows into their products or flows would find it useful to be able to reuse our components.

Here we should consider the options for both development and delivery.

1. One repo with many components producing one npm
2. One repo with many components producing multiple npms
3. Separate repos for each component producing separate npms

If we # 2 there will also be the question of which mono-repo structure do we use (e.g., rush, lerna, ...)

[jeffmcaffer]

As a suggestion, lets keep it simple and do #1 (one repo, one npm) and just make it easy for people to get the one component they need. This is more or less how react-bootstrap works. The only problem with this is where any of the components have non-trivial dependencies. For example, a github source location picker may call GitHub APIs. We should try to keep the dependencies to a minimum. We can do that though pluggability (e.g., users plug in a configured GitHub data object that can do the calls). Just need to factor that into the design.

[AlexWebYourmind]

@jeffmcaffer Thanks for the additional information.

To clarify, the deliverables for this issues are:

• New CD Repo
• Initial NPM Package Deployment
• Documentation (e.g. Readme.MD outlining the structure, how to use it etc)

Can you confirm the above is correct?

We can then open separated issues to e.g. make an existing component standalone (if not already) + add to new repo.

Thoughts?

[jeffmcaffer]

@AlexWebYourmind that is more or less right. I was chatting with @valtterihalla about #179 and thought that them adding the new source location picker would be a good opportunity to scope out a design for this new component structure. We can them move over any existing generic components as needed.

Hopefully @valtterihalla and team can put together some proposals and we can iterate and then implement.

[daniellandau]

Is the idea that users of components have to provide their own API endpoints for example for Github search, or should they call endpoints in ClearlyDefined? The ClearlyDefined API already does CORS allowing all origins so in principle e.g. GithubSelector.js and GithubCommitPicker.js are publishable almost as is, as they do not plug in to any redux state management and their only dependency to anywhere else in the project is to get the URL to shoot requests at.

These end points in the API are wide open to the whole world already, but of course exposing easy React components to NPM would probably increase their usage. This might then be either a good or a bad thing, depending on what kind of agreement we can get on Github (and other, Gitlab, Maven, NPM, etc) quotas.

That's for read-only and external-only API calls. What about actual ClearlyDefined data and embeddable components contributing back to ClearlyDefined? I think for those it would be quite necessary to just directly call api.clearlydefined.io.

Maybe we could just do all the calls to api.clearlydefined.io but really clearly mark the NPM packages as experimental with no guarantees of continuing access to the API.

[jeffmcaffer]

I'd rather not have the world funneling through ClearlyDefined to do these API calls. My thought was that we define a simple API (much like we have now for the "origin" calls from the client) and then people plug in a prop that has those functions. That way the component is complete generic and users can use whatever GitHub (for example) client they want preconfigured with auth etc as needed.

In our UI we would plug in something like we have now.

[jeffmcaffer]

@daniellandau did some nice work prototyping a direction for separating out the components but ultimately this is a very complicated and subtle space and people have already spent huge amounts of effort figuring it out. We could do something that works today for the UI but it would be better to head in a direction that could work for all the different repos or even merging all the repos into one in the future.

@storrisi did a quick poke ant Lerna and it was promising. It is a pretty fundamental change though so we should do it deliberately.

[storrisi]

The test i've made on Lerna gave as a result that Lerna is the right option for structuring a mono-repo application that could also maintain shared components with their own package management.

Seems to be a pretty good and powerful solution, but unfortunately requires the whole project to be managed by Lerna itself, not only a specific folder for standalone components. It means re-structure the entire src folder and probably update all the build and run process.

Meanwhile, while working on other components, i've structured them to be ready to a standalone refactoring, eventually.

@jeffmcaffer what do you think should be the direction for this purpose?

[dabutvin]

what do you think about creating a new github repo like https://github.com/clearlydefined/components and then setting up the lerna solution there?

[storrisi]

Well, the purpose here was to have a monorepo rather than multiple ones. Lerna exists to manage these kind of situations, using a separate repo it's much more like a classic library, which could be added to the project via a simple npm install

[dabutvin]

I meant 1 monorepo that contained all the components, not a repo per component.

I think there would be some value in developing these components outside the context of the website. what do you think?

[storrisi]

I meant 1 monorepo that contained all the components, not a repo per component.

This was already discussed with @jeffmcaffer , but the intention here is to develop all of these components in the same website repo.

To do that we have different options:

1. use Lerna to manage internal packages of shared components, with a separate npm library, but we have to refactor the entire project and let Lerna managing it
2. use a custom flow to achieve the same thing that Lerna does, but feels like reinventing the wheel and hard to maintain
3. considering a guide line of structuring these components, see the work done for the License Picker in #400

I suggest that at this stage the 3rd option could work for us. Giving guide lines on how to prepare these components would allow us to pick a solution in the near future when necessary.

Thoughts?

[dabutvin]

Agreed. We should move forward with option 3 for now so we can be ready

[storrisi]

Given that, we could maybe close this issue and eventually reopen it when it will become necessary again

[jeffmcaffer]

fine closing this. Note that my desire is to have ALL of the UI in one repo, not a separate repo for components and one for the app itself. If that is not possible or is hard then I'll live with it .

[dabutvin]

Okay. It's probably time to re-activate this.

Let's go ahead and go for the restructuring required to get lerna in here