"Headless" ICF - API access to functionality

[ahpook]

Is your feature request related to a problem?

Some users are working to integrate ICF into a larger "open source portal" context, where they already have a web app and additional tooling around OSS contributions (like which projects are available for contributions, who is allowed to contribute, etc). They'd like to make use of the internals of ICF - the mirror lifecycle management and synchronization - but wrap this in their own web service.

Describe the solution you'd like

The goal would be to make ICF "embeddable" in other applications. @ajhenry brainstormed a couple of options like creating a RPC client, making the app "import"-able as a package, or having a separate web service from the front end which could be called by another app. Not sure which has the highest effort/reward and would best meet the user need, so this requires more exploration and thought.

Describe alternatives you've considered

No response

Additional context

No response

[Miablo]

Some users are working to integrate ICF into a larger "open source portal" context, where they already have a web app and additional tooling around OSS contributions (like which projects are available for contributions, who is allowed to contribute, etc).

For Private Mirrors to have a 'headless' API functionality some of the workflows would change when the currently coupled UI is replaced with the user's larger 'open source portal', right?

Would this issue describe the creation of a stand-alone API that can then be onboarded internally by a team who does have that internal facing portal? The assumptions made here are that the user would:

1. have a db or some system of record of who is allowed to contribute
2. have a db or system of record of what projects can be contributed to
3. have a list of licenses or other parameters that will need to be checked prior to allow a contribution be made to a repo
4. require new user registration would be handled by the existing 'open source portal'
5. have SSO integration would be used instead of the OAuth w/ GitHub method used by the current application to verify the user's identify and then verify their access to the various orgs

Currently there are four routers:

1. Config
2. Git
3. Repos
4. Octokit

I would assume that these would become the end points or callable routes in this API? Would some additional functionality be useful for audit and/or compliance checks? (--Just starting to think out loud here--)

[ahpook]

@Miablo thanks for the write-up!

For Private Mirrors to have a 'headless' API functionality some of the workflows would change when the currently coupled UI is replaced with the user's larger 'open source portal', right?

Yes I think that's exactly right, and as you say at the bottom the different routers would become callable separately, just to execute the functionality they are responsible for, and the larger user auth/authz would be externalized since it's likely different for everyone.

[Miablo]

@ahpook What do you envision for this implementation? Are you ya'll locked in on using tRPC or would you all be open to using something like:

1. DeepKit
2. NestJs withfastify
3. or anything else that may be a good fit

I'm trying to think through the scope for this specific issue and it would make sense to have one issue (this current one) to create the API itself and then other new issues, if ya'll want, to create a solo component for the UI and a solo component for the web hooks.

I see priviate-mirrors as, at least, 3 potential products:

1. private-mirrors full app (UI, API, Webhooks) -- current state
2. private-mirrors API
3. private-mirrors Webhooks

This would then create separate consumable components of private mirrors or provide the full functioning app as well as an option for consumers to pick and choose their solution.

What are your thoughts?

[ahpook]

Hi Mio, I am but a simple product manager and don't have implementation opinions. I do think it'd be great to have small implementation steps that iterate towards the end state, and if creating separate issues to track that works best for you, go for it! 👍

Splitting it out as you describe is fine with me as long as there's an easy way for people to get started. The most common use case would be someone who wants the full app as an 'all in one' setup, so installing the UI should pull the other pieces in.

[Miablo]

Thank you, thank you for the keys to the kingdom! 😁 I will create a proposed API design doc / diagram to toss in here for review.

[Miablo]

@ahpook -- we just had a design session internally and talked about how for our initial pilot to use PMA we, OSPO, will be manually creating the forks for our testers. Because of this I wanted to propose adding a /forks endpoint to this new PMA API. The idea is that we can GET, POST, PATCH/PUT, or DELETE forks using the endpoint. More specifically though I wanted to focus on the POST method where we would create the fork.

I created a draft of a diagram to show how this endpoint could run through a few validations before creating the fork. The initial assumption is that if this create a fork button or whatever is used to trigger this endpoint is enabled then the user has successfully passed authentication checks (logged in to their gh account, etc.).

I wanted to get the conversation started regarding this endpoint. The area that is encapsulated in the diagram houses additional checks we could do as part of the fork creation request.

One big one is the license check and to block or prevent contributions from being made to projects who don't have approved licenses.

I would like to make these checks module somehow where a company can input a list or some other data source to use for approved licenses or approved projects they may have, etc.

Thoughts?

[ahpook]

One big one is the license check and to block or prevent contributions from being made to projects who don't have approved licenses.

This seems great to me! Do you expect the policy checks in the Fork exists -> No branch to be configurable or extensible? I.e. how would someone else with a different set of policies implement them?

Might be of interest: the finos/git-proxy project had developed a really similar concept of policy plugins - https://git-proxy.finos.org/docs/development/plugins

[Miablo]

Do you expect the policy checks in the Fork exists -> No branch to be configurable or extensible? I.e. how would someone else with a different set of policies implement them?

Yes! That is how I imagined it in my mind, but wasn't sure how to implement. I wonder if I can create some example or common custom checks that could be used by others as a template using git-proxy? I need to look into it more and see how git-proxy would work, but looks promising!

[Miablo]

https://github.com/finos/git-proxy/issues/594 <--- Looks like someone from Citi is working on adding the license check plugin 🥇