search

opendevstack / ods-provisioning-app

Provisioning app, which triggers project and component provisions (including Jira / Confluence / BitBucket and OCP resource creation)
Apache License 2.0
15 stars 20 forks source link

Better integration with Nexus #691

Open serverhorror opened 3 years ago

serverhorror commented 3 years ago

feature request

I'd like to see better integration with Nexus. Currently local usage of nexus:

We are looking at local usage of Nexus because we want to be able to run builds and test locally with an environment that resembles the CI environment as close as possible.

solution proposal

I see 2 base cases

  1. create a new project When a new project is created the corresponding groups should be created to or added in nexus so that the personal credentials can be used rather than sharing a secret between a potentially very large group. Not to mention all the hassle if people switch teams, this requires a secret rotation for a lot of people.

    • manager (possibly read/write?)
      • CI -- I'm not sure about this, I'd much prefer if CI was the only thing that could write to the repository
    • team
    • stakeholder

    groups should be added to nexus as read-only member of all "public" nexus repositories but not necesarily all repositories (although I can't see harm in that right now)

  2. create a new component within an existing project When a new component is created I would like to an option to create a Nexus repository.

    The reason for not doing that directly at the project level is that we don't know ahead of time which kind of repository is required for which component.

A perfect experience would be if the repositories are magically added to the "meta repositories" (lacking the proper term here) so that when Nexus is configured as the default Maven, PyPI, R, ... mirror all internal packages are available automatically.

alternate/current methods

Manually ordering the repositories is our only option right now. This has proven to be too cumbersome and error prone.

People actively avoid using Nexus as a means to distribute software as the process seems unclear and complicated. We are already seeing several implementations of "I pretend to be a repository but I'm not" in the wild and in use by various high profile projects.

additional context / required praise

❤️ You girls and guys rock! ❤️

stitakis commented 2 years ago

@serverhorror first, my apologies for the late feedback on this! Second, excellent idea. I'm for everything that empower developers to make their work more efficient and effective, like in this case, providing out of box nexus repositories for sharing libraries and other artifacts. Additionally I believe this enhancement could save effort for the platform supporters in the enterprise side.

@michaelsauter @clemensutschig @metmajer I vote for this enhancement. Anything from your side that would speak against it?

michaelsauter commented 2 years ago

I see the two use cases you mention as two separate requests.

The first one ("1. create a new project") has been discussed in the past, see https://github.com/opendevstack/ods-core/issues/321 and https://github.com/opendevstack/ods-core/issues/169. Work on that was stopped because it proved to be tricky for a variety of reasons. In general I would highly welcome progress on this as well, the shared secret is not a viable option I think.

The second one ("2. create a new component within an existing project") is new as far as I know. I have a few questions on that one:

serverhorror commented 2 years ago

The first one ("1. create a new project") has been discussed in the past [...]

I see, I'm looking at this as a very naive user. The "project" to me is some kind of namespace and I'd love to be able to add/remove project members and know that access is handled.

I think you're right that should probably be a proposal in itself. I'll stop mentioning that stuff here.

  • Is the prov app the right place? Right now it has no admin rights within Nexus.

In my head that's the place I'd go to if I wanted to add "stuff" to my project. That's just me and I might be wrong, maybe asking around does make sense?

  • Do we need a Nexus repo for each component, or would a repo per project suffice? (meaning a 1:1 relationship between Bitbucket project and Nexus repo)

TL;DR:

I think it would be nice to have an option at creation time and after the fact to just say:

Hey! This repo (component) now needs a place to put packages. It'll be of type XXX

Background info:

I can't speak for every project. The situation I am in is that we have a lot of projects that are "multi-language and multi-purpose".

You can think of it like "this stuff goes together well so I'll just put there". Could be pipelines, R packaes, Python packages, ... all in the same project and they do not form a coherent single product. It's a collection of products that are under a single pane of glass.

The reason why people choose this is quite pragmatic.

  • Is component provisioning the right time to create a Nexus repo? Could the need for this be unknown during provisioning, and discovered later on?

I am absolutely sure that this will come up after a repo has been created. I was thinking about the prov app because it would be the first place to look for. In my head it's a little like this:

  • I am provisioning a component without a Nexus repo
  • I am provisioning a component with a Nexus repo
  • I am provisioning a Nexus repo for an existing component

The case I am not sure about is "Provisioning a Nexus repo without a component", I have no good mental model about this. Maybe one could provision multiple Nexus Repos per component? Say "git lfs and Python Package" or "raw and CRAN"