apache / incubator-devlake

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
https://devlake.apache.org/
Apache License 2.0
2.55k stars 508 forks source link

[Feature][Deployment] Multy deploy in a single pipeline #6959

Closed antoniomuso closed 1 week ago

antoniomuso commented 7 months ago

Search before asking

Use case

My software in a single deploy pipeline can have deployment jobs for more than one product. This can happen when the application is a multi-tenant.

Description

The regex used in the scope config, even if it matches a specific job, assigns the tag deployment to the entire pipeline. The feature should allow the count of a single job as deployment or to mark the job matched by the scope config regex.

Related issues

No response

Are you willing to submit a PR?

Code of Conduct

klesh commented 7 months ago

Sounds interesting, could you share more concrete examples (fake data maybe) to help us understand the problem better? Thanks.

antoniomuso commented 7 months ago

Hi, @klesh. Thanks for the answer. I have a project, for example, let's call it anime that has different deploys for each region, anime.com/anima.us, etc...; the project in my organization is a single Gitlab repository, and when I do I release, I create a single pipeline where there are n manual deployment jobs one for each tenant. I want to track each job as a single deployment. Then, for example, if I have two tenants, anime.com and anime.us, I wish to obtain two deployments. I haven't found a way to do it with devlake without using a regex on SQL. The possible solution I suggest is to extend the domain data model, adding to the cicd_tasks table a column match that is true when the regex passes as input in the scope configuration matches the job.

klesh commented 7 months ago

Thanks for the thorough explanation. loop @Startrekzky

hezyin commented 5 months ago

Here's another data point contributed by a DevLake user:

Untitled-2023-09-13-1844

They release a new "cluster chart" at regular intervals, but enterprise customers, each with unique change requirements and schedules, may not upgrade immediately.

As shown in the diagram, they serve multiple customers. Each customer may have one or more management clusters, primarily for georeplication. Each management cluster oversees multiple workload clusters (corresponding to different development stages like dev, staging, prod).

This DevLake user's primary concern is to minimize the number of distinct cluster chart versions in circulation to reduce maintenance costs. They track the time from a new cluster chart release to its deployment across most workload clusters. Additionally, they need to segment deployment data by customer, management cluster name, type (WAS, Azure, VMware, bare metal), and workload cluster name.

To measure the percentage of workload clusters on a specific release version, we need to know how many workload clusters are there in total. We can approximate this number using the total number of workload clusters we have seen in the past.

Startrekzky commented 4 months ago

@hezyin Since the use case you brought up was slightly different from the original feature request, I created a new issue #7415 to track it.

github-actions[bot] commented 2 months ago

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] commented 2 weeks ago

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] commented 1 week ago

This issue has been closed because it has been inactive for a long time. You can reopen it if you encounter the similar problem in the future.