RPU GSoC project with terraform integration on infra's backend

[NotMyFault]

Service(s)

Other

Summary

ref https://github.com/jenkins-infra/repository-permissions-updater/pull/3998/files/1a7ce9fc6d849f5b42fe672696b9d6b7797c45b7#diff-52bce356bec038d45407bbdde669ca1dd0f9efb635eaf2d8debffc1ba1705eea and the comments.

Hey everyone,

Regarding the comment above, what would be needed from our side to achieve this goal and have it run on our infrastructure?

@alaurant fyi

Reproduction steps

No response

[krisstern]

@Alaurant

[dduportal]

In order to start the discussion, we might need a bit more context and detail. I'm not sure to fully get the picture with the only the linked comments :|

[krisstern]

c.c. @daniel-beck @timja

[NotMyFault]

In order to start the discussion, we might need a bit more context and detail. I'm not sure to fully get the picture with the only the linked comments :|

Oh apologizes, I should've included more information, my bad 😔 Basically, we want to run the GH workflow linked above using terraform, and would like to know, what we need to add from our side, to have it run the infra backend, iirc.

[dduportal]

In order to start the discussion, we might need a bit more context and detail. I'm not sure to fully get the picture with the only the linked comments :|

Oh apologizes, I should've included more information, my bad 😔 Basically, we want to run the GH workflow linked above using terraform, and would like to know, what we need to add from our side, to have it run the infra backend, iirc.

Thanks for the precision. What would be the reason to use Terraform (e.g. what is (are) the problem(s) statement(s)) ? I'm asking not only for listing the infra constraints (because these working hypothesis will define security, QoS, etc.) but also because it's still not clear why would we want to use Terraform (or ) for this compared to existing?

[Alaurant]

We currently employ complex logic within GitHub Actions to fetch line changes between two commits in a specific range of files. This approach assumes that every execution must succeed. Given this, we think that using Terraform could help us avoid reinventing the wheel.

As I'm relatively new to Terraform, I might not have fully grasped your question. If my response seems off-track or if there are specific aspects I should focus on, could you please provide some corrections or guidance? I appreciate your help.

[dduportal]

😅 I feel like the bad guy here, but I'm failing to see how Terraform would help in this.

I'm re-reading Daniel's comment at https://github.com/jenkins-infra/repository-permissions-updater/pull/3998/files/1a7ce9fc6d849f5b42fe672696b9d6b7797c45b7#r1685784761 and I understand it as the ${{ github.event.before }} (part of the commented line) to be the source (for the git diff) of file comparison. => problem underlined by Daniel is that this "source" could be a failed build (or unrelated build), hence the "it assume all build are working". I fail to see how Terraform could solve this specific problem.

I have the feeling that Terraform would replace the whole YML + java code which would be a different beast.

Ping @timja could you give some guidance to @Alaurant please?

[dduportal]

Regarding the comment above, what would be needed from our side to achieve this goal and have it run on our infrastructure?

With my "Infra Officer hat", if we were to host a Terraform project for RPU:

• Challenge number 1 would be the state (ref. https://developer.hashicorp.com/terraform/language/state) which must be stored in a private and shared location.
  • Existing infra terraform states are using (private and encrypted) Azure blob storage (https://developer.hashicorp.com/terraform/language/settings/backends/azurerm). It's only available by the infra.ci service accounts or Azure admins. Rationale is because it may contain sensitive data

2 question to answer then: => How sensitive would it be? => Does it need to be shared or do you store it in the repository and only need to "lock" executions (e.g. when changing it)?

[timja]

I fail to see how Terraform could solve this specific problem.

Terraform has state, rather than just trying to look at previous builds.

The idea would be using the example from here: https://registry.terraform.io/providers/integrations/github/latest/docs/resources/team_membership

To maintain all membership of the jenkinsci organisation and the teams that people are in.

1. Terraform would maintain the state rather than guessing based on GitHub commits
2. Its a standard IaC tool used everywhere, rather than a custom Java app
3. The state itself shouldn't be sensitive, but the access it grants is.
4. I'm not sure if you want to store the state in the repository, just an Azure storage account somewhere I'd expect.

[dduportal]

Hello folks, we have delayed any work on this task to mid-October as the infra team will be in limited availability