Support private repository extensions

[igorbrites]

Feature description 💡

I created a private terrakube-extensions, and now I need to use it on the executors, though it's showing this error to me:

ERROR org.terrakube.executor.service.scripts.ScriptEngineService - https://github.com/MyOrg/terrakube-extensions: Authentication is required but no CredentialsProvider has been registered

How do I pass the credentials? I thought the same GitHub App used to make Dex work was also being used here, but it's not.

Anything else?

Opened as a feature request because idk if it's a bug 😬

[alfespa17]

It is not really a bug, it will be a new feature.

You can customize the git repository but for now it does not support a private repository.

[alfespa17]

The executor component is cloning the directory in this part of the code without adding any kind of authentication as you can see here.

https://github.com/AzBuilder/terrakube/blob/adc227fde6c7a791f1b888dc06dd7e031b2b7f33/executor/src/main/java/org/terrakube/executor/service/scripts/ScriptEngineService.java#L107

This is the code when you are cloning a private git repository that is used inside a workspace that is really similar

https://github.com/AzBuilder/terrakube/blob/adc227fde6c7a791f1b888dc06dd7e031b2b7f33/executor/src/main/java/org/terrakube/executor/service/workspace/SetupWorkspaceImpl.java#L140

The parameter that the executor receives to execute a job is done in this part of the code, maybe adding some information about the private extension could work.

https://github.com/AzBuilder/terrakube/blob/adc227fde6c7a791f1b888dc06dd7e031b2b7f33/executor/src/main/java/org/terrakube/executor/service/mode/TerraformJob.java#L13

The API is sending the request with all the parameters to the executor in this part of the code

https://github.com/AzBuilder/terrakube/blob/adc227fde6c7a791f1b888dc06dd7e031b2b7f33/api/src/main/java/org/terrakube/api/plugin/scheduler/job/tcl/executor/ExecutorService.java#L63

https://github.com/AzBuilder/terrakube/blob/adc227fde6c7a791f1b888dc06dd7e031b2b7f33/api/src/main/java/org/terrakube/api/plugin/scheduler/job/tcl/executor/ExecutorContext.java#L14

And finally the Java class that represents the YAML file that you use to define extension is in this part ot the code.

https://github.com/AzBuilder/terrakube/blob/main/api/src/main/java/org/terrakube/api/plugin/scheduler/job/tcl/model/Flow.java

By the way I think you are the first person that want to use a private extension repository

[igorbrites]

Hmmm got it. Could we mount the SSH key we configure on the organization inside the executors and use it to clone the repo? I used https just because the default extensions repo is using it. And the reason I have a private repo is that I needed a MS Teams extension, to alert me about drift detection.

If it gets too complicated, I can try opening a PR on your extensions repo to add the MS Teams extension, but as I'm not a developer, idk if it would even work (I ~shamelessly copied~ inspire myself on the SlackApp one to create mine 🙃).

[alfespa17]

Mounting the SSH key in executor won't work because JGIT is not using that keys to clone the repository.

You could fork the extension repository and inside your fork you could create a branch with the new feature for MS teams.

In the helm chart you could customize the parameter like this:

executor:
  properties:
    toolsRepository: "https://github.com/mygithubuser/my-extensions"
    toolsBranch: "my_feature_branch"

Now you can test your new extension in a different repository and when you have it ready you could send the pull request with some sample code and some images so we can update the documentation.

All help is welcome this is an open project 👍

[igorbrites]

I've created a draft on terrakube-extensions repo: https://github.com/AzBuilder/terrakube-extensions/pull/36

I'll test it like you told me to, and then I open the PR for code reviews.