Allow for Jenkins project/folder level GitLab Token API credentials to be used for GitLab connection #631

Open acohen716 opened 7 years ago

Feature Request

In our enterprise we have an Enterprise GitLab and an Enterprise Jenkins set up such that groups have their own Jenkins folders and don't have access to the Global Jenkins configuration to add their GitLab API Token credentials for their GitLab groups projects at the global level. It appears that the gitlab-plugin pulls the GitLab connection info from the Manage Jenkins->Configure admin level which only the Enterprise Jenkins admin has access to. If the GitLab API Token was pulled from the project/folder/group level vs. the admin level then non-admins would be able to add their own GitLab API Token without requesting the Jenkins admin to add it at the global level.

Context

Gitlab plugin version: 1.4.2
Gitlab version: Community Edition 9.5.1
Jenkins version: CloudBees Jenkins Enterprise 2.46.24.0.2-fixed
Job type: Freestyle

We're also in need of this. @omehegan anything we can do to help advance this idea?

@bbodenmiller I would welcome a pull request that implements it :) Otherwise I'm not personally in a position to do it, I'm really just a shepherd for this plugin. Other volunteers write the code.

That's a bit of the problem... not sure where to start with the changes needed? Any idea who might be able to provide guidance?

Maybe @mreichel or @colourmeamused.

@acohen716 @bbodenmiller How is your enterprise Jenkins setup? Do you use the Folders plugin in combination with the RBAC plugin?

Yep, we're using the CloudBees Folders Plugin and CloudBees Role-based Access Control Plugin. Please let me know what other details about our setup would be useful to know about.

I looked into this, but unfortunately I don't have any idea how to implement this feature. The possibility to use environment variables on folder level seems to have moved to the folder plus plugin and I didn't see core support for folders. Maybe someone like @stephenc has ideas how to implement this, I don't.

No need to do anything with environment variables.

This all depends on how the plugin works.

If the plugin takes a list of GitLab servers that are configured in the global config screen and the credentials are defined in that screen and the user just enabled the GitLab plugin at the job level (or it gets enabled by magic) then you probably will be blocked on https://issues.jenkins-ci.org/browse/JENKINS-40275 (unless you are willing to rework the UI)
If the plugin presents at the job level with some configuration, then you could have a UI that says something like
- [ ] Use alternative authentication
This would then display a select credentials drop-down when checked.

Of course, I do not know exactly what use cases this plugin attempts to solve (a lot of the code looks like, what I would consider from a 30 second peek, sub-optimal attempts to solve problems that would be better standing on the scm-api functionality) so I cannot say for sure what way I would recommend approaching this, but it would be two of the above... and if you want something that works now... well that only leaves one option... and that option may involve reworking how the plugin does things

So as an FYI, this method is responsible for looking up the credentials. In the credentials API documentation I describe the factors to consider when deciding which credentials should be made available in different contexts

StandardCredentials credentials = CredentialsMatchers.firstOrNull(
            lookupCredentials(StandardCredentials.class, (Item) null, ACL.SYSTEM, new ArrayList<DomainRequirement>()),
            CredentialsMatchers.withId(apiTokenId));

is mostly ok for a use case where the credentials are defined at a global level.

If you want jobs to be able to override credentials then you'd go with something like:

StandardCredentials c = CredentialsMatchers.firstOrNull(
    CredentialsProvider.listCredentials(
      StandardCredentials.class,
      job,
      job instanceof Queue.Task
        ? Tasks.getAuthenticationOf((Queue.Task)job))
        : ACL.SYSTEM,
      URIRequirementBuilder.fromUri(gitlabUrl)
    ),
    CredentialsMatchers.allOf(
      CredentialsMatchers.withId(credentialsId),
      AuthenticationTokens.matcher(GitLabAuthentication.class)
    )
  );
GitLabAuthentication auth = AuthenticationTokens.convert(
  GitLabAuthentication.class,
  c
);

But that would require knowing the Job/Item against which the credentials are being requested, and consequently knowing which authentication should request the credentials.

Note, the example also switches to use the Authentication Tokens API plugin rather than a cumbersome

            if (credentials instanceof GitLabApiToken) {
                return ((GitLabApiToken) credentials).getApiToken().getPlainText();
            }
            if (credentials instanceof StringCredentials) {
                return ((StringCredentials) credentials).getSecret().getPlainText();
            }

Thank you @stephenc ! I hope that is the guidance @bbodenmiller needed to start

Hello, I experience that problem with our Jenkins at work, and I don't understand what precisely needs to be done in order to make things work. I followed the documentation of the GitLab Plugin to set up the API Token in the global configuration of Jenkins. But when configuring the jobs, I don't see the GitLab token API i configured.

Can somebody tell me what I need to do as a solution or as a workaround in order to be able to setup my job to work please ? here my global config: gitlab-plugin-issue-credentials

and here is the job config: gitlab-plugin-issue-job-configuration-api-token-not-showing-up

I expect to have the GitLab Token API credential in the red rectangle listed in the job, but it does not show up in the dropdown list.

Additional information: My repo is a project inside a group. An the project is only accessible for members of the group. I then used my own personal account to generate in GitLab a new API key for jenkins.

Thank you very much for your help.

Hi again, I answer to myself :-)

So for the ones having the same difficulties: the solution to make the jobs work:

Make sure to have a ssh key created in the jenkins home: /var/lib/jenkins/.ssh/id_rsa and /var/lib/jenkins/.ssh/id_rsa.pub (you can follow these instructions to help: connecting with ssh by executing then with the jenkins user: sudo su jenkins
make sure git is installed on the server (apt-get install git if not installed)
In Jenkins: in the job configuration, locate the GitLab connection dropdown setting and select the connection that has been defined in the global configuration of Jenkins.
lower in the job configuration page, do not select credentials.

And that's it. You should be done with it. Thank you all.

Hi Alain-

Don't think the sshkey solution would work for some multi-tenant configs.

Re: the below, if the job triggers the plugin, the job being an instance in a folder would allow us to interrogate the Folder credentials I'd think. Then we'd just have to populate the credentials dropdown or check cred IDs for pipeline.

If you want jobs to be able to override credentials then you'd go with something like:

StandardCredentials c = CredentialsMatchers.firstOrNull(
    CredentialsProvider.listCredentials(
      StandardCredentials.class,
      job,
      job instanceof Queue.Task
        ? Tasks.getAuthenticationOf((Queue.Task)job))
        : ACL.SYSTEM,
      URIRequirementBuilder.fromUri(gitlabUrl)
    ),
    CredentialsMatchers.allOf(
      CredentialsMatchers.withId(credentialsId),
      AuthenticationTokens.matcher(GitLabAuthentication.class)
    )
  );
GitLabAuthentication auth = AuthenticationTokens.convert(
  GitLabAuthentication.class,
  c
);

But that would require knowing the Job/Item against which the credentials are being requested, and consequently knowing which authentication should request the credentials.

Do we have any update here? The problem exist year but still actual for us. We are using the latest scope of plugins/tools. Also I didn't actually get on who side the problem (Gitlub plugin or CloudBees). In my opinion its bug into Gitlub plugin due we get problem with your GitLabConnectionProperty.getClient(...). Any tricks with 'ssh' or 'env folder permission' don't work (due the different causes). So.. does some one can help with it? Save project into folders and use gitlab plugin is very important for our CI. p.s. I couldn't find the root problem into code by my self and can't make fix. Because I still don't get where I should try to find problem.

I think the easy and clear way to reproducing it that: 1) create folder folder-name 2) create job into folder: folder-name/mr-job 3) set trigger for some note (comment) 4) create new job and copy from folder-name/mr-job 5) add note/comment and trigger them

into jenkins console invoke next:

def build_into_folder = Jenkins.instance.getItemByFullName('folder-name/mr-job').getBuildByNumber(1)
def build_into_root = Jenkins.instance.getItemByFullName('mr-job').getBuildByNumber(1)

println build_into_folder .getCause(com.dabsquared.gitlabjenkins.cause.GitLabWebHookCause.class) // should have it
println build_into_root .getCause(com.dabsquared.gitlabjenkins.cause.GitLabWebHookCause.class) // should have it

println build_into_folder.getParent().getProperty(com.dabsquared.gitlabjenkins.connection.GitLabConnectionProperty.class).getClient() // will be null
println build_into_root.getParent().getProperty(com.dabsquared.gitlabjenkins.connection.GitLabConnectionProperty.class).getClient() // will have value

@Osmyslitelny it is not clear to me whether your issue is the same as the one described here. This issue is a feature request, asking that the plugin support folder-scoped GitLab API credentials. We currently do not support that. I would be happy to review and merge a PR if someone submits one, but I do not have the ability to do the work. There is nothing stopping you from having jobs in folders triggered by GitLab, using the existing global API credentials support.

I don't understand what you are trying to illustrate with the Groovy code you have included here.

Hi, Is there anyone to perform the revue of PR 916? It should resolve this issue.

Would also like to see #916 merged 😄

I am taking over a lot of the maintaining. I am working through the open PR's now and will have a release out this weekend. Apologize for the delay

Any news on #916 ? I'd like to see it merged as well ;)

Happening tonight

Hi Marky,

Are there any news to this PR? I do not want to rush you in any way, but we are waiting desperately for that feature :)

Thank you very much for your time and support.

Kind regards

Great! Thank you for merging this new feature. I am looking forward for the new release including this feature :)

Thank you.

jenkinsci / gitlab-plugin

Allow for Jenkins project/folder level GitLab Token API credentials to be used for GitLab connection #631

Feature Request

Context