ros-infrastructure / buildfarm_deployment

Apache License 2.0
30 stars 39 forks source link

configure_git_user script returns error #224

Closed jonazpiazu closed 4 years ago

jonazpiazu commented 4 years ago

The configure_git_user.groovy script returns the following error:

java -jar /usr/share/jenkins/jenkins-cli.jar -s http://127.0.0.1:8080 -auth admin:changeme groovy = < /tmp/configure_git_user.groovy

ERROR: Unexpected exception occurred while performing groovy command.
java.lang.NullPointerException: Cannot invoke method setCreateAccountBasedOnEmail() on null object
    at org.codehaus.groovy.runtime.NullObject.invokeMethod(NullObject.java:91)
[...]

The call to jenkins.model.Jenkins.getInstance().getDescriptor("hudson.plugins.git.GitSCM") returns a null pointer.

jonazpiazu commented 4 years ago

If I run println(Jenkins.instance.pluginManager.plugins), I get the following list:

[Plugin:command-launcher, Plugin:jobrequeue, Plugin:embeddable-build-status, Plugin:ant, Plugin:workflow-support, Plugin:PrioritySorter, Plugin:icon-shim, Plugin:antisamy-markup-formatter, Plugin:translation, Plugin:workflow-scm-step, Plugin:workflow-job, Plugin:script-security, Plugin:publish-over, Plugin:audit-trail, Plugin:external-monitor-job, Plugin:copyartifact, Plugin:windows-slaves, Plugin:branch-api, Plugin:jquery, Plugin:pollscm, Plugin:groovy-postbuild, Plugin:timestamper, Plugin:workflow-api, Plugin:monitoring, Plugin:ldap, Plugin:xunit, Plugin:javadoc, Plugin:build-timeout, Plugin:groovy, Plugin:pam-auth, Plugin:workflow-step-api, Plugin:description-setter, Plugin:systemloadaverage-monitor, Plugin:matrix-project, Plugin:disable-failed-job, Plugin:jobConfigHistory, Plugin:durable-task, Plugin:swarm, Plugin:cloudbees-folder, Plugin:trilead-api, Plugin:collapsing-console-sections, Plugin:jackson2-api, Plugin:log-parser, Plugin:mapdb-api, Plugin:bazaar, Plugin:email-ext, Plugin:workflow-multibranch, Plugin:scm-api, Plugin:mailer, Plugin:extra-columns, Plugin:github-api, Plugin:jdk-tool, Plugin:run-condition, Plugin:display-url-api, Plugin:apache-httpcomponents-client-4-api, Plugin:badge, Plugin:bouncycastle-api, Plugin:greenballs, Plugin:purge-build-queue-plugin, Plugin:matrix-auth, Plugin:structs, Plugin:junit, Plugin:htmlpublisher, Plugin:jquery-detached, Plugin:token-macro, Plugin:ace-editor, Plugin:workflow-cps]

The git plugin is missing from the list, so that's the reason why it could not be retrieved above.

I am running Jenkins 2.204.1.

jonazpiazu commented 4 years ago

It looks like many of the plugins installed are old versions:

Dependency errors:

Some plugins could not be loaded due to unsatisfied dependencies. Fix these issues and restart Jenkins to restore the functionality provided by these plugins.

Credentials Plugin version 2.3.0
Structs Plugin version 1.17 is older than required. To fix, install version 1.19 or later.
Downstream dependency errors:

These plugins failed to load because of one or more of the errors above. Fix those and these plugins will load again.
[...]

If I update the plugins, and rerun the Groovy script, it finishes successfully.

Any reason not to update the plugin versions installed by the deploy?

m-amatsu commented 4 years ago

@jonazpiazu

Hi, I am experiencing the same problem. I updated and restarted Structs Plugin from the Genkins GUI, but when I run ./reconfigure.bash master, I get an error in the same way.

When I check the version of Structs Plugin from Genkins GUI again, the version that I updated earlier seems to return to Previous version.

If you have any steps to work around this problem, could you please share them with me?

I am running Jenkins 2.190.1 I've had similar problems with newer versions than this one. Therefore, we are now working back to this version, which we were told had a proven track record.

buildfarm_deployment is forked from 57906f0ca360aab760ee9b91db129e1b3ee6bd77.

jonazpiazu commented 4 years ago

If you have any steps to work around this problem, could you please share them with me?

You can try this branch

m-amatsu commented 4 years ago

@jonazpiazu

Thanks to you, we've worked around this problem.

[Note] I edited the reconfigure.sh and rewrote it to state the following

BUILDFARM_DEPLOYMENT_URL=https://github.com/tecnalia-advancedmanufacturing-robotics/buildfarm_deployment.git
BUILDFARM_DEPLOYMENT_BRANCH=admin-jnlp-temp

When I ran ./reconfigure.sh master , I got an admin login error for jenkins, although the problem was worked around

So, once I modified /var/lib/jenkins/config.xml as follows and ran ./reconfigure.sh master again.

<useSecurity>false</useSecurity>

I did so, and ./reconfigure.sh master finished successfully. Also, /var/lib/jenkins/config.xml was automatically rewritten and returned as follows.

<useSecurity>true</useSecurity>
nuclearsandwich commented 4 years ago

@jonazpiazu thanks for helping out here and in #221.

Any reason not to update the plugin versions installed by the deploy?

In general updating plugins is a pretty frought process for the official buildfarms. In order to preserve the build history data we can't start from scratch and #160 prevents us from easily being able to re-run puppet on our production hosts. This means that before we do any plugin upgrades we have to be reasonably confident that they'll not create regressions as the only way to recover from a bad one is to restore from a machine backup.

I have ideas for improving this going forward but need to find the time to implement them.

nuclearsandwich commented 4 years ago

With #221 merged I'm going to close this issue.

jonazpiazu commented 4 years ago

This means that before we do any plugin upgrades we have to be reasonably confident that they'll not create regressions as the only way to recover from a bad one is to restore from a machine backup.

@nuclearsandwich I have a simple setup based on Vagrant + VMs to easily deploy and configure a Buildfarm instance. I have used it to test changes in a safe environment. I could share it if you think it could be useful for you.

nuclearsandwich commented 4 years ago

I have a simple setup based on Vagrant + VMs to easily deploy and configure a Buildfarm instance. I have used it to test changes in a safe environment. I could share it if you think it could be useful for you.

I have a cobble of terraform scripts to do likewise in EC2 but I'd love to see your test setup as well as it may be something we can employ for more frequent testing.

We're mostly using chef for new config management projects and I've just started trying to get test kitchen working in https://github.com/osrf/osrf_jenkins_agent (worth noting that the cookbook there is for the OSRF buildfarm which is used by our Simulation team, it's not a ROS buildfarm agent.)

jonazpiazu commented 4 years ago

I have a cobble of terraform scripts to do likewise in EC2 but I'd love to see your test setup as well as it may be something we can employ for more frequent testing.

This is quite hardcoded and ad-hoc, but happy to share if it is useful:

https://github.com/rosin-project/buildfarm_tools