createdByJenkinsAutojobs removed by Jenkins

[aklemp]

Hi,

when creating a Jenkins job using autojobs, it has an element createdByJenkinsAutojobs that is important to delete only this job if the branch is removed. Unfortunately, Jenkins (any version 1.5x I have tried) is removing this element when the job is run for the first time. Thus, the jobs are never deleted again by autojobs.

When opening the page "Manage Jenkins" with newly created jobs, they are listed here because the configuration format is invalid. Viewing the details, there is a MissingFieldException: No field 'createdByJenkinsAutojobs' found in class 'hudson.maven.MavenModuleSet'.

How is this supposed to work? Is there a workaround to keep the element that Jenkins obviously cannot know?

One solution would be to create a Jenkins plugin declaring the required configuration elements so Jenkins can recognize them. The format of the XML would change slightly because plugin version and the like are added.

Kind regards, aklemp

[aklemp]

Hello again,

I just created a Jenkins plugin declaring the configuration data and patched the python scripts to use the correct XML elements. The first tests indicate that it works.

If you are interested I can provide the implementation. However, I'm still curious why nobody else seems to have the problem of removed configuration data.

Kind regards, aklemp

[gvalkov]

Hello @aklemp,

Interesting indeed - perhaps the cleanup functionality isn't seeing a lot of use or Jenkins' behaviour somehow changed. I'm definitely interested in your solution. Maybe it makes sense to have a generic Jenkins plugin for storing such configuration items. I'll bring this up on the jenkins-users mailing list and see if someone has an idea.

Thank you for taking interest in jenkins-autojobs. The issues you've submitted have been very thorough.

Kind regards, Georgi

[aklemp]

Hi Georgi,

thank you for investigating the issue. Meanwhile I send you the changes I've made locally.

The Jenkins plugin basically consists only of one class:

public class AutojobsProperty extends JobProperty<AbstractProject<?, ?>> {
  private String ref;
  private String tag;

  @DataBoundConstructor
  public AutojobsProperty(String ref, String tag) {
    this.ref = ref;
    this.tag = tag;
  }

  public String getRef() {
    return ref;
  }

  public String getTag() {
    return tag;
  }

  @Extension
  public static class AutojobsPropertyDescriptor extends JobPropertyDescriptor {
    @Override
    public String getDisplayName() {
      return Messages.AutojobsProperty_DisplayName();
    }

    @Override
    public boolean isApplicable(@SuppressWarnings("rawtypes") Class<? extends Job> jobType) {
      return AbstractProject.class.isAssignableFrom(jobType);
    }
  }
}

The main.py has to be changed in the cleanup method, supposing that the above class is in package "somepackage".

# tag = '</createdByJenkinsAutojobs>'
# tagxpath = 'createdByJenkinsAutojobs/tag/text()'
tag = '</somepackage.AutojobsProperty>'
tagxpath = '//somepackage.AutojobsProperty/tag/text()'

The job.py has to be changed as well:

# info_el = etree.SubElement(self.xml, 'createdByJenkinsAutojobs')
props = self.xml.xpath('/*/properties')
if len(props) == 0:
  props = etree.SubElement(self.xml, 'properties')
else:
  props = props[0]
info_el = etree.SubElement(props, 'somepackage.AutojobsProperty')
info_el.set('plugin', 'autojobs-plugin@1.0')

Kind regards, aklemp

[alexak]

Hi,. @aklemp : I am also interested in your workaround. Can you provide pßlease your plugin? Or is your solution already integrated in current version?

[aklemp]

Hi @alexak,

you can find the workaround and the important plugin code in my comment of 24 Nov 2014. Unfortunately, I cannot provide the plugin as is because I created it for one of my customers.

Kind regards, aklemp

[gvalkov]

The only response I got on jenkins-dev was to use job properties instead of messing around with the job's config.xml.

[StephenGregory]

One possibility is to get a list of all job names that auto jobs would create, and compare that to which of those jobs exist after autojobs creates jobs for new branches. Then you could delete the difference (of course, this assumes that one isn't naming jobs the same as autojobs' convention.

[sustmi]

I get the <createdByJenkinsAutojobs> tag deleted every time I save job configuration. Even clicking the deactivate button deletes it.

As Jenkins is really strict about job's config.xml format and does not allow <createdByJenkinsAutojobs> tag (unless you load custom extension), can we have the information encoded in the job description text?

[gvalkov]

Since commit de969781244919d1f648e3ab89a7e73826d2cc1f, jenkins-autojobs will store its metadata in the job description of managed jobs (thank you @sustmi for the idea). The following two strings will be appended to the description:

(created by jenkins-autojobs)
(jenkins-autojobs-tag: tagname)  # only if you're using the tagging feature

The old behaviour is still available by setting the tag-method option to element.

[gvalkov]

I want to add one more piece of functionality that helps with transitioning from tag-method: element to tag-method: description and I'll release a new version. Comments and suggestions are more than welcome in the meantime.

[sustmi]

Wow, you did it in no time, @gvalkov, wonderful!

I tested it, but I am getting this error:

Traceback (most recent call last):
  File "/usr/bin/jenkins-makejobs-git", line 9, in <module>
    load_entry_point('jenkins-autojobs==0.16.2', 'console_scripts', 'jenkins-makejobs-git')()
  File "/usr/lib/python2.6/site-packages/jenkins_autojobs-0.16.2-py2.6.egg/jenkins_autojobs/git.py", line 130, in _main
    main.main(argv[1:], config=config, create_job=create_job, list_branches=list_branches)
  File "/usr/lib/python2.6/site-packages/jenkins_autojobs-0.16.2-py2.6.egg/jenkins_autojobs/main.py", line 159, in main
    job_name = create_job(branch, tmpl, config, branch_config)
  File "/usr/lib/python2.6/site-packages/jenkins_autojobs-0.16.2-py2.6.egg/jenkins_autojobs/git.py", line 121, in create_job
    tag_method=config['tag-method']
  File "/usr/lib/python2.6/site-packages/jenkins_autojobs-0.16.2-py2.6.egg/jenkins_autojobs/job.py", line 90, in create
    self.tag_config(tag, tag_method)
  File "/usr/lib/python2.6/site-packages/jenkins_autojobs-0.16.2-py2.6.egg/jenkins_autojobs/job.py", line 73, in tag_config
     if mark not in desc_el.text:
 TypeError: argument of type 'NoneType' is not iterable

job.py:

 72:    desc_el = self.xml.xpath('/project/description')[0]
 73:    if mark not in desc_el.text:
 74:        desc_el.text += mark
 75:    if tag not in desc_el.text

[sustmi]

config.xml:

<project>
  <actions></actions>
  <description></description>
  <keepDependencies>false</keepDependencies>
  <properties></properties>
...

[gvalkov]

It's been only 11 months since the issue was opened :)

I guess I was too keen on finding a way that was fully transparent to the end user. Using the description field is by all means a "good enough" solution and I'm glad that you bough it up.

Thank you for trying HEAD out - the TypeError was fixed in 21bb8f8fcb5f228f3f64a1d7509b4414d22b8fcb. I hope to have a release by the end of the week ... not a lot of time for OSS lately, I'm afraid.

[sustmi]

Of course I meant the time from me suggesting the move to description and you implementing it. :D

I tested https://github.com/gvalkov/jenkins-autojobs/commit/21bb8f8fcb5f228f3f64a1d7509b4414d22b8fcb and it works! :+1: Thank you very much, Georgi.

[gvalkov]

Jenkins-autojobs 0.17.0 is on PyPi with the discussed fix. Here's a brief description, straight from the changelog:

Jenkins-autojobs will now use the job description field to store its metadata. Expect to see (created by jenkins-autojobs) and (jenkins-autojobs-tag: $tagname [...]) lines appended to the description of all managed jobs.

This metadata was previously stored as elements in the job’s config.xml. Unfortunately, any manual reconfiguration of the job would cause Jenkins to remove these extra elements. This issue is described in greater detail in issue #28. The old behavior can be kept by setting the tag-method option to element.

[sustmi]

:+1: