jenkins-autojobs creates additional description elements for non-Freestyle projects

[gotgenes]

We have a template based on a Maven project that has a description filled in. The fix in 0619831 for Issue #54 causes an additional <description> element to be inserted prior to the one that exists for the Maven project template.

<?xml version="1.0" encoding="UTF-8"?><maven2-moduleset plugin="maven-plugin@2.12.1">
  <actions/>
  <description>
(created by jenkins-autojobs)
(jenkins-autojobs-tag: amplisequi-ci-autojobs)</description><description>&lt;h1&gt;ACME UI CI Build&lt;/h1&gt;</description>
  <keepDependencies>false</keepDependencies>
  <properties>
    <jenkins.model.BuildDiscarderProperty>
…

The expected behavior is that

(created by jenkins-autojobs)
(jenkins-autojobs-tag: amplisequi-ci-autojobs)

would be appended to the existing <description> element's contents.

This is also causing jenkins-autojobs to fail to delete jobs when branches get deleted from the remote repository.

Would it be reasonable to search for the first <description> element in the XML document, both during the creation and deletion stages?

[lxndrhnz]

We have the same problem in our installation.

I had a closer look at the change in https://github.com/gvalkov/jenkins-autojobs/commit/0619831e4809ba7807b6fea575febdcefacc8b95. I think it solved issue #54 not because of the changed xpath but because of the desc_el null handling. But this null handling is the reason that an additional <description> is created.

In maven based projects, there is no /project element at the document root. They start with /maven2-moduleset. The description is unter /maven2-moduleset/description. The xpath expression will never find the description. And jenkins removes the doubled description whenever a job is modified (same problem as with the custom tag).

I attached a patch for jenkins_autojobs/job.py with a changed xpath expression. At least in our installation that solves this issue, but we only manage maven based jobs with autojobs. Someone has to verify that this is also working with other job types.

patch.zip

[gvalkov]

Hello @gotgenes and @lxndrhnz and thank you both for the good write up. I've made a fix, that should at least make it work reliably with freestyle and maven projects: 4204b995da5cebaa13aa569dc53a99c95312f0e4

I'll roll out a new release as soon as I get some more testing done. Any help with testing would be greatly appreciated. Fiy, the development tip can be installed with:

pip install -U git+git://github.com/gvalkov/jenkins-autojobs.git@4204b995da5cebaa13aa569dc53a99c95312f0e4

[gotgenes]

Hi @gvalkov. Thanks for your response. I have concern that this change

for parent_xpath in '//maven2-moduleset', '//project':

will become brittle. It is hard to say there are not other styles of projects that have other root elements.

I go back to my question above:

Would it be reasonable to search for the first <description> element in the XML document, both during the creation and deletion stages?

[gvalkov]

It would be brittle indeed - I implemented your proposal in 06f91960a3c486b14e5fbbcc9ccafc4d69481b71 alongside the previous workaround.

[gvalkov]

I'm starting to think that #54 was also about a maven job. It would be nice to assume that a job's config.xml always has a description element, in which simply searching for the first 'description' element should be the only necessary solution.

[gotgenes]

I just wanted to clarify, that it's not the fault of jenkins-autojobs this is brittle. There are a couple of reasons this is a tough problem to tackle:

1. jenkins-autojobs started with a sensible idea of adding its own element (tag) to the jenkins' jobs. Unfortunately, it seems this violates some sort of XML schema for the jobs, and causes Jenkins to complain loudly that one or more jobs are in an unexpected format (screenshot below)
2. My guess is this motivated co-opting some existing element, and I believe @gvalkov's choice of co-opting the <description> element is the best.
3. Unfortunately, the Maven plugin authors have taken liberty with the schema of the Jenkins jobs, and do not follow the same schema as the "Freestyle Job" for Jenkins. There isn't much the jenkins-autojobs authors can do about this. "The cat is out of the bag," as they say.

So the question now is, what's the least fragile thing jenkins-autojobs can do to support these variances from the expected XML schema.

Here's the list of project types we have available in our current Jenkins configuration right now:

1. Freestyle project
2. Maven project
3. External Job
4. Multi-configuration project

The only two I've matched up with jenkins-autojobs to date are Freestyle and Maven. I'm not sure what the schemas for the other two look like; that is, I don't know if Maven is a rogue, or if each of them have their own schemas. I'll have to check that out.

If you want to try a more structured approach, we could define a Job class for each different project type (like FreestyleJob, MavenJob, etc.), and use polymorphism to just call the respective instance's job.set_description(), job.get_description() (or make it a property). I don't know how much architecture you want to put into this project, though.

[gotgenes]

@gvalkov 06f9196 appears to have resolved the issue for our Maven project jobs. The existing description element is now being modified instead of a new one being created. Thanks!

[gvalkov]

Hi @gotgenes. Thank you for taking the time to test the change and for doing such a great write up on the core issue. I've pushed the fix to PyPi with jenkins-autojobs 0.17.4.

[gvalkov]

I must admit that the project is starting to suffer a bit from the lack of architecture. I wish I could sit down and redesign parts of it, but lately I've only had time for bug fixing.

[gotgenes]

@gvalkov You do not have to apologize. You have contributed this project. The changes here fixed the issue, and jenkins-autojobs already provides value to its users.

[lxndrhnz]

Hi,

the fix solved this issue for us. Thank you very much. Nevertheless let me share my thoughts about the curent solution with you.

It's not autojobs fault that plugin authors introduced own schemas and you have to know all of their specials.

The current solution solves this with iterating trough all "currently known" config root elements and looking for a description on the next level. Good thing: It's easy to extend the list of known roots. Bad thing: You have to extend the list of known roots for evert project type.

Educated guess: No plugin author will create a config that doesn't support a description on this level because displaying the projects (whatever type) description is an essental part of jenkins view on a project. On the other hand multiple description elements on this level are unlikely and not supported by jenkins (see my comment above, jenkins removes duplicate descriptions).

Based on this you can simplify and generalize your solution by using the following xpath expression for selecting the description element on the first level of the config.xml instead of iteration through all "known roots". /*/description

You don't have to change the way you solved this. I just wanted to throw this idea in for discussion.

Thanks again for this cool project and fixing the bugs that fast.