search

collective / collective.xmltestreport

Test runner which can output an XML report compatible with JUnit and Jenkins
https://pypi.org/project/collective.xmltestreport
7 stars 8 forks source link

XML test reports not published for jenkins since 1.2.7 #7

Closed mitchellrj closed 9 years ago

mitchellrj commented 12 years ago

Since updating from 1.2.6 to 1.2.8, XML test reports are not generated on our Jenkins server.

Configuration has not changed between these updates, and all tests continue to run and pass.

Configuration in buildout:

[test]
recipe = collective.xmltestreport
eggs = 
    my_egg [test]
defaults = ['--xml', '--auto-color', '--auto-progress']

Downgrading to 1.2.6 meant reports were generated again.

domenkozar commented 12 years ago

It's because collective.xmltestreport changed location where reports are published, unfortunately without some clear warning except for changelog.

tisto commented 12 years ago

Domen, could you please elaborate that? Are you saying that someone changed the API/output within a minor version upgrade without any notice?

domenkozar commented 12 years ago

Probably not intentionally, buy yes: https://github.com/collective/collective.xmltestreport/commit/fd4e4d814bdb3c643b678dd4f1d0bdf0fdfb00b0#L0L36

zopyx commented 12 years ago

That's exactly what happened with 1.2.7. Someone changed the output directory from parts/xmltestreport to /xmltestreports.

Fixing 20 Jenkins configurations was not funny. In fact this unprofessional change pi**ed me on a bit.

domenkozar commented 12 years ago

IMO, expect this to happen to OSS that has no general dictator :) Too late to do anything now, except for documenting the change more clear.

tisto commented 12 years ago

I don't see how a dictator would not have prevented this from happening. Changing the API in a minor version is unacceptable and should be considered a bug. This is especially true for Open Source projects. It breaks the contract/trust between the user and the developers. All API changes need to be properly documented and announced with a major version change. In general APIs should only change for a good reason (which I don't see here). IMHO all releases since 1.2.6 should be considered brown bag releases. I understand why Andreas and other do not want to re-change their paths though. Errors happen, though, we should at least make it clear to the person who introduced this change that it was the wrong thing to do. I still don't see where this happened exactly in the commit though.

tisto commented 12 years ago

I will revert that commit https://github.com/collective/collective.xmltestreport/commit/fd4e4d814bdb3c643b678dd4f1d0bdf0fdfb00b0 and make a new release in the 1.2.x branch. I won't change all the Jenkins buildouts AND the configuration for all Plone coredev Jenkins jobs just because someone just ignored conventions that are essential for open source projects to function! As I said, errors happen but this does not mean that we just have to accept them. This commit not only changed the name of the output folder but also added another "working-directory" inside the tests folder which is just plain stupid! This is a bug that we have to fix..period.

domenkozar commented 12 years ago

Hmmm - this would break jenkins setups I have - again. Maybe some automatic script that changes all jenkins configuration files can be provided?

tisto commented 12 years ago

No, and honestly I don't know why we should bother. This is a matter of principle. Changing the API in a minor version was wrong, doing releases including this bug was wrong, adapting the buildouts to this change was wrong, not reporting a bug for this was wrong.

The right thing to do is to report this bug (which is what mitchellrj did) and fix it (what I plan to do). Shall we really punish the people that did the right thing (and all the people that just pinned to 1.2.6 so far)?

I don't see how we can consider Plone to be a a mature Open Source project, if we just ignore the basic principles for software releases with exuses like "it is too late to do anything now". I know it really sucks if you have to change your settings again, but this should not prevent us from doing the right thing.

Apart from that, if we would keep it like it is, we would have to change the output directory in a later release anyway. Why the hell do we need a "working-directory" inside the existing tests output directory?

zopyx commented 12 years ago

+1 on Timo's comment. The change introducing new behavior and changing paths must be reverted. This was a major foul.