Closed gunnarmorling closed 11 years ago
emmanuelbernard
commented
11 years ago It's probably just me inferring too much but I though this new approach would solve the section number changing issue we faced. I though @SpecAssertion would contain the sentence(s) at stake directly. But that might be less practical that the section + index model used right now.
Thoughts?
emmanuelbernard
commented
11 years ago Why is the commit message within parens and quotes?
hferentschik
commented
11 years ago I have just randomly review one. This one looks suspicious.
Probably the reordering. The mapping between the audit file and the tests is still a manual process and as part of the audit file generation the section ids have changed (aligning w/ the section numbers of the spec now).
I like the approach, but we need to be careful as well. When the spec changes (new sentences, reordering etc) numbering in the generated audit file will change and you need to make sure to adjust the tests. This is not an issue when the audit file is generated manually, since there the order of the assertions does not need to match the order in the spec.
gunnarmorling
commented
11 years ago I though @SpecAssertion would contain the sentence(s) at stake directly. But that might be less practical that the section + index model used right now.
Hmm, then we would have to sync the actual tests with the spec. I think the current approach basically works, in particular when the pain of copying sentences from the spec to the TCK is taken away.
One idea might actually be to express the contents of the audit file (currently XML) in form of Java code, e.g. a Java interface per section. The actual tests could somehow refer to/implement these interfaces, causing compile failures in the tests if the spec changes. This would require quite some changes to the general TCK architecture; the current proposal just automates the generation of the audit file, but leaves the general approach as is.
gunnarmorling
commented
11 years ago Why is the commit message within parens and quotes?
Do you mean the title of this pull request? I've just copied that from the ticket, the actual commit message is without parentheses/quotes.
gunnarmorling
commented
11 years ago I like the approach, but we need to be careful as well. When the spec changes (new sentences, reordering etc) numbering in the generated audit file will change and you need to make sure to adjust the tests.
That's right. Note that the audit file is generated upon request though, not automatically with every build. You still have to invoke the generation and commit the generated file manually. When doing this you can check the changes and adapt the tests if required.
gunnarmorling
commented
11 years ago
Aligning chapter/section references in existing tests and audit file with spec as of BV 1.1. As of this pull request the audit file is generated from the actual spec (see BVAL-349), but contains no additional tests for BV 1.1 yet.