search

jakartaee / validation-tck

Jakarta Validation TCK
http://validator.hibernate.org
Apache License 2.0
38 stars 33 forks source link

BVTCK-58 Excluding tests by default which cannot be asserted portably… #65

Closed gunnarmorling closed 9 years ago

gunnarmorling commented 9 years ago

… as their expected exceptions may be either raised at deploy or runtime

The affected tests (annotated with @OptionalTest) can be included by specifying -DincludeOptionalTests=true.

gunnarmorling commented 9 years ago

@emmanuelbernard, @hferentschik, can you give this a look?

gunnarmorling commented 9 years ago

@emmanuelbernard, @hferentschik, any thoughts regarding this fix for BVTCK-58?

hferentschik commented 9 years ago

I guess you want to refer to BVTCK-68, right?

Anyways, on second thought I am not sure about this anymore. There is nothing wrong with the infrastructure you put in place, but in this case I would argue that the test should be marked as non-testable in the spec and removed from the TCK. Marking it as optional somehow implies that the indented behavior is somehow preferred, but as discussed in the issue itself there is nothing in the spec which specifies the exact behavior in this case. IMO it is just not testable in the BV / EE scope.

If anything, we are dealing with some Wildfly specific behavior. If we want to keep testing for it, I would recommend to make this an integration test of Validator.

gunnarmorling commented 9 years ago

I guess you want to refer to BVTCK-68, right?

Yes, the commit message is right.

IMO it is just not testable in the BV / EE scope.

I don't think that's true per-se, more a restriction of the current testing infra: We cannot make an assertion on an exception to be either raised during deployment or at first validator usage. This may change though at some point.

The test is disabled by default, so the TCK will be passed now, no matter what the behavior of a given implementation is. As it solves BVTCK-68, can't we move forward with it?

gunnarmorling commented 9 years ago

@emmanuelbernard, @hferentschik: Force pushed an update. Note what I would have preferred to do, but if it allows us to move on, then be it.

hferentschik commented 9 years ago

Thanks, merged.

Follow up JIRA - https://hibernate.atlassian.net/browse/BVTCK-70