Open brettz9 opened 4 years ago
I might add my validity checks are akin to that added in https://github.com/cypress-io/cypress-documentation/pull/1919 (at https://docs.cypress.io/faq/questions/using-cypress-faq.html#Can-I-check-that-a-form%E2%80%99s-HTML-form-validation-is-shown-when-an-input-is-invalid )
My apologies re: the environmental issue; it seems that my issue there may have been not returning the config
object. However, the main problem with validity checking remains.
So the problem here is not that it's not finding elements with a psuedo-selector of invalid
, it's that the actions being performed through Cypress are not triggering the validation checks on the form for some reason.
You can use jQuery to query for input:invalid
on the application under test after Cypress performs the type and even the click to submit the form and see that it finds no invalid elements.
spec.js
it('does not find invalid input', () => {
cy.visit('index.html')
cy.get('input').type('a')
// cy.contains('Update').click()
cy.get('input:invalid').should('have.length', 1)
})
index.html
<!DOCTYPE html>
<body>
<input minlength="3">
<button type="submit">Update</button>
</body>
</html>
Events triggered during type.
If you type manually into the form while the test is running, it will all of a sudden pass.
In case it helps: I have 2 forms, one with a text and password input, and one with a text, password, and numerical input. In both cases only the text input exhibits this bug. The password and numerical inputs behave as expected - form validation popups are appearing. This remains true if I reorder the inputs on the page and change the order I type into them in Cypress. Node 12.13.0 Cypress 4.8.0 Chrome 83.0.4103.97 Windows 10 Home v1903 build 18362.900
Just to add my own experience: getting this bug on a minlength
validation on a password
input:
when using cy.get().type()
to type text that should be too short, I am able to submit the form both manually and through Cypress.
In essence, it looks as if Cypress is not sending the necessary key events for the form to perform validation.
I came across this issue as well. The minlength attribute will be ignored. Validation of the pattern
attribute pattern=".{6}"
however will be triggered. Unfortunately the resulting message ("Please match the requested format") is very vague and rather confusing for the user. Further adding setCustomValidity manually wouldn't be my favourite option.
I tried form.reportValidity()
. But somehow, it doesn't work. It always return true when there's an error in the form.
According to MDN, minLength
is only triggered if value is changed by user.
Is this planned, e.g., on a Roadmap, or given a priority? It is a pretty fundamental need to be able to check forms, testing for validity (for those using the standard mechanisms for doing so). Thanks in advance...
@brett9 Unfortunately, it's hard to find browser API that triggers form validation. Maybe, this can be solved with native events.
@sainthkh : Thanks for the info and idea. If this would be best addressed by Native Browser Events, I would hope its consideration might be added to the list on #311 , @brian-mann .
An update: for me the validations are all working, you can see in https://glebbahmutov.com/cypress-examples/6.4.0/recipes/form-validation.html
Note: cy.submit
does not check the validity of the form before submitting https://github.com/cypress-io/cypress/issues/14911 while clicking on the submit button does check
The code for this is done in cypress-io/cypress#14965, but has yet to be released.
We'll update this issue and reference the changelog when it's released.
This change was reverted in https://github.com/cypress-io/cypress/pull/15302
Sorry for the delay in replying...
As one example, I have:
<input type="password" autocomplete="new-password" class="form-control" required="" name="pass" id="pass-tf" data-name="reset-pass" minlength="6">
but this code fails:
const tooShortPassword = 'a';
cy.get('[data-name="reset-pass"]').type(tooShortPassword);
cy.get('[data-name="reset-pass"]:invalid').should('have.length', 1); // Should be 1, but gives 0
The minimal test repo in the original post covers this and the other cases I believe are still occurring.
I.e., it's not just about the submission.
@brettz9 Yeah, you're right. I'm actually not able to get the original example to work with the new implementation. https://github.com/brettz9/cypress-test-tiny/tree/not-getting-invalidity?rgh-link-date=2020-03-09T04%3A34%3A03Z cc @sainthkh
.submit()
failing the test as the PR described it should.Confirmed that #14965 didn't fix the issue.
But the problem is that it cannot be fixed now. Because all validations are not created equal in browsers. As for the validations like required
, mix
, min
, they always work whether it's a user input or not. But for those like minlength
, maxlength
, they are only triggered when users input the value.
If you want to test it yourself, sample code is below:
<input id="x" minlength="3" value="a" />
<input id="y" required />
cy.get('#x').then($x => expect($x[0].checkValidity()).to.be.false) // => fail
cy.get('#y').then($y => expect($y[0].checkValidity()).to.be.false) // => succeed
Native events might solve the problem. But I'm not sure about that.
Is there any solution on this ? I am facing the same.
I could solve this problem using "realType" from cypress-real-events, then the input field is validated as with real text input. However, this solution is not suitable, for example, for Firefox.
Current behavior:
Not finding invalid elements, whether by:
cy.get('<selector>:invalid')
validity
propertyDesired behavior:
Correctly indicate that the element is indeed invalid (so that assertions can be based on this).
Test code to reproduce
These tests both fail in showing any problems with invalidity, despite the HTML element being queried having
minlength
attribute of 3 and our test typing only a single letter ("a"):Here's the minimal repo to reproduce: (note that I've added a cypress install as I think you may want to do in your boilerplate also since your docs refer to installing cypress locally)
https://github.com/brettz9/cypress-test-tiny/tree/not-getting-invalidity
(i.e., on the
not-getting-invalidity
branch)Outdated info
I've also tacked on an environmental variable test (same as #2605 ?) ```js it('should obtain environmental variables', function () { cy.log(Cypress.env('secret')) expect(Cypress.env('secret')).to.equal('Be good'); }); ``` ...which is not *consistently* getting the `secret` despite it being set in `plugins/index.js` ```js module.exports = (on, config) => { config.env = config.env || {}; config.env.secret = 'Be good'; return config; ); ``` ...consistent with the approach described for dynamically setting the environmental variables at https://docs.cypress.io/api/plugins/configuration-api.html#Switch-between-multiple-configuration-files or https://docs.cypress.io/api/cypress-api/env.html#From-a-plugin.Versions
4.1.0, Mac OSX 10.15.2, Chrome 80.0.3987.132 (Official Build) (64-bit)