`button` element presence can cause `role='button'` elements not to be found

tesk9 commented 2 years ago

I think elm-program-test 3.6.1 introduced a new bug around finding clickable "buttons".

For html that includes an arbitrary element with role="button", elm-program-test's clickButton works great unless there's also a regular button in the view as well.

So a test like this one:

issue144 : Test
issue144 =
    describe "regression test for #144"
        [ test "finds role='button' span when another button is also present" <|
            \() ->
                TestingProgram.startView
                    (Html.span []
                        [ Html.span
                            [ -- not including tabindex or other necessary attributes for test simplicity
                              Html.Attributes.attribute "role" "button"
                            , Html.Events.onClick (Log "CLICK")

                            -- Comment the following line in order to get a slighlty different failure:
                            , Html.Attributes.disabled False
                            ]
                            [ Html.map never (Html.text "Clickable element") ]
                        , Html.button [] [ Html.text "Panda" ]
                        ]
                    )
                    |> ProgramTest.clickButton "Clickable element"
                    |> ProgramTest.done
        ]

Produces a failure like this:

➜  elm-program-test (issue-144/3-6-1) npm test                                                          ✭

> test
> elm-test && npm run-script build-examples && elm make --docs docs.json && elm-format --validate . && elm diff

Compiling > Starting tests

elm-test 0.19.1-revision7
-------------------------

Running 162 tests. To reproduce these results, run: elm-test --fuzz 100 --seed 26956269746913

↓ ProgramTestTests.UserInput.ClickButtonTest
↓ clicking buttons
↓ regression test for #144
✗ finds role='button' span when another button is also present

    clickButton "Clickable element":
    ▼ Query.fromHtml

        <body>
            <span>
                <span role="button" disabled=false>
                    Clickable element
                </span>
                <button>
                    Panda
                </button>
            </span>
        </body>

    ▼ Query.has [ text "HTML expected by the call to: clickButton "Clickable element"" ]

    ✗ has text "HTML expected by the call to: clickButton "Clickable element""

    Expected one of the following to exist:
    - an element with role="button" (not disabled) and onClick and text "Clickable element"
        PLEASE REPORT THIS AT <https://github.com/avh4/elm-program-test/issues>: Got a failure message from Test.Html.Query that we couldn't parse: PLEASE REPORT THIS AT <https://github.com/avh4/elm-program-test/issues>: firstErrorOf: asked to report an error but none of the choices failed

I committed this test on a fork of elm-program-test here: https://github.com/tesk9/elm-program-test/tree/issue-144/3-6-1

I also added the same test against 3.6.0 to demo what the behavior used to be here: https://github.com/tesk9/elm-program-test/tree/issue-144/3-6-0

cc @Arkham.

tesk9 commented 2 years ago

(also -- I thought this would be issue 144, and clearly it's not. Sorry for the confusing naming)

avh4 commented 2 years ago

Does it work in https://package.elm-lang.org/packages/avh4/elm-program-test/3.6.2/ ?

avh4 commented 2 years ago

Or more accurately: the given test passes again in 3.6.2. Are there now any additional new errors you get on your test suite?

tesk9 commented 2 years ago

Yes, there are more errors -- I'm working to try to isolate them. I think at least one of the new errors should be happening. It looks like potentially a previously-passing test should have been failing. I should have some (hopefully not too messy) results to share soon!

avh4 commented 2 years ago

Ah yeah, the new expected errors would be if there's both a <button> and a <something role="button"> that both match the same text, that will now be an error but would previously pass. (To fix, you'd need to use within to narrow the search scope.)

Lmk about others. Feel free to start issues that are messy or just have a rough sentence sort of about what's failing or just the "pleaseReport:" line, since that might be enough for me to guess what's wrong.

tesk9 commented 2 years ago

Okay! So one of the tests that's failing seems to be finding a partial match ("Jeff Smith_3") for the button text that we're actually trying to click ("3") inside of a disabled menu component (the contents of the menu are in the DOM, but have properties like aria-hidden="true" on them.

I think the error parsing maybe has a mistake in it? It seems like "Query.find always expects to find 1 element, but it found 2 instead." is a result that elm-program-test should expect to get.

The 2 matches are both real buttons.

--

Edit: I can definitely work around this, now that I know what's wrong with it.

avh4 commented 2 years ago

=>

https://github.com/avh4/elm-program-test/issues/154 (Fixable, though I think it will only improve the error message you're seeing; not actually make your test pass)
https://github.com/avh4/elm-program-test/issues/155 (I think not fixable given the current capabilities of elm-explorations/test)
https://github.com/avh4/elm-program-test/issues/156 (I think current behavior is maybe the only one that makes sense without being able to determine computed CSS styles)

avh4 / elm-program-test

`button` element presence can cause `role='button'` elements not to be found #149