Inconsistency in AngularJS and Mithril implementations

[owickstrom]

Hi!

As part of writing a generative exploratory browser testing tool, I'm writing a state machine specification for TodoMVC. I'm trying it on the major JS framework implementations, deriving the correct behavior from whatever seems to be the common behavior.

When I ran it on the AngularJS and Mithril apps, it found the same inconsistency with my spec in both apps, that could be considered a bug. (I'm leaving that judgment up to you.) After the testing tool has shrunk down to a minimal failing case, I get the following minimal behavior:

Test runner output

``` Running test with size: 10 Running test with size: 20 Test failed. Shrinking... 1. State • byCss ".todo-list li" • text (byCss ".todo-list li label") • text (byCss ".todoapp .filters .selected") • text (byCss ".todoapp .todo-count strong") • property "checked" (byCss ".todo-list li input[type=checkbox]") • property "value" (byCss ".new-todo") - String "" 2. key press 'a' 3. State • byCss ".todo-list li" • text (byCss ".todo-list li label") • text (byCss ".todoapp .filters .selected") • text (byCss ".todoapp .todo-count strong") • property "checked" (byCss ".todo-list li input[type=checkbox]") • property "value" (byCss ".new-todo") - String "a" 4. key press '\57351' 5. State • byCss ".todo-list li" - Element {ref = "1931affa-8b3c-47f2-896c-4cffd66f9429"} • text (byCss ".todo-list li label") - "a" • text (byCss ".todoapp .filters .selected") - "All" • text (byCss ".todoapp .todo-count strong") - "1" • property "checked" (byCss ".todo-list li input[type=checkbox]") - Bool False • property "value" (byCss ".new-todo") - String "" 6. key press 'a' 7. State • byCss ".todo-list li" - Element {ref = "1931affa-8b3c-47f2-896c-4cffd66f9429"} • text (byCss ".todo-list li label") - "a" • text (byCss ".todoapp .filters .selected") - "All" • text (byCss ".todoapp .todo-count strong") - "1" • property "checked" (byCss ".todo-list li input[type=checkbox]") - Bool False • property "value" (byCss ".new-todo") - String "a" 8. click .todoapp .filters a:not(.selected)[1] 9. State • byCss ".todo-list li" • text (byCss ".todo-list li label") • text (byCss ".todoapp .filters .selected") - "Completed" • text (byCss ".todoapp .todo-count strong") - "1" • property "checked" (byCss ".todo-list li input[type=checkbox]") • property "value" (byCss ".new-todo") - String "" ```

I suspect that doesn't make too much sense (I haven't spent much time on nice output yet). To clarify, it can be read as:

1. Press a
2. Press Enter
3. (a new item is added)
4. Press a
5. Click the second non-selected filter (which is "Completed")
6. (the input value is cleared)

In that last step, my specification rejects the behavior. The other mainstream framework implementations retain the pending input value when you change filter.

In any case, I thought I might report this and you can decide if you consider it a bug, and if it's worth doing anything about it.

Cheers!

[AndreasHassing]

In case anyone is wondering what this generative exploratory browser tool is: https://webcheck.tools/

This is super cool @owickstrom, looking forward to updates.

Here is someone smarter than me saying something smart about it: https://twitter.com/ScottWlaschin/status/1273884973578309632

[owickstrom]

A follow-up blog post on testing the remaining implementations: https://wickstrom.tech/programming/2020/07/02/the-todomvc-showdown-testing-with-webcheck.html. I hope it can be of use! Let me know if there are any questions.