runtimeverification / mir-semantics

10 stars 3 forks source link

BUG test suite passes parsing ambiguities #235

Closed dkcumming closed 3 months ago

dkcumming commented 12 months ago

Problem

The pytest runner appears to classify ambiguous parses as passing. Running a test with a parsing ambiguity directly will create the expected [Error] Inner Parser: Parsing ambiguity. error. However make compiletest-parse-fail will not filter this test into compiletest-parse-fail.tsv, and make test-integration-parse will pass the test.

daniel@daniel-MS-7E06 kmir$ poetry run kmir parse --output pretty src/tests/integration/test-data/compiletest-rs/ui/traits/multidispatch-infer-convert-target.mir 
[Error] Inner Parser: Parsing ambiguity.
1: syntax Ref ::= "&" TypeNoBounds
    `&__MIR-TYPE-SYNTAX_Ref_TypeNoBounds`(`___MIR-TYPE-SYNTAX_TypePathEndSegment_PathIdentSegment_PathIdentSegmentEndSuffix`(#token("_3","LocalToken"),`_MIR-TYPE-SYNTAX_PathIdentSegmentSuffix`(.KList)))
2: syntax AddressOf ::= "&" PtrModifiers Place
    `&___MIR-RVALUE-SYNTAX_AddressOf_PtrModifiers_Place`(`_MIR-RVALUE-SYNTAX_PtrModifiers`(.KList),#token("_3","LocalToken"))
        Source(/tmp/tmpvdbnwuo6)
        Location(78,14,78,17)
        78 |            _8 = &_3;
           .                 ^~~
//--snip--

daniel@daniel-MS-7E06 kmir$ make test-integration-parse | grep "multidispatch-infer-convert-target.mir"
src/tests/integration/test_parse.py::test_compiletest[compiletest-rs/ui/traits/multidispatch-infer-convert-target.mir] 
[gw3] [ 57%] PASSED src/tests/integration/test_parse.py::test_compiletest[compiletest-rs/ui/traits/multidispatch-infer-convert-target.mir]

Solution

Find and fix issue in test runner and re-run tests

dkcumming commented 12 months ago

I had the test runner send the completed process of the above test to a file:

args=[
    '/tmp/pytest-of-daniel/pytest-51/popen-gw3/kbuild0/v6.0.145/llvm/parser_Mir_MIR-SYNTAX', 
    '/tmp/pytest-of-daniel/pytest-51/popen-gw3/test_compiletest_compiletest_r1/preprocessed.mir'
], 
returncode=0, 
stdout=
//--snip--
inj{SortAssign{}, SortStatementKind{}}(
    Lbl\'UndsEqlsUndsUnds\'MIR-STATEMENT-SYNTAX\'Unds\'Assign\'Unds\'Place\'Unds\'RValue{}(
        inj{SortLocalToken{}, SortPlace{}}(
            \\dv{SortLocalToken{}}(
                "_19"
            )
        ),
        Lblamb{SortRValue{}}(
            inj{SortAddressOf{}, SortRValue{}}(
                Lbl\'And-UndsUndsUnds\'MIR-RVALUE-SYNTAX\'Unds\'AddressOf\'Unds\'PtrModifiers\'Unds\'Place{}(
                    Lbl\'Unds\'MIR-RVALUE-SYNTAX\'Unds\'PtrModifiers{}(),inj{SortLocalToken{}, SortPlace{}}(
                        \\dv{SortLocalToken{}}(
                            "_20"
                        )
                    )
                )
            ),
            inj{SortRef{}, SortRValue{}}(
                Lbl\'And-UndsUnds\'MIR-TYPE-SYNTAX\'Unds\'Ref\'Unds\'TypeNoBounds{}(
                    inj{SortTypePathEndSegment{}, SortTypeNoBounds{}}(
                        Lbl\'UndsUndsUnds\'MIR-TYPE-SYNTAX\'Unds\'TypePathEndSegment\'Unds\'PathIdentSegment\'Unds\'PathIdentSegmentEndSuffix{}(
                            inj{SortLocalToken{}, SortPathIdentSegment{}}(
                                \\dv{SortLocalToken{}}(
                                    "_20"
                                )
                            ),inj{SortPathIdentSegmentSuffix{}, SortPathIdentSegmentEndSuffix{}}(
                                Lbl\'Unds\'MIR-TYPE-SYNTAX\'Unds\'PathIdentSegmentSuffix{}()
                            )
                        )
                    )
                )
            )
        )
    )
)
//--snip--

In particular ... Lblamb{ ... is described in this documentation on ambiguous languages.

Likely the difference stems from a fork in the production of the parsers. Investigation confirmed that this branch is not taken with poetry run kmir parse ... however the test runner does take the branch.

yanliu18 commented 8 months ago

@dkcumming would you re-evaluate if this bug is still valid? Since in the refactor, we would fix the glr-bison-parer. The parsing behavior should be consistent now.

dkcumming commented 8 months ago

It seems that the test I used when I first ran this is successful on both jit and bison-glr parser. However some tests will pass with bison-glr parser, and fail with error code 134 for jit. This is a list of all those that have this behaviour (there are 186).

parse-mir/lvalue-as-type.mir
compiletest-rs/ui/closures/2229_closure_analysis/run_pass/struct-pattern-matching-with-methods.mir
parse-mir/rust-expression-precedence.mir
compiletest-rs/ui/bind-by-move.mir
compiletest-rs/ui/hygiene/hygiene.mir
compiletest-rs/ui/overloaded/overloaded_deref_with_ref_pattern_issue15609.mir
compiletest-rs/ui/overloaded/overloaded-index-assoc-list.mir
compiletest-rs/ui/range_inclusive.mir
compiletest-rs/ui/sse2.mir
compiletest-rs/ui/associated-types/associated-types-binding-in-trait.mir
compiletest-rs/ui/half-open-range-patterns/range_pat_interactions0.mir
compiletest-rs/ui/env-funky-keys.mir
compiletest-rs/ui/no-core-2.mir
compiletest-rs/ui/try-operator-hygiene.mir
compiletest-rs/ui/numbers-arithmetic/u128-as-f32.mir
compiletest-rs/ui/numbers-arithmetic/next-power-of-two-overflow-ndebug.mir
compiletest-rs/ui/log-knows-the-names-of-variants.mir
compiletest-rs/ui/nul-characters.mir
compiletest-rs/ui/stdlib-unit-tests/log-knows-the-names-of-variants-in-std.mir
compiletest-rs/ui/numbers-arithmetic/u128.mir
compiletest-rs/ui/numbers-arithmetic/i128.mir
compiletest-rs/ui/derive-uninhabited-enum-38885.mir
compiletest-rs/ui/packed/packed-struct-vec.mir
compiletest-rs/ui/attrs-resolution.mir
compiletest-rs/ui/let-else/let-else-binding-explicit-mut-pass.mir
compiletest-rs/ui/let-else/let-else-ref-bindings-pass.mir
compiletest-rs/ui/let-else/let-else-bindings.mir
compiletest-rs/ui/union/union-packed.mir
compiletest-rs/ui/macros/macro-pat-pattern-followed-by-or.mir
compiletest-rs/ui/let-else/const-fn.mir
compiletest-rs/ui/let-else/let-else-temp-borrowck.mir
compiletest-rs/ui/macros/macro-interpolation.mir
compiletest-rs/ui/let-else/let-else-deref-coercion-annotated.mir
compiletest-rs/ui/let-else/issue-99975.mir
compiletest-rs/ui/let-else/let-else.mir
compiletest-rs/ui/let-else/let-else-source-expr-nomove-pass.mir
compiletest-rs/ui/let-else/let-else-run-pass.mir
compiletest-rs/ui/or-patterns/exhaustiveness-pass.mir
compiletest-rs/ui/or-patterns/basic-switch.mir
compiletest-rs/ui/macros/rfc-3086-metavar-expr/count-and-length-are-distinct.mir
compiletest-rs/ui/macros/try-macro.mir
compiletest-rs/ui/for-loop-while/foreach-external-iterators.mir
compiletest-rs/ui/for-loop-while/for-loop-unconstrained-element-type-i32-fallback.mir
compiletest-rs/ui/or-patterns/bindings-runpass-2.mir
compiletest-rs/ui/macros/macro-literal.mir
compiletest-rs/ui/macros/macro-pat-pattern-followed-by-or-in-2021.mir
compiletest-rs/ui/for-loop-while/for-loop-goofiness.mir
compiletest-rs/ui/derives/derive-partial-ord.mir
compiletest-rs/ui/for-loop-while/for-loop-mut-ref-element.mir
compiletest-rs/ui/for-loop-while/while-prelude-drop.mir
compiletest-rs/ui/for-loop-while/for-loop-macro.mir
compiletest-rs/ui/for-loop-while/foreach-external-iterators-loop.mir
compiletest-rs/ui/for-loop-while/issue-69841.mir
compiletest-rs/ui/macros/typeck-macro-interaction-issue-8852.mir
compiletest-rs/ui/for-loop-while/foreach-external-iterators-hashmap.mir
compiletest-rs/ui/for-loop-while/foreach-external-iterators-hashmap-break-restart.mir
compiletest-rs/ui/for-loop-while/foreach-external-iterators-nested.mir
compiletest-rs/ui/for-loop-while/foreach-external-iterators-break.mir
compiletest-rs/ui/macros/macro-pat-neg-lit.mir
compiletest-rs/ui/for-loop-while/loop-label-shadowing.mir
compiletest-rs/ui/zero-sized/zero-sized-btreemap-insert.mir
compiletest-rs/ui/zero-sized/zero-sized-linkedlist-push.mir
compiletest-rs/ui/zero-sized/zero-sized-binary-heap-push.mir
compiletest-rs/ui/or-patterns/struct-like.mir
compiletest-rs/ui/macros/macro-tt-followed-by-seq.mir
compiletest-rs/ui/or-patterns/basic-switchint.mir
compiletest-rs/ui/or-patterns/const-fn.mir
compiletest-rs/ui/or-patterns/if-let-while-let.mir
compiletest-rs/ui/or-patterns/issue-70413-no-unreachable-pat-and-guard.mir
compiletest-rs/ui/or-patterns/let-pattern.mir
compiletest-rs/ui/macros/issue-99265.mir
compiletest-rs/ui/macros/issue-8851.mir
compiletest-rs/ui/consts/const-match-pattern-arm.mir
compiletest-rs/ui/macros/macro-pat-follow.mir
compiletest-rs/ui/consts/unstable-precise-live-drops-in-libcore.mir
compiletest-rs/ui/consts/const-enum-structlike.mir
compiletest-rs/ui/macros/macro-pat.mir
compiletest-rs/ui/consts/issue-90762.mir
compiletest-rs/ui/consts/chained-constants-stackoverflow.mir
compiletest-rs/ui/consts/const-adt-align-mismatch.mir
compiletest-rs/ui/generator/match-bindings.mir
compiletest-rs/ui/generator/issue-58888.mir
compiletest-rs/ui/consts/const_constructor/const_constructor_qpath.mir
compiletest-rs/ui/consts/const-int-arithmetic.mir
compiletest-rs/ui/rfcs/rfc-2005-default-binding-mode/ref-region.mir
compiletest-rs/ui/self/explicit-self-closures.mir
compiletest-rs/ui/drop/issue-100276.mir
compiletest-rs/ui/rfcs/rfc-2005-default-binding-mode/general.mir
compiletest-rs/ui/consts/const_in_pattern/issue-65466.mir
compiletest-rs/ui/consts/const_in_pattern/issue-62614.mir
compiletest-rs/ui/consts/const_in_pattern/custom-eq-branch-pass.mir
compiletest-rs/ui/drop/drop-trait-enum.mir
compiletest-rs/ui/consts/const_in_pattern/custom-eq-branch-warn.mir
compiletest-rs/ui/drop/drop-on-empty-block-exit.mir
compiletest-rs/ui/drop/drop-if-let-binding.mir
compiletest-rs/ui/drop/issue-35546.mir
compiletest-rs/ui/rfcs/rfc-2175-or-if-while-let/basic.mir
compiletest-rs/ui/consts/control-flow/feature-gate-const-if-match.mir
compiletest-rs/ui/consts/control-flow/drop-precise.mir
compiletest-rs/ui/try-trait/try-poll.mir
compiletest-rs/ui/consts/const-eval/strlen.mir
compiletest-rs/ui/deref-patterns/refs.mir
compiletest-rs/ui/coercion/coerce-reborrow-imm-vec-arg.mir
compiletest-rs/ui/deref-patterns/basic.mir
compiletest-rs/ui/match/issue-82392.mir
compiletest-rs/ui/impl-trait/lifetimes2.mir
compiletest-rs/ui/mir/issue-99852.mir
compiletest-rs/ui/mir/issue-77359-simplify-arm-identity.mir
compiletest-rs/ui/mir/mir_build_match_comparisons.mir
compiletest-rs/ui/mir/mir_augmented_assignments.mir
compiletest-rs/ui/mir/issue-78496.mir
compiletest-rs/ui/mir/mir_match_arm_guard.mir
compiletest-rs/ui/pattern/issue-8351-1.mir
compiletest-rs/ui/pattern/move-ref-patterns/issue-53840.mir
compiletest-rs/ui/mir/mir_codegen_array_2.mir
compiletest-rs/ui/array-slice-vec/repeated-vector-syntax.mir
compiletest-rs/ui/mir/mir_codegen_array.mir
compiletest-rs/ui/mir/issue-66851.mir
compiletest-rs/ui/pattern/bindings-after-at/borrowck-pat-ref-both-sides.mir
compiletest-rs/ui/array-slice-vec/vec-macro-no-std.mir
compiletest-rs/ui/pattern/bindings-after-at/bind-by-copy.mir
compiletest-rs/ui/pattern/bindings-after-at/or-patterns.mir
compiletest-rs/ui/panic-runtime/abort.mir
compiletest-rs/ui/enum-discriminant/get_discr.mir
compiletest-rs/ui/panic-runtime/lto-abort.mir
compiletest-rs/ui/functions-closures/capture-clauses-boxed-closures.mir
compiletest-rs/ui/functions-closures/capture-clauses-unboxed-closures.mir
compiletest-rs/ui/array-slice-vec/mutability-inherits-through-fixed-length-vec.mir
compiletest-rs/ui/enum-discriminant/niche.mir
compiletest-rs/ui/enum-discriminant/issue-70509-partial_eq.mir
compiletest-rs/ui/codegen/issue-16602-3.mir
compiletest-rs/ui/cast/supported-cast.mir
compiletest-rs/ui/pattern/issue-8351-2.mir
compiletest-rs/ui/pattern/issue-6449.mir
compiletest-rs/ui/pattern/issue-11577.mir
compiletest-rs/ui/pattern/issue-10392.mir
compiletest-rs/ui/traits/issue-72455.mir
compiletest-rs/ui/pattern/issue-74954.mir
compiletest-rs/ui/binding/match-str.mir
compiletest-rs/ui/binding/fn-pattern-expected-type-2.mir
compiletest-rs/ui/binding/match-ref-binding-mut-option.mir
compiletest-rs/ui/traits/bug-7183-generics.mir
compiletest-rs/ui/binding/or-pattern.mir
compiletest-rs/ui/binding/nested-matchs.mir
compiletest-rs/ui/binding/match-tag.mir
compiletest-rs/ui/binding/match-enum-struct-0.mir
compiletest-rs/ui/binding/match-vec-alternatives.mir
compiletest-rs/ui/binding/match-ref-binding-in-guard-3256.mir
compiletest-rs/ui/binding/match-beginning-vert.mir
compiletest-rs/ui/binding/if-let.mir
compiletest-rs/ui/binding/match-ref-binding.mir
compiletest-rs/ui/binding/use-uninit-match.mir
compiletest-rs/ui/binding/match-arm-statics.mir
compiletest-rs/ui/binding/mut-in-ident-patterns.mir
compiletest-rs/ui/binding/match-enum-struct-1.mir
compiletest-rs/ui/binding/shadow.mir
compiletest-rs/ui/borrowck/borrowck-univariant-enum.mir
compiletest-rs/ui/binding/use-uninit-match2.mir
compiletest-rs/ui/binding/match-pattern-no-type-params.mir
compiletest-rs/ui/binding/nested-pattern.mir
compiletest-rs/ui/borrowck/borrowck-slice-pattern-element-loan-rpass.mir
compiletest-rs/ui/binding/match-in-macro.mir
compiletest-rs/ui/borrowck/borrowck-pat-enum.mir
compiletest-rs/ui/binding/match-with-ret-arm.mir
compiletest-rs/ui/borrowck/issue-51348-multi-ref-mut-in-guard.mir
compiletest-rs/ui/const-generics/arg-in-pat-1.mir
compiletest-rs/ui/runtime/atomic-print.mir
compiletest-rs/ui/const-generics/type-dependent/issue-61936.mir
compiletest-rs/ui/const-generics/issues/issue-61336-1.mir
compiletest-rs/ui/const-generics/generic_const_exprs/subexprs_are_const_evalutable.mir
compiletest-rs/ui/const-generics/generic_const_exprs/issue-97047-ice-2.mir
compiletest-rs/ui/deriving/deriving-cmp-generic-struct.mir
compiletest-rs/ui/deriving/deriving-cmp-generic-enum.mir
compiletest-rs/ui/deriving/deriving-clone-enum.mir
compiletest-rs/ui/deriving/deriving-show-2.mir
compiletest-rs/ui/deriving/deriving-enum-single-variant.mir
compiletest-rs/ui/deriving/deriving-cmp-shortcircuit.mir
compiletest-rs/ui/deriving/deriving-clone-generic-enum.mir
compiletest-rs/ui/deriving/deriving-via-extension-struct-like-enum-variant.mir
compiletest-rs/ui/deriving/deriving-default-enum.mir
compiletest-rs/ui/deriving/deriving-show.mir
compiletest-rs/ui/deriving/deriving-via-extension-hash-enum.mir
compiletest-rs/ui/deriving/deriving-via-extension-enum.mir
compiletest-rs/ui/deriving/deriving-cmp-generic-struct-enum.mir
compiletest-rs/ui/generics/generic-tag-match.mir
compiletest-rs/ui/generics/generic-tag-values.mir
mariaKt commented 6 months ago

These results are for the KMIR semantics tag: v0.2.43, with the corresponding K version v6.3.58.

I repeated the experiment of running the test parser.

make test-integration-parse

Then testing the tests designated as passed with the command bellow.

poetry run kmir parse --output pretty path-to-test

There were 192 tests that failed, previously indicated as passing with the test parser.

In these tests (6), the parser indicates a syntax error. The kmir parser fails with error code 1.

src/tests/integration/test-data/compiletest-rs/ui/consts/const-enum-cast.mir
src/tests/integration/test-data/compiletest-rs/ui/rfcs/rfc-2151-raw-identifiers/items.mir
src/tests/integration/test-data/compiletest-rs/ui/rfcs/rfc-2151-raw-identifiers/basic.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/expr-match-panic.mir
src/tests/integration/test-data/compiletest-rs/ui/lazy-type-alias-impl-trait/nested.mir
src/tests/integration/test-data/compiletest-rs/ui/dyn-star/const.mir

In these tests (14), the recursion limit is reached.

src/tests/integration/test-data/compiletest-rs/ui/numbers-arithmetic/u128-as-f32.mir
src/tests/integration/test-data/compiletest-rs/ui/numbers-arithmetic/u128.mir
src/tests/integration/test-data/compiletest-rs/ui/numbers-arithmetic/i128.mir
src/tests/integration/test-data/compiletest-rs/ui/macros/macro-literal.mir
src/tests/integration/test-data/compiletest-rs/ui/macros/issue-99265.mir
src/tests/integration/test-data/compiletest-rs/ui/macros/rfc-3086-metavar-expr/count-and-length-are-distinct.mir
src/tests/integration/test-data/compiletest-rs/ui/union/union-packed.mir
src/tests/integration/test-data/compiletest-rs/ui/range_inclusive.mir
src/tests/integration/test-data/compiletest-rs/ui/cast/supported-cast.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/match-vec-alternatives.mir
src/tests/integration/test-data/compiletest-rs/ui/enum-discriminant/get_discr.mir
src/tests/integration/test-data/compiletest-rs/ui/enum-discriminant/niche.mir
src/tests/integration/test-data/compiletest-rs/ui/consts/chained-constants-stackoverflow.mir
src/tests/integration/test-data/compiletest-rs/ui/consts/const-int-arithmetic.mir

In these tests (172), the failure is observed at the kmir print with an error code of 134. The issue is not necessarily in kmir print (or kore_print), but may be in the generated KORE.

src/tests/integration/test-data/compiletest-rs/ui/numbers-arithmetic/next-power-of-two-overflow-ndebug.mir
src/tests/integration/test-data/parse-mir/rust-expression-precedence.mir
src/tests/integration/test-data/compiletest-rs/ui/macros/macro-pat-neg-lit.mir
src/tests/integration/test-data/compiletest-rs/ui/macros/macro-pat.mir
src/tests/integration/test-data/compiletest-rs/ui/macros/issue-8851.mir
src/tests/integration/test-data/compiletest-rs/ui/macros/macro-pat-follow.mir
src/tests/integration/test-data/parse-mir/lvalue-as-type.mir
src/tests/integration/test-data/compiletest-rs/ui/macros/macro-pat-pattern-followed-by-or.mir
src/tests/integration/test-data/compiletest-rs/ui/macros/macro-interpolation.mir
src/tests/integration/test-data/compiletest-rs/ui/macros/try-macro.mir
src/tests/integration/test-data/compiletest-rs/ui/macros/macro-pat-pattern-followed-by-or-in-2021.mir
src/tests/integration/test-data/compiletest-rs/ui/macros/macro-tt-followed-by-seq.mir
src/tests/integration/test-data/compiletest-rs/ui/derive-uninhabited-enum-38885.mir
src/tests/integration/test-data/compiletest-rs/ui/attrs-resolution.mir
src/tests/integration/test-data/compiletest-rs/ui/stdlib-unit-tests/log-knows-the-names-of-variants-in-std.mir
src/tests/integration/test-data/compiletest-rs/ui/macros/typeck-macro-interaction-issue-8852.mir
src/tests/integration/test-data/compiletest-rs/ui/drop/drop-trait-enum.mir
src/tests/integration/test-data/compiletest-rs/ui/drop/drop-on-empty-block-exit.mir
src/tests/integration/test-data/compiletest-rs/ui/derives/derive-partial-ord.mir
src/tests/integration/test-data/compiletest-rs/ui/drop/drop-if-let-binding.mir
src/tests/integration/test-data/compiletest-rs/ui/drop/issue-100276.mir
src/tests/integration/test-data/compiletest-rs/ui/runtime/atomic-print.mir
src/tests/integration/test-data/compiletest-rs/ui/drop/issue-35546.mir
src/tests/integration/test-data/compiletest-rs/ui/or-patterns/let-pattern.mir
src/tests/integration/test-data/compiletest-rs/ui/or-patterns/issue-70413-no-unreachable-pat-and-guard.mir
src/tests/integration/test-data/compiletest-rs/ui/half-open-range-patterns/range_pat_interactions0.mir
src/tests/integration/test-data/compiletest-rs/ui/or-patterns/basic-switchint.mir
src/tests/integration/test-data/compiletest-rs/ui/or-patterns/exhaustiveness-pass.mir
src/tests/integration/test-data/compiletest-rs/ui/or-patterns/if-let-while-let.mir
src/tests/integration/test-data/compiletest-rs/ui/or-patterns/basic-switch.mir
src/tests/integration/test-data/compiletest-rs/ui/or-patterns/const-fn.mir
src/tests/integration/test-data/compiletest-rs/ui/let-else/const-fn.mir
src/tests/integration/test-data/compiletest-rs/ui/pattern/issue-10392.mir
src/tests/integration/test-data/compiletest-rs/ui/pattern/issue-8351-2.mir
src/tests/integration/test-data/compiletest-rs/ui/pattern/issue-74954.mir
src/tests/integration/test-data/compiletest-rs/ui/or-patterns/bindings-runpass-2.mir
src/tests/integration/test-data/compiletest-rs/ui/or-patterns/struct-like.mir
src/tests/integration/test-data/compiletest-rs/ui/pattern/issue-11577.mir
src/tests/integration/test-data/compiletest-rs/ui/pattern/issue-6449.mir
src/tests/integration/test-data/compiletest-rs/ui/pattern/issue-8351-1.mir
src/tests/integration/test-data/compiletest-rs/ui/pattern/bindings-after-at/or-patterns.mir
src/tests/integration/test-data/compiletest-rs/ui/pattern/bindings-after-at/bind-by-copy.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/if-let.mir
src/tests/integration/test-data/compiletest-rs/ui/pattern/bindings-after-at/borrowck-pat-ref-both-sides.mir
src/tests/integration/test-data/compiletest-rs/ui/pattern/move-ref-patterns/issue-53840.mir
src/tests/integration/test-data/compiletest-rs/ui/for-loop-while/for-loop-macro.mir
src/tests/integration/test-data/compiletest-rs/ui/for-loop-while/for-loop-goofiness.mir
src/tests/integration/test-data/compiletest-rs/ui/for-loop-while/loop-label-shadowing.mir
src/tests/integration/test-data/compiletest-rs/ui/for-loop-while/for-loop-mut-ref-element.mir
src/tests/integration/test-data/compiletest-rs/ui/for-loop-while/foreach-external-iterators-break.mir
src/tests/integration/test-data/compiletest-rs/ui/for-loop-while/while-prelude-drop.mir
src/tests/integration/test-data/compiletest-rs/ui/sse2.mir
src/tests/integration/test-data/compiletest-rs/ui/for-loop-while/foreach-external-iterators-hashmap.mir
src/tests/integration/test-data/compiletest-rs/ui/for-loop-while/foreach-external-iterators-hashmap-break-restart.mir
src/tests/integration/test-data/compiletest-rs/ui/nul-characters.mir
src/tests/integration/test-data/compiletest-rs/ui/for-loop-while/for-loop-unconstrained-element-type-i32-fallback.mir
src/tests/integration/test-data/compiletest-rs/ui/for-loop-while/foreach-external-iterators-loop.mir
src/tests/integration/test-data/compiletest-rs/ui/for-loop-while/foreach-external-iterators.mir
src/tests/integration/test-data/compiletest-rs/ui/no-core-2.mir
src/tests/integration/test-data/compiletest-rs/ui/for-loop-while/issue-69841.mir
src/tests/integration/test-data/compiletest-rs/ui/log-knows-the-names-of-variants.mir
src/tests/integration/test-data/compiletest-rs/ui/for-loop-while/foreach-external-iterators-nested.mir
src/tests/integration/test-data/compiletest-rs/ui/env-funky-keys.mir
src/tests/integration/test-data/compiletest-rs/ui/hygiene/hygiene.mir
src/tests/integration/test-data/compiletest-rs/ui/bind-by-move.mir
src/tests/integration/test-data/compiletest-rs/ui/rfcs/rfc-2175-or-if-while-let/basic.mir
src/tests/integration/test-data/compiletest-rs/ui/deref-patterns/basic.mir
src/tests/integration/test-data/compiletest-rs/ui/rfcs/rfc-2005-default-binding-mode/general.mir
src/tests/integration/test-data/compiletest-rs/ui/rfcs/rfc-2005-default-binding-mode/ref-region.mir
src/tests/integration/test-data/compiletest-rs/ui/impl-trait/lifetimes2.mir
src/tests/integration/test-data/compiletest-rs/ui/zero-sized/zero-sized-btreemap-insert.mir
src/tests/integration/test-data/compiletest-rs/ui/zero-sized/zero-sized-binary-heap-push.mir
src/tests/integration/test-data/compiletest-rs/ui/closures/2229_closure_analysis/run_pass/struct-pattern-matching-with-methods.mir
src/tests/integration/test-data/compiletest-rs/ui/functions-closures/capture-clauses-unboxed-closures.mir
src/tests/integration/test-data/compiletest-rs/ui/functions-closures/capture-clauses-boxed-closures.mir
src/tests/integration/test-data/compiletest-rs/ui/associated-types/associated-types-binding-in-trait.mir
src/tests/integration/test-data/compiletest-rs/ui/generator/match-bindings.mir
src/tests/integration/test-data/compiletest-rs/ui/try-operator-hygiene.mir
src/tests/integration/test-data/compiletest-rs/ui/generator/issue-58888.mir
src/tests/integration/test-data/compiletest-rs/ui/zero-sized/zero-sized-linkedlist-push.mir
src/tests/integration/test-data/compiletest-rs/ui/mir/mir_augmented_assignments.mir
src/tests/integration/test-data/compiletest-rs/ui/overloaded/overloaded_deref_with_ref_pattern_issue15609.mir
src/tests/integration/test-data/compiletest-rs/ui/mir/issue-99852.mir
src/tests/integration/test-data/compiletest-rs/ui/match/issue-82392.mir
src/tests/integration/test-data/compiletest-rs/ui/traits/bug-7183-generics.mir
src/tests/integration/test-data/compiletest-rs/ui/mir/issue-78496.mir
src/tests/integration/test-data/compiletest-rs/ui/self/explicit-self-closures.mir
src/tests/integration/test-data/compiletest-rs/ui/traits/issue-72455.mir
src/tests/integration/test-data/compiletest-rs/ui/overloaded/overloaded-index-assoc-list.mir
src/tests/integration/test-data/compiletest-rs/ui/mir/mir_codegen_array.mir
src/tests/integration/test-data/compiletest-rs/ui/mir/mir_build_match_comparisons.mir
src/tests/integration/test-data/compiletest-rs/ui/mir/mir_codegen_array_2.mir
src/tests/integration/test-data/compiletest-rs/ui/mir/issue-66851.mir
src/tests/integration/test-data/compiletest-rs/ui/mir/issue-77359-simplify-arm-identity.mir
src/tests/integration/test-data/compiletest-rs/ui/mir/mir_match_arm_guard.mir
src/tests/integration/test-data/compiletest-rs/ui/borrowck/borrowck-pat-enum.mir
src/tests/integration/test-data/compiletest-rs/ui/deriving/deriving-clone-enum.mir
src/tests/integration/test-data/compiletest-rs/ui/deriving/deriving-clone-generic-enum.mir
src/tests/integration/test-data/compiletest-rs/ui/deriving/deriving-cmp-generic-struct.mir
src/tests/integration/test-data/compiletest-rs/ui/borrowck/issue-51348-multi-ref-mut-in-guard.mir
src/tests/integration/test-data/compiletest-rs/ui/let-else/let-else-temp-borrowck.mir
src/tests/integration/test-data/compiletest-rs/ui/let-else/let-else-deref-coercion-annotated.mir
src/tests/integration/test-data/compiletest-rs/ui/deriving/deriving-enum-single-variant.mir
src/tests/integration/test-data/compiletest-rs/ui/let-else/issue-99975.mir
src/tests/integration/test-data/compiletest-rs/ui/let-else/let-else-run-pass.mir
src/tests/integration/test-data/compiletest-rs/ui/borrowck/borrowck-univariant-enum.mir
src/tests/integration/test-data/compiletest-rs/ui/let-else/let-else-ref-bindings-pass.mir
src/tests/integration/test-data/compiletest-rs/ui/let-else/let-else.mir
src/tests/integration/test-data/compiletest-rs/ui/borrowck/borrowck-slice-pattern-element-loan-rpass.mir
src/tests/integration/test-data/compiletest-rs/ui/deriving/deriving-show.mir
src/tests/integration/test-data/compiletest-rs/ui/let-else/let-else-bindings.mir
src/tests/integration/test-data/compiletest-rs/ui/let-else/let-else-binding-explicit-mut-pass.mir
src/tests/integration/test-data/compiletest-rs/ui/deriving/deriving-via-extension-enum.mir
src/tests/integration/test-data/compiletest-rs/ui/let-else/let-else-source-expr-nomove-pass.mir
src/tests/integration/test-data/compiletest-rs/ui/deriving/deriving-cmp-generic-struct-enum.mir
src/tests/integration/test-data/compiletest-rs/ui/deriving/deriving-via-extension-hash-enum.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/match-arm-statics.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/match-beginning-vert.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/match-in-macro.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/match-ref-binding-in-guard-3256.mir
src/tests/integration/test-data/compiletest-rs/ui/deriving/deriving-show-2.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/match-enum-struct-0.mir
src/tests/integration/test-data/compiletest-rs/ui/deriving/deriving-cmp-shortcircuit.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/use-uninit-match2.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/mut-in-ident-patterns.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/match-tag.mir
src/tests/integration/test-data/compiletest-rs/ui/deriving/deriving-via-extension-struct-like-enum-variant.mir
src/tests/integration/test-data/compiletest-rs/ui/deriving/deriving-default-enum.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/match-ref-binding-mut-option.mir
src/tests/integration/test-data/compiletest-rs/ui/deriving/deriving-cmp-generic-enum.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/use-uninit-match.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/nested-matchs.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/match-ref-binding.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/fn-pattern-expected-type-2.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/match-pattern-no-type-params.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/or-pattern.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/match-enum-struct-1.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/match-with-ret-arm.mir
src/tests/integration/test-data/compiletest-rs/ui/packed/packed-struct-vec.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/match-str.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/nested-pattern.mir
src/tests/integration/test-data/compiletest-rs/ui/binding/shadow.mir
src/tests/integration/test-data/compiletest-rs/ui/generics/generic-tag-values.mir
src/tests/integration/test-data/compiletest-rs/ui/generics/generic-tag-match.mir
src/tests/integration/test-data/compiletest-rs/ui/deref-patterns/refs.mir
src/tests/integration/test-data/compiletest-rs/ui/const-generics/arg-in-pat-1.mir
src/tests/integration/test-data/compiletest-rs/ui/enum-discriminant/issue-70509-partial_eq.mir
src/tests/integration/test-data/compiletest-rs/ui/const-generics/type-dependent/issue-61936.mir
src/tests/integration/test-data/compiletest-rs/ui/const-generics/generic_const_exprs/subexprs_are_const_evalutable.mir
src/tests/integration/test-data/compiletest-rs/ui/const-generics/generic_const_exprs/issue-97047-ice-2.mir
src/tests/integration/test-data/compiletest-rs/ui/array-slice-vec/repeated-vector-syntax.mir
src/tests/integration/test-data/compiletest-rs/ui/const-generics/issues/issue-61336-1.mir
src/tests/integration/test-data/compiletest-rs/ui/array-slice-vec/vec-macro-no-std.mir
src/tests/integration/test-data/compiletest-rs/ui/array-slice-vec/mutability-inherits-through-fixed-length-vec.mir
src/tests/integration/test-data/compiletest-rs/ui/panic-runtime/abort.mir
src/tests/integration/test-data/compiletest-rs/ui/panic-runtime/lto-abort.mir
src/tests/integration/test-data/compiletest-rs/ui/codegen/issue-16602-3.mir
src/tests/integration/test-data/compiletest-rs/ui/coercion/coerce-reborrow-imm-vec-arg.mir
src/tests/integration/test-data/compiletest-rs/ui/consts/issue-90762.mir
src/tests/integration/test-data/compiletest-rs/ui/consts/const-enum-structlike.mir
src/tests/integration/test-data/compiletest-rs/ui/consts/const-match-pattern-arm.mir
src/tests/integration/test-data/compiletest-rs/ui/consts/unstable-precise-live-drops-in-libcore.mir
src/tests/integration/test-data/compiletest-rs/ui/consts/const-adt-align-mismatch.mir
src/tests/integration/test-data/compiletest-rs/ui/consts/const_in_pattern/issue-65466.mir
src/tests/integration/test-data/compiletest-rs/ui/consts/const_constructor/const_constructor_qpath.mir
src/tests/integration/test-data/compiletest-rs/ui/consts/const_in_pattern/issue-62614.mir
src/tests/integration/test-data/compiletest-rs/ui/consts/const_in_pattern/custom-eq-branch-warn.mir
src/tests/integration/test-data/compiletest-rs/ui/consts/const_in_pattern/custom-eq-branch-pass.mir
src/tests/integration/test-data/compiletest-rs/ui/consts/const-eval/strlen.mir
src/tests/integration/test-data/compiletest-rs/ui/consts/control-flow/drop-precise.mir
src/tests/integration/test-data/compiletest-rs/ui/consts/control-flow/feature-gate-const-if-match.mir
src/tests/integration/test-data/compiletest-rs/ui/try-trait/try-poll.mir
dkcumming commented 6 months ago

I think that things have changed with KMIR and the original framing of this problem has shifted slightly. So I believe that the JIT parser is no longer in use and that can be seen from these lines that would ensure the parser field of the KMIR object is always the GLR parser. So the failing results when running locally are not able to be attributed to the JIT parser. To confirm this you can examine proc_res and see that it has the same stdout / kore between the test runner and local runner.

When running locally the error comes from the call to print_kore_to_xxx, which will error at the call to pyk function kore_print. The reason this errors when running locally is that unless changed, the default output is Pretty on cli which will error if tests have Lblamb (more below). However pytest does not call the parse through the cli path, and calls parse directly, this means that output defaults to none which means that it avoid the call to kore_print.

So currently, the JIT parser is not being called, and the output of the tests is the same between local and pytest, however only running locally will attempt to print them. I believe (unconfirmed) that the 134 occurs when the test has an Lblamb node from an ambiguity, and the printer does not have any instruction for how to print them. I could be wrong though, but this is my suspicion.

mariaKt commented 6 months ago

Repeating the experiment above, but using this command instead

poetry run kmir parse --output kore path-to-test

Then only 6 tests fail. These are the ones where the parser indicates a syntax error.

From the remaining 186 tests: