llir / llvm

Library for interacting with LLVM IR in pure Go.
https://llir.github.io/document/
BSD Zero Clause License
1.19k stars 78 forks source link

fix remaining 0.03% of failing test cases from LLVM test case suite #111

Closed mewmew closed 4 years ago

mewmew commented 4 years ago

Currently we have integrated test cases from Coreutils, SQLite and the official LLVM project, and as of #110 we are passing all test cases of implemented language features of LLVM. There are still a few language features left to implement, that were added in LLVM 9.0, and these are tracked by #101, #106, and #43.

To reduce the time needed to run go test on llir/llvm, we have only enabled a carefully selected subset of the test cases, as there are 20k+ test cases of LLVM.

As of rev fba5559079e076debfe70cb2bbb81bf183f28437, if we were to enable all test cases of Coreutils, SQLite and LLVM, we would pass ~96.5% of the test cases (20010/20723 = 96.5%).

As such, this issue tracks the remaining 3.5% of the test cases which are currently failing (713 test cases fail of the total 20723 test cases).

We will start to categorize the failing test cases, and quite likely there are some ~20 language concepts left to implement which will help us pass all test cases of LLVM.

The list of currently failing test cases are as follows.

Not yet resolved (0.2%)

Hexadecimal floating-point literals (0.2% of failing test cases)

0.2% of currently failing test cases (2/713) are related to hexadecimal floating-point literals.

This is tracked by issue #31.

remaining test cases failing
* [llvm/test/CodeGen/PowerPC/fast-isel-call.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/PowerPC/fast-isel-call.ll) - strconv.ParseUint: parsing "1397723CCABD0000401C666660000000`value out of range * [llvm/test/Transforms/InstCombine/fneg.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/InstCombine/fneg.ll) - strconv.ParseUint: parsing "7FF80000000000000`value out of range
mewmew commented 4 years ago

@dannypsnl, would you like to take a look at the missing enums? :)

Edit I went ahead and added the enums.

mewmew commented 4 years ago

Resolved (99.9%)

Alignment attribute of function definitions (39% of previously failing test cases)

39% of the previously failing test cases (279/713) were related to alignment attribute of function definitions.

Fixed in rev d94c2ac4e5403daba2cc8c52da25aca4f78601f4.

test cases failing related to function alignment
* `llvm/test/Analysis/CostModel/SystemZ/intrinsic-cost-crash.ll` * `llvm/test/Analysis/DependenceAnalysis/MIVCheckConst.ll` * `llvm/test/Analysis/DependenceAnalysis/NonAffineExpr.ll` * `llvm/test/Analysis/MemorySSA/cyclicphi.ll` * `llvm/test/Analysis/MemorySSA/loop-rotate-inv-template.ll` * `llvm/test/Analysis/MemorySSA/loop_rotate_remove_trivial_phis.ll` * `llvm/test/Analysis/MemorySSA/reduce_clobber_limit.ll` * `llvm/test/Analysis/MemorySSA/update_unroll.ll` * `llvm/test/Analysis/ScalarEvolution/pr22674.ll` * `llvm/test/Analysis/TypeBasedAliasAnalysis/PR17620.ll` * `llvm/test/Analysis/ValueTracking/func-ptr-lsb.ll` * `llvm/test/Bitcode/compatibility-3.8.ll` * `llvm/test/Bitcode/compatibility-3.9.ll` * `llvm/test/Bitcode/compatibility-4.0.ll` * `llvm/test/Bitcode/compatibility-5.0.ll` * `llvm/test/Bitcode/compatibility-6.0.ll` * `llvm/test/CodeGen/AArch64/PBQP-csr.ll` * `llvm/test/CodeGen/AArch64/analyzecmp.ll` * `llvm/test/CodeGen/AArch64/and-mask-removal.ll` * `llvm/test/CodeGen/AArch64/arm64-2011-03-09-CPSRSpill.ll` * `llvm/test/CodeGen/AArch64/arm64-2012-01-11-ComparisonDAGCrash.ll` * `llvm/test/CodeGen/AArch64/arm64-andCmpBrToTBZ.ll` * `llvm/test/CodeGen/AArch64/arm64-atomic.ll` * `llvm/test/CodeGen/AArch64/arm64-bitfield-extract.ll` * `llvm/test/CodeGen/AArch64/branch-relax-alignment.ll` * `llvm/test/CodeGen/AArch64/sched-past-vector-ldst.ll` * `llvm/test/CodeGen/AArch64/vector_merge_dep_check.ll` * `llvm/test/CodeGen/AMDGPU/hsa-func-align.ll` * `llvm/test/CodeGen/AMDGPU/hsa-func.ll` * `llvm/test/CodeGen/AMDGPU/nop-data.ll` * `llvm/test/CodeGen/ARM/2010-06-21-nondarwin-tc.ll` * `llvm/test/CodeGen/ARM/2010-07-26-GlobalMerge.ll` * `llvm/test/CodeGen/ARM/2010-08-04-StackVariable.ll` * `llvm/test/CodeGen/ARM/2011-05-04-MultipleLandingPadSuccs.ll` * `llvm/test/CodeGen/ARM/2012-01-24-RegSequenceLiveRange.ll` * `llvm/test/CodeGen/ARM/2012-01-26-CoalescerBug.ll` * `llvm/test/CodeGen/ARM/2012-04-02-TwoAddrInstrCrash.ll` * `llvm/test/CodeGen/ARM/2012-04-10-DAGCombine.ll` * `llvm/test/CodeGen/ARM/2016-05-01-RegScavengerAssert.ll` * `llvm/test/CodeGen/ARM/Windows/wineh-basic.ll` * `llvm/test/CodeGen/ARM/alias_store.ll` * `llvm/test/CodeGen/ARM/arm-and-tst-peephole.ll` * `llvm/test/CodeGen/ARM/call-tc.ll` * `llvm/test/CodeGen/ARM/crash-greedy.ll` * `llvm/test/CodeGen/ARM/crash.ll` * `llvm/test/CodeGen/ARM/fast-isel-align.ll` * `llvm/test/CodeGen/ARM/gv-stubs-crash.ll` * `llvm/test/CodeGen/ARM/ifcvt-branch-weight-bug.ll` * `llvm/test/CodeGen/ARM/inlineasm-switch-mode-oneway-from-arm.ll` * `llvm/test/CodeGen/ARM/inlineasm-switch-mode-oneway-from-thumb.ll` * `llvm/test/CodeGen/ARM/inlineasm-switch-mode.ll` * `llvm/test/CodeGen/ARM/interval-update-remat.ll` * `llvm/test/CodeGen/ARM/ldrcppic.ll` * `llvm/test/CodeGen/ARM/neon_spill.ll` * `llvm/test/CodeGen/ARM/select_xform.ll` * `llvm/test/CodeGen/ARM/vmul.ll` * `llvm/test/CodeGen/ARM/vst2.ll` * `llvm/test/CodeGen/Generic/exception-handling.ll` * `llvm/test/CodeGen/Hexagon/P08214.ll` * `llvm/test/CodeGen/Hexagon/addrmode-keepdeadphis.ll` * `llvm/test/CodeGen/Hexagon/bit-bitsplit-regclass.ll` * `llvm/test/CodeGen/Hexagon/bit-loop-rc-mismatch.ll` * `llvm/test/CodeGen/Hexagon/branchfolder-keep-impdef.ll` * `llvm/test/CodeGen/Hexagon/bug14859-iv-cleanup-lpad.ll` * `llvm/test/CodeGen/Hexagon/bug14859-split-const-block-addr.ll` * `llvm/test/CodeGen/Hexagon/find-loop-instr.ll` * `llvm/test/CodeGen/Hexagon/global-ctor-pcrel.ll` * `llvm/test/CodeGen/Hexagon/hexagon-verify-implicit-use.ll` * `llvm/test/CodeGen/Hexagon/hvx-loopidiom-memcpy.ll` * `llvm/test/CodeGen/Hexagon/hwloop-subreg.ll` * `llvm/test/CodeGen/Hexagon/opt-addr-mode-subreg-use.ll` * `llvm/test/CodeGen/Hexagon/rdf-filter-defs.ll` * `llvm/test/CodeGen/Hexagon/regscavengerbug.ll` * `llvm/test/CodeGen/Hexagon/save-kill-csr.ll` * `llvm/test/CodeGen/Hexagon/swp-change-deps.ll` * `llvm/test/CodeGen/Hexagon/swp-loop-carried-crash.ll` * `llvm/test/CodeGen/Mips/pbqp-reserved-physreg.ll` * `llvm/test/CodeGen/NVPTX/function-align.ll` * `llvm/test/CodeGen/PowerPC/BreakableToken-reduced.ll` * `llvm/test/CodeGen/PowerPC/aantidep-inline-asm-use.ll` * `llvm/test/CodeGen/PowerPC/ec-input.ll` * `llvm/test/CodeGen/PowerPC/fast-isel-icmp-split.ll` * `llvm/test/CodeGen/PowerPC/fast-isel-load-store-vsx.ll` * `llvm/test/CodeGen/PowerPC/fastcc_stacksize.ll` * `llvm/test/CodeGen/PowerPC/glob-comp-aa-crash.ll` * `llvm/test/CodeGen/PowerPC/licm-remat.ll` * `llvm/test/CodeGen/PowerPC/memset-nc-le.ll` * `llvm/test/CodeGen/PowerPC/memset-nc.ll` * `llvm/test/CodeGen/PowerPC/merge-st-chain-op.ll` * `llvm/test/CodeGen/PowerPC/pr15031.ll` * `llvm/test/CodeGen/PowerPC/pr18663-2.ll` * `llvm/test/CodeGen/PowerPC/pr18663.ll` * `llvm/test/CodeGen/PowerPC/pr24636.ll` * `llvm/test/CodeGen/PowerPC/pr27350.ll` * `llvm/test/CodeGen/PowerPC/pr28130.ll` * `llvm/test/CodeGen/PowerPC/rlwimi-and.ll` * `llvm/test/CodeGen/PowerPC/split-index-tc.ll` * `llvm/test/CodeGen/PowerPC/stwu8.ll` * `llvm/test/CodeGen/PowerPC/tls-cse.ll` * `llvm/test/CodeGen/PowerPC/toc-load-sched-bug.ll` * `llvm/test/CodeGen/SPARC/missinglabel.ll` * `llvm/test/CodeGen/Thumb/2011-05-11-DAGLegalizer.ll` * `llvm/test/CodeGen/Thumb2/constant-islands.ll` * `llvm/test/CodeGen/Thumb2/crash.ll` * `llvm/test/CodeGen/Thumb2/ifcvt-rescan-bug-2016-08-22.ll` * `llvm/test/CodeGen/X86/2009-11-17-UpdateTerminator.ll` * `llvm/test/CodeGen/X86/2010-01-13-OptExtBug.ll` * `llvm/test/CodeGen/X86/2010-03-17-ISelBug.ll` * `llvm/test/CodeGen/X86/2010-04-06-SSEDomainFixCrash.ll` * `llvm/test/CodeGen/X86/2010-04-29-CoalescerCrash.ll` * `llvm/test/CodeGen/X86/2010-06-01-DeadArg-DbgInfo.ll` * `llvm/test/CodeGen/X86/2010-07-11-FPStackLoneUse.ll` * `llvm/test/CodeGen/X86/2010-08-04-StackVariable.ll` * `llvm/test/CodeGen/X86/2010-09-30-CMOV-JumpTable-PHI.ll` * `llvm/test/CodeGen/X86/2011-03-08-Sched-crash.ll` * `llvm/test/CodeGen/X86/2012-01-10-UndefExceptionEdge.ll` * `llvm/test/CodeGen/X86/2012-04-09-TwoAddrPassBug.ll` * `llvm/test/CodeGen/X86/2012-05-19-CoalescerCrash.ll` * `llvm/test/CodeGen/X86/2012-08-28-UnsafeMathCrash.ll` * `llvm/test/CodeGen/X86/2012-10-02-DAGCycle.ll` * `llvm/test/CodeGen/X86/2012-10-03-DAGCycle.ll` * `llvm/test/CodeGen/X86/2012-12-14-v8fp80-crash.ll` * `llvm/test/CodeGen/X86/asm-label2.ll` * `llvm/test/CodeGen/X86/atom-fixup-lea4.ll` * `llvm/test/CodeGen/X86/cmp.ll` * `llvm/test/CodeGen/X86/code_placement_eh.ll` * `llvm/test/CodeGen/X86/crash.ll` * `llvm/test/CodeGen/X86/dbg-changes-codegen.ll` * `llvm/test/CodeGen/X86/debug-loclists.ll` * `llvm/test/CodeGen/X86/fast-isel-x86-64.ll` * `llvm/test/CodeGen/X86/hidden-vis-pic.ll` * `llvm/test/CodeGen/X86/large-gep-chain.ll` * `llvm/test/CodeGen/X86/merge-vector-stores-scale-idx-crash.ll` * `llvm/test/CodeGen/X86/misched-aa-colored.ll` * `llvm/test/CodeGen/X86/misched-aa-mmos.ll` * `llvm/test/CodeGen/X86/misched-new.ll` * `llvm/test/CodeGen/X86/norex-subreg.ll` * `llvm/test/CodeGen/X86/pr14098.ll` * `llvm/test/CodeGen/X86/pr38743.ll` * `llvm/test/CodeGen/X86/stdcall-notailcall.ll` * `llvm/test/CodeGen/X86/tail-dup-repeat.ll` * `llvm/test/CodeGen/X86/tailcall-returndup-void.ll` * `llvm/test/CodeGen/X86/tailcall-ri64.ll` * `llvm/test/CodeGen/X86/twoaddr-sink-terminator.ll` * `llvm/test/CodeGen/X86/vector-trunc-widen.ll` * `llvm/test/CodeGen/X86/x86-mixed-alignment-dagcombine.ll` * `llvm/test/DebugInfo/AArch64/cfi-eof-prologue.ll` * `llvm/test/DebugInfo/ARM/cfi-eof-prologue.ll` * `llvm/test/DebugInfo/COFF/fpo-argsize.ll` * `llvm/test/DebugInfo/COFF/lambda.ll` * `llvm/test/DebugInfo/COFF/scopes.ll` * `llvm/test/DebugInfo/COFF/type-quals.ll` * `llvm/test/DebugInfo/COFF/types-calling-conv.ll` * `llvm/test/DebugInfo/COFF/types-recursive-unnamed.ll` * `llvm/test/DebugInfo/Generic/2010-03-24-MemberFn.ll` * `llvm/test/DebugInfo/Generic/2010-04-06-NestedFnDbgInfo.ll` * `llvm/test/DebugInfo/Generic/PR20038.ll` * `llvm/test/DebugInfo/Generic/def-line.ll` * `llvm/test/DebugInfo/Generic/dwarf-public-names.ll` * `llvm/test/DebugInfo/Generic/incorrect-variable-debugloc.ll` * `llvm/test/DebugInfo/Generic/linkage-name-abstract.ll` * `llvm/test/DebugInfo/Generic/member-order.ll` * `llvm/test/DebugInfo/Generic/recursive_inlining.ll` * `llvm/test/DebugInfo/Generic/tu-composite.ll` * `llvm/test/DebugInfo/X86/2011-12-16-BadStructRef.ll` * `llvm/test/DebugInfo/X86/DW_AT_linkage_name.ll` * `llvm/test/DebugInfo/X86/DW_AT_object_pointer.ll` * `llvm/test/DebugInfo/X86/cu-ranges-odr.ll` * `llvm/test/DebugInfo/X86/dbg-declare-arg.ll` * `llvm/test/DebugInfo/X86/dwarf-public-names.ll` * `llvm/test/DebugInfo/X86/elf-names.ll` * `llvm/test/DebugInfo/X86/fission-inline.ll` * `llvm/test/DebugInfo/X86/generate-odr-hash.ll` * `llvm/test/DebugInfo/X86/gnu-public-names.ll` * `llvm/test/DebugInfo/X86/linkage-name.ll` * `llvm/test/DebugInfo/X86/pr11300.ll` * `llvm/test/DebugInfo/X86/pr12831.ll` * `llvm/test/DebugInfo/X86/reference-argument.ll` * `llvm/test/DebugInfo/X86/sret.ll` * `llvm/test/DebugInfo/X86/void-typedef.ll` * `llvm/test/DebugInfo/X86/xray-split-dwarf-interaction.ll` * `llvm/test/ExecutionEngine/OrcLazy/global-ctors-and-dtors.ll` * `llvm/test/ExecutionEngine/OrcLazy/hello.ll` * `llvm/test/Feature/alignment.ll` * `llvm/test/Instrumentation/SanitizerCoverage/coverage-dbg.ll` * `llvm/test/LTO/Resolution/X86/Inputs/appending-var-2.ll` * `llvm/test/Linker/Inputs/alignment.ll` * `llvm/test/Linker/Inputs/replaced-function-matches-first-subprogram.ll` * `llvm/test/Linker/alignment.ll` * `llvm/test/Linker/replaced-function-matches-first-subprogram.ll` * `llvm/test/Linker/type-unique-odr-b.ll` * `llvm/test/Linker/type-unique-simple2-a.ll` * `llvm/test/Linker/type-unique-simple2-b.ll` * `llvm/test/Linker/type-unique-type-array-a.ll` * `llvm/test/Linker/type-unique-type-array-b.ll` * `llvm/test/SymbolRewriter/rewrite.ll` * `llvm/test/ThinLTO/X86/Inputs/dicompositetype-unique-alias.ll` * `llvm/test/ThinLTO/X86/Inputs/dicompositetype-unique2.ll` * `llvm/test/ThinLTO/X86/Inputs/referenced_by_constant.ll` * `llvm/test/ThinLTO/X86/dicompositetype-unique-alias.ll` * `llvm/test/ThinLTO/X86/dicompositetype-unique2.ll` * `llvm/test/Transforms/AlignmentFromAssumptions/start-unk.ll` * `llvm/test/Transforms/CodeGenPrepare/basic.ll` * `llvm/test/Transforms/DeadStoreElimination/cs-cs-aliasing.ll` * `llvm/test/Transforms/GVN/crash.ll` * `llvm/test/Transforms/GVN/nonescaping-malloc.ll` * `llvm/test/Transforms/GVN/stale-loop-info.ll` * `llvm/test/Transforms/GlobalDCE/global_ctors_integration.ll` * `llvm/test/Transforms/GlobalOpt/cxx-dtor.ll` * `llvm/test/Transforms/Inline/crash2.ll` * `llvm/test/Transforms/Inline/devirtualize-2.ll` * `llvm/test/Transforms/Inline/devirtualize.ll` * `llvm/test/Transforms/InstCombine/crash.ll` * `llvm/test/Transforms/InstSimplify/require-dominator.ll` * `llvm/test/Transforms/JumpThreading/ddt-crash.ll` * `llvm/test/Transforms/JumpThreading/ddt-crash2.ll` * `llvm/test/Transforms/JumpThreading/landing-pad.ll` * `llvm/test/Transforms/LICM/loopsink-pr39570.ll` * `llvm/test/Transforms/LoopDeletion/2011-06-21-phioperands.ll` * `llvm/test/Transforms/LoopSimplify/dbg-loc.ll` * `llvm/test/Transforms/LoopStrengthReduce/2011-10-14-IntPtr.ll` * `llvm/test/Transforms/LoopStrengthReduce/2012-03-15-nopreheader.ll` * `llvm/test/Transforms/LoopStrengthReduce/2013-01-14-ReuseCast.ll` * `llvm/test/Transforms/LoopStrengthReduce/X86/2009-11-10-LSRCrash.ll` * `llvm/test/Transforms/LoopUnroll/2011-08-08-PhiUpdate.ll` * `llvm/test/Transforms/LoopUnroll/2011-08-09-PhiUpdate.ll` * `llvm/test/Transforms/LoopUnswitch/2011-09-26-EHCrash.ll` * `llvm/test/Transforms/LoopUnswitch/2012-04-30-LoopUnswitch-LPad-Crash.ll` * `llvm/test/Transforms/LoopVectorize/X86/vector_ptr_load_store.ll` * `llvm/test/Transforms/LoopVectorize/lcssa-crash.ll` * `llvm/test/Transforms/MergeFunc/2011-02-08-RemoveEqual.ll` * `llvm/test/Transforms/MergeFunc/2013-01-10-MergeFuncAssert.ll` * `llvm/test/Transforms/MergeFunc/crash.ll` * `llvm/test/Transforms/MergeFunc/inttoptr-address-space.ll` * `llvm/test/Transforms/MergeFunc/inttoptr.ll` * `llvm/test/Transforms/MergeFunc/vector.ll` * `llvm/test/Transforms/NewGVN/crash.ll` * `llvm/test/Transforms/NewGVN/nonescaping-malloc.ll` * `llvm/test/Transforms/NewGVN/phi-edge-handling.ll` * `llvm/test/Transforms/NewGVN/pr31501.ll` * `llvm/test/Transforms/NewGVN/stale-loop-info.ll` * `llvm/test/Transforms/PhaseOrdering/2010-03-22-empty-baseclass.ll` * `llvm/test/Transforms/PhaseOrdering/gdce.ll` * `llvm/test/Transforms/Reg2Mem/crash.ll` * `llvm/test/Transforms/SLPVectorizer/X86/crash_bullet3.ll` * `llvm/test/Transforms/SLPVectorizer/X86/resched.ll` * `llvm/test/Transforms/SimpleLoopUnswitch/2011-09-26-EHCrash.ll` * `llvm/test/Transforms/SimpleLoopUnswitch/2012-04-30-LoopUnswitch-LPad-Crash.ll` * `llvm/test/Transforms/Util/strip-nonlinetable-debuginfo-containingtypes.ll` * `llvm/test/Bitcode/DILocation-implicit-code.ll` * `llvm/test/CodeGen/AArch64/vararg-tallcall.ll` * `llvm/test/CodeGen/ARM/lsr-undef-in-binop.ll` * `llvm/test/CodeGen/Mips/lw16-base-reg.ll` * `llvm/test/CodeGen/PowerPC/noPermuteFormasking.ll` * `llvm/test/CodeGen/WinCFGuard/cfguard.ll` * `llvm/test/DebugInfo/COFF/global-type-hashes.ll` * `llvm/test/DebugInfo/COFF/global_visibility.ll` * `llvm/test/DebugInfo/COFF/inheritance.ll` * `llvm/test/DebugInfo/COFF/thunk.ll` * `llvm/test/DebugInfo/COFF/types-cvarargs.ll` * `llvm/test/DebugInfo/COFF/types-data-members.ll` * `llvm/test/DebugInfo/COFF/vftables.ll` * `llvm/test/DebugInfo/COFF/virtual-method-kinds.ll` * `llvm/test/DebugInfo/COFF/virtual-methods.ll` * `llvm/test/DebugInfo/COFF/vtable-optzn-array.ll` * `llvm/test/DebugInfo/X86/decl-derived-member.ll` * `llvm/test/DebugInfo/X86/template_function_decl.ll` * `llvm/test/ThinLTO/X86/linkonce_aliasee_ref_import.ll` * `llvm/test/ThinLTO/X86/pr35472.ll` * `llvm/test/ThinLTO/X86/weak_externals.ll` * `llvm/test/Transforms/GlobalOpt/evaluate-bitcast-2.ll` * `llvm/test/Transforms/GlobalOpt/evaluate-bitcast-3.ll` * `llvm/test/Transforms/GlobalOpt/evaluate-constfold-call.ll` * `llvm/test/Transforms/GlobalOpt/static-const-bitcast.ll` * `llvm/test/Transforms/InstCombine/intptr6.ll` * `llvm/test/Transforms/LoopVectorize/pr36311.ll` * `llvm/test/Transforms/MergeICmps/X86/pr36557.ll` * `llvm/test/Transforms/SLPVectorizer/X86/pr35497.ll` * `llvm/test/Transforms/SampleProfile/inline-combine.ll`

Grammar related to Global Alignment Attribute (2% of previously failing test cases)

2% of the previously failing test cases (15/713) were related to alignment attribute of global declarations.

Fixed in rev d94c2ac4e5403daba2cc8c52da25aca4f78601f4.

test cases failing related to Global Alignment Attribute grammar
* `llvm/test/CodeGen/AMDGPU/elf.metadata.ll` * `llvm/test/CodeGen/SystemZ/int-cmp-36.ll` * `llvm/test/CodeGen/SystemZ/int-cmp-37.ll` * `llvm/test/CodeGen/SystemZ/int-cmp-38.ll` * `llvm/test/CodeGen/SystemZ/int-cmp-39.ll` * `llvm/test/CodeGen/SystemZ/int-cmp-40.ll` * `llvm/test/CodeGen/SystemZ/int-cmp-41.ll` * `llvm/test/CodeGen/SystemZ/int-cmp-42.ll` * `llvm/test/CodeGen/SystemZ/int-cmp-43.ll` * `llvm/test/CodeGen/SystemZ/int-move-08.ll` * `llvm/test/CodeGen/SystemZ/int-move-09.ll` * `llvm/test/CodeGen/X86/alignment.ll` * `llvm/test/Instrumentation/AddressSanitizer/global_metadata_external_comdat.ll` * `llvm/test/MC/WebAssembly/explicit-sections.ll` * `llvm/test/Transforms/GlobalOpt/SROA-section.ll`

Grammar related to callbr Instruction (3% of previously failing test cases)

3% of the previously failing test cases (23/713) were related to alignment attribute of global declarations.

Fixed in rev a26170572a7b745bbfaf195da89d5790fc1afab3.

test cases failing related to callbr Instruction grammar
* `llvm/test/Bitcode/callbr.ll` - syntax error at [line 9](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/callbr.ll#L9) * `llvm/test/CodeGen/AArch64/callbr-asm-label.ll` - syntax error at [line 13](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/AArch64/callbr-asm-label.ll#L13) * `llvm/test/CodeGen/AArch64/callbr-asm-obj-file.ll` - syntax error at [line 24](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/AArch64/callbr-asm-obj-file.ll#L24) * `llvm/test/CodeGen/SystemZ/asm-20.ll` - syntax error at [line 7](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/SystemZ/asm-20.ll#L7) * `llvm/test/CodeGen/X86/callbr-asm-bb-exports.ll` - syntax error at [line 26](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/callbr-asm-bb-exports.ll#L26) * `llvm/test/CodeGen/X86/callbr-asm-blockplacement.ll` - syntax error at [line 88](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/callbr-asm-blockplacement.ll#L88) * `llvm/test/CodeGen/X86/callbr-asm-branch-folding.ll` - syntax error at [line 122](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/callbr-asm-branch-folding.ll#L122) * `llvm/test/CodeGen/X86/callbr-asm-destinations.ll` - syntax error at [line 11](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/callbr-asm-destinations.ll#L11) * `llvm/test/CodeGen/X86/callbr-asm-errors.ll` - syntax error at [line 11](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/callbr-asm-errors.ll#L11) * `llvm/test/CodeGen/X86/callbr-asm-label-addr.ll` - syntax error at [line 13](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/callbr-asm-label-addr.ll#L13) * `llvm/test/CodeGen/X86/callbr-asm-obj-file.ll` - syntax error at [line 11](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/callbr-asm-obj-file.ll#L11) * `llvm/test/CodeGen/X86/callbr-asm-outputs.ll` - syntax error at [line 11](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/callbr-asm-outputs.ll#L11) * `llvm/test/CodeGen/X86/callbr-asm.ll` - syntax error at [line 26](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/callbr-asm.ll#L26) * `llvm/test/Instrumentation/MemorySanitizer/msan_asm_conservative.ll` - syntax error at [line 281](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Instrumentation/MemorySanitizer/msan_asm_conservative.ll#L281) * `llvm/test/Transforms/GVN/callbr-loadpre-critedge.ll` - syntax error at [line 29](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/GVN/callbr-loadpre-critedge.ll#L29) * `llvm/test/Transforms/GVN/callbr-scalarpre-critedge.ll` - syntax error at [line 26](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/GVN/callbr-scalarpre-critedge.ll#L26) * `llvm/test/Transforms/Inline/blockaddress.ll` - syntax error at [line 62](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/Inline/blockaddress.ll#L62) * `llvm/test/Transforms/Inline/callbr.ll` - syntax error at [line 16](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/Inline/callbr.ll#L16) * `llvm/test/Transforms/JumpThreading/callbr-edge-split.ll` - syntax error at [line 40](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/JumpThreading/callbr-edge-split.ll#L40) * `llvm/test/Transforms/LoopRotate/callbr.ll` - syntax error at [line 83](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/LoopRotate/callbr.ll#L83) * `llvm/test/Transforms/LoopUnroll/callbr.ll` - syntax error at [line 35](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/LoopUnroll/callbr.ll#L35) * `llvm/test/Transforms/LoopUnswitch/callbr.ll` - syntax error at [line 46](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/LoopUnswitch/callbr.ll#L46) * `llvm/test/Transforms/SpeculateAroundPHIs/pr42991.ll` - syntax error at [line 8](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/SpeculateAroundPHIs/pr42991.ll#L8)

Grammar related to Metadata (0.2% of previously failing test cases)

0.2% of the previously failing test cases (2/713) were related to metadata.

Fixed in rev a9beaf3778cac7e7cfb19acca7099e8e2aa925ee.

test cases failing related to Metadata grammar
* `llvm/test/Assembler/DICommonBlock.ll` - syntax error at [line 16](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Assembler/DICommonBlock.ll#L16) * `llvm/test/DebugInfo/Generic/DICommonBlock.ll` - syntax error at [line 32](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/DebugInfo/Generic/DICommonBlock.ll#L32)

Additional metadata field type support (0.2% of previously failing test cases)

0.2% of the previously failing test cases (2/713) were related to metadata field type.

test cases failing related to metadata field type
* [llvm/test/DebugInfo/Generic/containing-type-extension.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/DebugInfo/Generic/containing-type-extension.ll) - support for metadata DICompositeType vtableHolder field type *metadata.DIBasicType not yet implemented * [llvm/test/DebugInfo/X86/containing-type-extension-rust.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/DebugInfo/X86/containing-type-extension-rust.ll) - support for metadata DICompositeType vtableHolder field type *metadata.DIBasicType not yet implemented

Null pointer function type (3% of previously failing test cases)

3% of the previously failing test cases (24/713) were related to null pointer function types.

All test cases previously failed with invalid type of null pointer constant; expected *types.PointerType, got *types.FuncType.

Fixed in rev 9637c11b4d40ce359074707e80e070016e714ec4.

test cases failing related to null pointer function types
* [llvm/test/CodeGen/ARM/2008-02-29-RegAllocLocal.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/ARM/2008-02-29-RegAllocLocal.ll) * [llvm/test/CodeGen/ARM/2009-05-05-DAGCombineBug.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/ARM/2009-05-05-DAGCombineBug.ll) * [llvm/test/CodeGen/PowerPC/2007-08-04-CoalescerAssert.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/PowerPC/2007-08-04-CoalescerAssert.ll) * [llvm/test/CodeGen/PowerPC/2007-09-12-LiveIntervalsAssert.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/PowerPC/2007-09-12-LiveIntervalsAssert.ll) * [llvm/test/CodeGen/PowerPC/2007-10-21-LocalRegAllocAssert.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/PowerPC/2007-10-21-LocalRegAllocAssert.ll) * [llvm/test/CodeGen/PowerPC/2008-06-19-LegalizerCrash.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/PowerPC/2008-06-19-LegalizerCrash.ll) * [llvm/test/CodeGen/PowerPC/cr-spills.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/PowerPC/cr-spills.ll) * [llvm/test/CodeGen/X86/2007-11-04-LiveIntervalCrash.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/2007-11-04-LiveIntervalCrash.ll) * [llvm/test/CodeGen/X86/2008-04-15-LiveVariableBug.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/2008-04-15-LiveVariableBug.ll) * [llvm/test/CodeGen/X86/fold-load.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/fold-load.ll) * [llvm/test/CodeGen/X86/fold-pcmpeqd-2.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/fold-pcmpeqd-2.ll) * [llvm/test/CodeGen/X86/sibcall-3.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/sibcall-3.ll) * [llvm/test/CodeGen/X86/split-eh-lpad-edges.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/split-eh-lpad-edges.ll) * [llvm/test/Transforms/ADCE/2003-09-10-UnwindInstFail.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/ADCE/2003-09-10-UnwindInstFail.ll) * [llvm/test/Transforms/IndVarSimplify/2004-04-05-InvokeCastCrash.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/IndVarSimplify/2004-04-05-InvokeCastCrash.ll) * [llvm/test/Transforms/InstCombine/invoke.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/InstCombine/invoke.ll) * [llvm/test/Transforms/InstSimplify/call.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/InstSimplify/call.ll) * [llvm/test/Transforms/LoopRotate/crash.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/LoopRotate/crash.ll) * [llvm/test/Transforms/SimplifyCFG/invoke.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/SimplifyCFG/invoke.ll) * [llvm/test/CodeGen/AArch64/aarch64-address-type-promotion-assertion.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/AArch64/aarch64-address-type-promotion-assertion.ll) * [llvm/test/CodeGen/ARM/2012-01-23-PostRA-LICM.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/ARM/2012-01-23-PostRA-LICM.ll) * [llvm/test/CodeGen/ARM/2012-01-26-CopyPropKills.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/ARM/2012-01-26-CopyPropKills.ll) * [llvm/test/CodeGen/PowerPC/extra-toc-reg-deps.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/PowerPC/extra-toc-reg-deps.ll) * [llvm/test/Transforms/CodeExtractor/PartialInlineORECrash.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/CodeExtractor/PartialInlineORECrash.ll)

Missing enum (0.8% of previously failing test cases)

0.8% of previously failing test cases (6/713) were related to missing enums
.

test cases failing related to missing enums
* [llvm/test/CodeGen/AArch64/shadow-call-stack.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/AArch64/shadow-call-stack.ll) - unable to locate FuncAttr enum corresponding to "shadowcallstack" * [llvm/test/CodeGen/X86/nocf_check.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/nocf_check.ll) - unable to locate FuncAttr enum corresponding to "nocf_check" * [llvm/test/DebugInfo/Generic/directives-only.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/DebugInfo/Generic/directives-only.ll) - unable to locate EmissionKind enum corresponding to "DebugDirectivesOnly" * [llvm/test/DebugInfo/NVPTX/debug-empty.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/DebugInfo/NVPTX/debug-empty.ll) - unable to locate EmissionKind enum corresponding to "DebugDirectivesOnly" * [llvm/test/DebugInfo/NVPTX/debug-file-loc-only.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/DebugInfo/NVPTX/debug-file-loc-only.ll) - unable to locate EmissionKind enum corresponding to "DebugDirectivesOnly" * [llvm/test/Transforms/SimplifyCFG/opt-for-fuzzing.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/SimplifyCFG/opt-for-fuzzing.ll) - unable to locate FuncAttr enum corresponding to "optforfuzzing"

Support for negative IDs of local, global and label identifiers

Fixed with rev 3c508edc05be898274a574040d24dccaa181122f.

Support unsigned hexadecimal integer literals

Fixed by @dannypsnl in #114.

Address space not computed correctly for indirect symbols (0.4% of failing test cases)

FIxed in rev 31b91b45a51c3ccb511aafd3425822829a03b7c4.

0.4% of currently failing test cases (3/713) are related to constant expression type mismatch.

test cases failing related to constant expression type mismatch
* [llvm/test/Feature/alias2.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Feature/alias2.ll) - constant expression type mismatch; expected "i32 addrspace(2)*", got "i32*" * [llvm/test/Assembler/addrspacecast-alias.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Assembler/addrspacecast-alias.ll) - constant expression type mismatch; expected "i8 addrspace(2)* addrspace(3)*", got "i8 addrspace(2)**" * [llvm/test/Transforms/InstCombine/objsize-address-space.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/InstCombine/objsize-address-space.ll) - constant expression type mismatch of `i8* getelementptr inbounds ([60 x i8], [60 x i8]* @a_alias, i32 0, i32 0)`; expected "i8*", got "i8 addrspace(3)*"

Broken as intended (7% of previously failing test cases)

These test cases fail because they contain invalid LLVM IR. As such, reporting a syntax error is the right thing to do.

7% of previously failing test cases (55/713) are failing on purpose.

Invalid syntax or semantics

test cases failing on purpose due to invalid syntax or semantics
* `llvm/test/Assembler/invalid-immarg2.ll` - syntax error at [line 4](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Assembler/invalid-immarg2.ll#L4) * `llvm/test/Assembler/invalid-immarg3.ll` - syntax error at [line 4](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Assembler/invalid-immarg3.ll#L4) * `llvm/test/Verifier/swifterror2.ll` - syntax error at [line 4](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Verifier/swifterror2.ll#L4) * `llvm/test/Verifier/swifterror3.ll` - syntax error at [line 4](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Verifier/swifterror3.ll#L4) * `llvm/test/CodeGen/AVR/progmem-extended.ll` - syntax error at [line 4](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/AVR/progmem-extended.ll#L4) * `llvm/test/CodeGen/X86/GC/badrootproto.ll` - syntax error at [line 13](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/GC/badrootproto.ll#L13) * `llvm/test/CodeGen/X86/GC/badwriteproto.ll` - syntax error at [line 20](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/GC/badwriteproto.ll#L20) * [llvm/test/Verifier/dbg-difile-crash.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Verifier/dbg-difile-crash.ll) - support for metadata DISubprogram file field type *metadata.DICompileUnit not yet implemented * [llvm/test/Verifier/dbg-invalid-retaintypes.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Verifier/dbg-invalid-retaintypes.ll) - support for metadata DICompileUnit retainedTypes field type *metadata.DISubprogram not yet implemented * [llvm/test/Bitcode/weak-cmpxchg-upgrade.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/weak-cmpxchg-upgrade.ll) - global identifier "@test" already present; prev `define i32 @test(i32* %addr, i32 %old, i32 %new) { * [llvm/test/CodeGen/WebAssembly/inline-asm-m.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/WebAssembly/inline-asm-m.ll) - unable to locate metadata ID "!1" * [llvm/test/Transforms/NewGVN/invariant.group.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/NewGVN/invariant.group.ll) - unable to locate global identifier "@llvm.strip.invariant.group.p0i8" * [llvm/test/Verifier/2002-11-05-GetelementptrPointers.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Verifier/2002-11-05-GetelementptrPointers.ll) - unable to index into element of pointer type `{ i32, i32* }`; for more information, see http://llvm.org/docs/GetElementPtr.html#what-is-dereferenced-by-gep * [llvm/test/CodeGen/X86/nonconst-static-iv.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/nonconst-static-iv.ll) - constant expression type mismatch in `{ i8 } insertvalue ({ i8 } select (i1 ptrtoint (i32* @1 to i1), { i8 } { i8 1 }, { i8 } { i8 2 }), i8 0, 0)`; expected "{ i8 }", got "i8"

Require pre-processing

test cases failing on purpose as they require pre-processing
* `llvm/test/CodeGen/Hexagon/intrinsics/atomic_load.ll` - syntax error at [line 21](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/Hexagon/intrinsics/atomic_load.ll#L21) * `llvm/test/CodeGen/Hexagon/intrinsics/atomic_store.ll` - syntax error at [line 22](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/Hexagon/intrinsics/atomic_store.ll#L22) * `llvm/test/CodeGen/Hexagon/intrinsics/atomicrmw_addsub_native.ll` - syntax error at [line 27](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/Hexagon/intrinsics/atomicrmw_addsub_native.ll#L27) * `llvm/test/CodeGen/Hexagon/intrinsics/atomicrmw_bitwise_native.ll` - syntax error at [line 41](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/Hexagon/intrinsics/atomicrmw_bitwise_native.ll#L41) * `llvm/test/CodeGen/Hexagon/intrinsics/atomicrmw_nand.ll` - syntax error at [line 32](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/Hexagon/intrinsics/atomicrmw_nand.ll#L32) * `llvm/test/CodeGen/Hexagon/intrinsics/fence.ll` - syntax error at [line 13](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/Hexagon/intrinsics/fence.ll#L13) * `llvm/test/CodeGen/WinEH/wineh-noret-cleanup.ll` - syntax error at [line 13](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/WinEH/wineh-noret-cleanup.ll#L13) * `llvm/test/CodeGen/WinEH/wineh-statenumbering-cleanups.ll` - syntax error at [line 13](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/WinEH/wineh-statenumbering-cleanups.ll#L13) * `llvm/test/DebugInfo/AArch64/line-header.ll` - syntax error at [line 5](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/DebugInfo/AArch64/line-header.ll#L5) * `llvm/test/Linker/link-flags.ll` - syntax error at [line 7](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Linker/link-flags.ll#L7) * [llvm/test/Transforms/GCOVProfiling/function-numbering.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/GCOVProfiling/function-numbering.ll) - unable to locate metadata ID "!14" * [llvm/test/Transforms/GCOVProfiling/global-ctor.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/GCOVProfiling/global-ctor.ll) - unable to locate metadata ID "!16" * [llvm/test/Transforms/GCOVProfiling/linkagename.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/GCOVProfiling/linkagename.ll) - unable to locate metadata ID "!9" * [llvm/test/Transforms/GCOVProfiling/return-block.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/GCOVProfiling/return-block.ll) - unable to locate metadata ID "!19" * [llvm/test/Transforms/GCOVProfiling/three-element-mdnode.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/GCOVProfiling/three-element-mdnode.ll) - unable to locate metadata ID "!10" * `llvm/test/CodeGen/NVPTX/nvvm-reflect.ll` - unable to locate metadata ID "!0"

Old grammar compatibility test cases

Test cases for compatibility with older versions of LLVM IR. We only support the latest version of LLVM IR, as upgrading LLVM IR files to the latest grammar is better handled by the official tools of LLVM (e.g. ll-dis foo.bc)

test cases failing related to compatibility with an older grammar of LLVM IR
* `llvm/test/Bitcode/cmpxchg-upgrade.ll` - syntax error at [line 11](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/cmpxchg-upgrade.ll#L11) * `llvm/test/Bitcode/compatibility-3.6.ll` - syntax error at [line 177](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/compatibility-3.6.ll#L177) * `llvm/test/Bitcode/compatibility-3.7.ll` - syntax error at [line 177](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/compatibility-3.7.ll#L177) * `llvm/test/Bitcode/drop-debug-info.3.5.ll` - syntax error at [line 28](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/drop-debug-info.3.5.ll#L28) * `llvm/test/Bitcode/function-local-metadata.3.5.ll` - syntax error at [line 27](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/function-local-metadata.3.5.ll#L27) * `llvm/test/Bitcode/highLevelStructure.3.2.ll` - syntax error at [line 79](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/highLevelStructure.3.2.ll#L79) * `llvm/test/Bitcode/linkage-types-3.2.ll` - syntax error at [line 56](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/linkage-types-3.2.ll#L56) * `llvm/test/Bitcode/memInstructions.3.2.ll` - syntax error at [line 328](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/memInstructions.3.2.ll#L328) * `llvm/test/Bitcode/metadata.3.5.ll` - syntax error at [line 24](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/metadata.3.5.ll#L24) * `llvm/test/Bitcode/miscInstructions.3.2.ll` - syntax error at [line 21](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/miscInstructions.3.2.ll#L21) * `llvm/test/Bitcode/upgrade-loop-metadata.ll` - syntax error at [line 34](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/upgrade-loop-metadata.ll#L34) * `llvm/test/Bitcode/upgrade-subprogram.ll` - syntax error at [line 16](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/upgrade-subprogram.ll#L16) * `llvm/test/Verifier/2004-05-21-SwitchConstantMismatch.ll` - syntax error at [line 5](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Verifier/2004-05-21-SwitchConstantMismatch.ll#L5) * `llvm/test/Bitcode/local-linkage-default-visibility.3.4.ll` - syntax error at [line 28](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/local-linkage-default-visibility.3.4.ll#L28) * `llvm/test/Bitcode/DIExpression-4.0.ll` - syntax error at [line 8](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/DIExpression-4.0.ll#L8) * `llvm/test/Bitcode/DIGlobalVariableExpression.ll` - syntax error at [line 25](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/DIGlobalVariableExpression.ll#L25) * `llvm/test/Bitcode/diglobalvariable-3.8.ll` - syntax error at [line 14](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/diglobalvariable-3.8.ll#L14) * `llvm/test/Bitcode/dityperefs-3.8.ll` - syntax error at [line 39](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/dityperefs-3.8.ll#L39) * `llvm/test/DebugInfo/X86/no-public-sections.ll` - syntax error at [line 19](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/DebugInfo/X86/no-public-sections.ll#L19) * `llvm/test/Bitcode/DISubprogram-distinct-definitions.ll` - syntax error at [line 9](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/DISubprogram-distinct-definitions.ll#L9) * `llvm/test/Bitcode/DIGlobalVariableExpression2.ll` - syntax error at [line 22](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/DIGlobalVariableExpression2.ll#L22) * `llvm/test/Bitcode/DILocalVariable-explicit-tags.ll` - syntax error at [line 18](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/DILocalVariable-explicit-tags.ll#L18) * [llvm/test/CodeGen/ARM/2009-10-16-Scope.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/ARM/2009-10-16-Scope.ll) - support for metadata DILocalVariable file field type *metadata.DICompileUnit not yet implemented * [llvm/test/CodeGen/X86/2009-02-12-DebugInfoVLA.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/2009-02-12-DebugInfoVLA.ll) - support for metadata DILocalVariable file field type *metadata.DICompileUnit not yet implemented * [llvm/test/CodeGen/X86/2009-10-16-Scope.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/2009-10-16-Scope.ll) - support for metadata DILocalVariable file field type *metadata.DICompileUnit not yet implemented * [llvm/test/CodeGen/X86/2010-02-01-DbgValueCrash.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/2010-02-01-DbgValueCrash.ll) - support for metadata DILocalVariable file field type *metadata.DICompileUnit not yet implemented * [llvm/test/DebugInfo/Generic/inheritance.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/DebugInfo/Generic/inheritance.ll) - support for metadata DILocalVariable file field type *metadata.DICompileUnit not yet implemented * [llvm/test/Linker/debug-info-version-b.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Linker/debug-info-version-b.ll) - support for metadata DICompileUnit file field type *metadata.String not yet implemented * [llvm/test/Verifier/DILocation-parents.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Verifier/DILocation-parents.ll) - support for metadata DILexicalBlockFile file field type *metadata.Tuple not yet implemented * [llvm/test/Verifier/dbg.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Verifier/dbg.ll) - support for metadata DILocation inlinedAt field type *metadata.Tuple not yet implemented * [llvm/test/Verifier/mdcompositetype-templateparams-tuple.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Verifier/mdcompositetype-templateparams-tuple.ll) - support for metadata DICompositeType templateParams field type *metadata.DITemplateTypeParameter not yet implemented * [llvm/test/Bitcode/DISubprogram-v5.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/DISubprogram-v5.ll) - unable to locate DIFlag enum corresponding to "DIFlagMainSubprogram" * [llvm/test/Bitcode/upgrade-dbg-checksum.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/upgrade-dbg-checksum.ll) - unable to locate ChecksumKind enum corresponding to "CSK_None"

Incorrect .ll extension

test cases failing due to incorrect `.ll` extension
* `llvm/test/MC/ARM/thumb2-narrow-dp.ll` - syntax error at [line 1](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/MC/ARM/thumb2-narrow-dp.ll#L1) * `llvm/test/MC/Disassembler/AMDGPU/aperture-regs.ll` - syntax error at [line 1](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/MC/Disassembler/AMDGPU/aperture-regs.ll#L1) * `llvm/test/Other/lit-globbing.ll` - syntax error at [line 1](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Other/lit-globbing.ll#L1)
mewmew commented 4 years ago

Test cases to fix in a future release (after v0.3)

To prioritize for the v0.3 release, these test cases have been targeted for a future release of llir/llvm.

Update to grammar needed (0.9% of failing test cases)

0.9% of currently failing test cases (7/713) are related to grammar.

Grammar related to Function String Attribute

ref: #40.

test cases failing likely related to Function String Attribute grammar
* `llvm/test/Bitcode/attributes.ll` - syntax error at [line 266](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Bitcode/attributes.ll#L266) * `llvm/test/Transforms/Inline/inline-varargs.ll` - syntax error at [line 6](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Transforms/Inline/inline-varargs.ll#L6)

Grammar related to Module Summary

ref: #43.

test cases failing related to Module Summary grammar
* `llvm/test/Assembler/thinlto-summary.ll` - syntax error at [line 6](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Assembler/thinlto-summary.ll#L6) * `llvm/test/Assembler/thinlto-vtable-summary.ll` - syntax error at [line 30](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/Assembler/thinlto-vtable-summary.ll#L30) * `llvm/test/LTO/X86/Inputs/dllimport.ll` - syntax error at [line 16](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/LTO/X86/Inputs/dllimport.ll#L16) * `llvm/test/LTO/X86/dllimport.ll` - syntax error at [line 28](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/LTO/X86/dllimport.ll#L28) * `llvm/test/Assembler/asm-path-writer.ll` - syntax error at line 5

align attribute

ref: #40.

used in call instruction:

used in return attribute:

Merge function attribute definitions (0.4% of failing test cases)

Moved to dedicated issue #117.

attributes #0 = { norecurse nounwind uwtable }
attributes #0 = { norecurse nounwind readnone uwtable }

llir/llvm rejects the input and gives the error attribute group ID "#0" already present.

opt accepts the input merges the function attribute definition into:

attributes #0 = { norecurse nounwind readnone uwtable }

0.4% of currently failing test cases (3/713) are related to function attribute definitions not being merged.

test cases failing related to function attribute definitions not being merged
* [llvm/test/CodeGen/BPF/select_ri.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/BPF/select_ri.ll) - attribute group ID "#0" already present; prev `attributes #0 = { norecurse nounwind readonly }`, new `attributes #0 = { norecurse nounwind readonly }` * [llvm/test/CodeGen/X86/code-model-elf-memset.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/CodeGen/X86/code-model-elf-memset.ll) - attribute group ID "#0" already present; prev `attributes #0 = { noinline nounwind uwtable }`, new `attributes #0 = { argmemonly nounwind uwtable }` * [llvm/test/DebugInfo/unrolled-loop-remainder.ll](https://github.com/llir/testdata/blob/68fdb7c8ce371954493b215f547f5261572aadae/llvm/test/DebugInfo/unrolled-loop-remainder.ll) - attribute group ID "#0" already present; prev `attributes #0 = { norecurse nounwind uwtable }`, new `attributes #0 = { norecurse nounwind readnone uwtable }`

Assign global IDs and support unnamed globals (0.1% of failing test cases)

Moved to dedicated issue #118.

llir/llvm currently rejects llvm/test/CodeGen/X86/extractps.ll as it contains multiple function definitions with the name @"", which causes a name collision.

However, opt accepts this input and treats @"" as unnamed, and assigns global IDs to these unamed global identifiers.

Using opt -S -o a.ll llvm/test/CodeGen/X86/extractps.ll produces:

@0 = external global float

define void @1() { ; <--- given ID 1 from unnamed ""
...
}
define void @2() { ; <--- given ID 2 from unnamed ""
...
}

Extract of llvm/test/CodeGen/X86/extractps.ll:

@0 = external global float

define void @""() {
   load float, float* @0, align 16
   insertelement <4 x float> undef, float %1, i32 0
   call <4 x float> @llvm.x86.sse.rsqrt.ss( <4 x float> %2 )
   extractelement <4 x float> %3, i32 0
   store float %4, float* @0, align 16
   ret void
}
define void @""() {
   load float, float* @0, align 16
   insertelement <4 x float> undef, float %1, i32 1
   call <4 x float> @llvm.x86.sse.rsqrt.ss( <4 x float> %2 )
   extractelement <4 x float> %3, i32 1
   store float %4, float* @0, align 16
   ret void
}

declare <4 x float> @llvm.x86.sse.rsqrt.ss(<4 x float>)

Only one test case exists which has multiple unnamed global definitions, namely:

dannypsnl commented 4 years ago

For literal 1397723CCABD0000401C666660000000, u0x400000000 and 7FF80000000000000 I created a quick reproduce example:

func main() {
    if _, err := strconv.ParseUint("1397723CCABD0000401C666660000000", 16, 64); err != nil {
        fmt.Printf("%s\n", err)
    }
    if _, err := strconv.ParseUint("u0x400000000", 2, 64); err != nil {
        fmt.Printf("%s\n", err)
    }
    if _, err := strconv.ParseUint("7FF80000000000000", 16, 64); err != nil {
        fmt.Printf("%s\n", err)
    }
}
// Output:
//
// strconv.ParseUint: parsing "1397723CCABD0000401C666660000000": value out of range
// strconv.ParseUint: parsing "u0x400000000": invalid syntax
// strconv.ParseUint: parsing "7FF80000000000000": value out of range

For out of range we probably need something like https://github.com/cockroachdb/cockroach/blob/master/pkg/util/uint128/uint128.go

That's the easy part, but for u0x400000000 I can't find document mentions how LLVM integer literal express. Seems like I have to take a look at IR Tokenizer.

mewmew commented 4 years ago

@dannypsnl, thanks for taking a look at this! Here are all the "value out of range" errors I get when parsing all test cases:

mewmew commented 4 years ago

That's the easy part, but for u0x400000000 I can't find document mentions how LLVM integer literal express. Seems like I have to take a look at IR Tokenizer.

Signed and unsigned hexadecimal integers. Here's another example (from llvm/test/Integer/constexpr_bt.ll):

mewmew commented 4 years ago

For literal 1397723CCABD0000401C666660000000, u0x400000000 and 7FF80000000000000 I created a quick reproduce example:

I took a quick look at the test case containing 1397723CCABD0000401C666660000000 (llvm/test/CodeGen/PowerPC/fast-isel-call.ll) and it turns out its a floating-point constant in hexadecimal notation. I'll make sure to update the error message so it doesn't strip the 0x prefix.

As it's a floating-point constant and not an integer constant, this test case would be fixed by #31.

From llvm/test/CodeGen/PowerPC/fast-isel-call.ll:

call void @double_foo(double 0x1397723CCABD0000401C666660000000)

Edit: Oh, and it turns out 7FF80000000000000 was also a floating-point constant in hex representation. So both are part of #47

From llvm/test/Transforms/InstCombine/fneg.ll:

%m = fmul <4 x double> %x, <double 42.0, double 0x7FF80000000000000, double 0x7FF0000000000000, double undef>
mewmew commented 4 years ago

All test cases are now resolved, except for those related to hexadecimal floating-pointer literals. There is a dedicated issue to track progress on adding complete support for hexadecimal floating-point literals, namely #31.

A few test cases are related to module summaries, and a few other aspects of LLVM IR that have been marked for a future release. The dedicated tracking issues are #40, #43, #117 and #118.

As such, we consider this meta issue complete and will it close it for now. Follow-up for future discussions are referred to the issue threads of each dedicated issue.