search

trailofbits / vast

VAST is an experimental compiler pipeline designed for program analysis of C and C++. It provides a tower of IRs as MLIR dialects to choose the best fit representations for a program analysis or further program abstraction.
https://trailofbits.github.io/vast/
Apache License 2.0
378 stars 23 forks source link

[Bug]: Assert due to invalid cast #579

Open kumarak opened 3 months ago

kumarak commented 3 months ago

VAST version

master

LLVM version

18.1.4

Operating system

MacOS

Description

Asserts due to invalid cast while lowering curl codebase.

* thread #109, stop reason = hit program assert
    frame #0: 0x0000000193da6a60 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x0000000193ddec20 libsystem_pthread.dylib`pthread_kill + 288
    frame #2: 0x0000000193ceba20 libsystem_c.dylib`abort + 180
    frame #3: 0x0000000193cead10 libsystem_c.dylib`__assert_rtn + 284
  * frame #4: 0x000000010821cf70 mx-index`decltype(auto) llvm::cast<vast::hl::LValueType, mlir::Type>(Val=0x00000003933870a8) at Casting.h:566:3
    frame #5: 0x000000010815a0f0 mx-index`vast::hl::LValueType mlir::Type::cast<vast::hl::LValueType>(this=0x00000003933870a8) const at Types.h:340:10
    frame #6: 0x000000010816d7c4 mx-index`vast::hl::AssignOp::build(odsBuilder=0x00000003933896a8, odsState=0x00000003933871a8, dst=vast::Value @ 0x00000003933870d0, src=vast::Value @ 0x00000003933870c8) at HighLevel.cpp.inc:2927:35
    frame #7: 0x0000000102ee2e44 mx-index`vast::hl::AssignOp mlir::OpBuilder::create<vast::hl::AssignOp, mlir::OpResult&, mlir::OpResult&>(this=0x00000003933896a8, location=Location @ 0x0000000393387198, args=0x00000003933873b8, args=0x00000003933873b0) at Builders.h:494:5
    frame #8: 0x0000000102ee2d8c mx-index`auto vast::cg::builder_t<vast::cg::visitor_instance<indexer::(anonymous namespace)::MXVisitorConfig>>::create<vast::hl::AssignOp, mlir::Location, mlir::OpResult&, mlir::OpResult&>(this=0x0000000393389698, args=0x00000003933873a0, args=0x00000003933873b8, args=0x00000003933873b0) at CodeGenBuilder.hpp:163:44
    frame #9: 0x0000000102ee2d38 mx-index`auto vast::cg::default_stmt_visitor<vast::cg::visitor_instance<indexer::(anonymous namespace)::MXVisitorConfig>>::make<vast::hl::AssignOp, mlir::Location, mlir::OpResult&, mlir::OpResult&>(this=0x0000000393389698, args=0x00000003933873a0, args=0x00000003933873b8, args=0x00000003933873b0) at CodeGenStmtVisitor.hpp:66:39
    frame #10: 0x0000000102ee2ce8 mx-index`mlir::Operation* vast::cg::default_stmt_visitor<vast::cg::visitor_instance<indexer::(anonymous namespace)::MXVisitorConfig>>::VisitAssignBinOp<vast::hl::AssignOp>(this=0x0000000393389698, op=0x0000000169366220) at CodeGenStmtVisitor.hpp:244:20
    frame #11: 0x0000000102ecfc3c mx-index`vast::cg::default_stmt_visitor<vast::cg::visitor_instance<indexer::(anonymous namespace)::MXVisitorConfig>>::VisitBinAssign(this=0x0000000393389698, op=0x0000000169366220) at CodeGenStmtVisitor.hpp:283:20
    frame #12: 0x0000000102ecdc78 mx-index`clang::StmtVisitorBase<llvm::make_const_ptr, vast::cg::default_stmt_visitor<vast::cg::visitor_instance<indexer::(anonymous namespace)::MXVisitorConfig>>, mlir::Operation*>::Visit(this=0x0000000393389698, S=0x0000000169366220) at StmtVisitor.h:72:26
    frame #13: 0x0000000102ecd9f0 mx-index`auto vast::cg::fallback_visitor<vast::cg::visitor_instance<indexer::(anonymous namespace)::MXVisitorConfig>, indexer::(anonymous namespace)::MXDefaultCodeGenVisitor, vast::cg::unsup_visitor, vast::cg::unreach_visitor>::visit_with_fallback<clang::Stmt const*>(this=0x0000000393389698, token=0x0000000169366220) at FallBackVisitor.hpp:31:47
    frame #14: 0x0000000102ecd9c0 mx-index`vast::cg::fallback_visitor<vast::cg::visitor_instance<indexer::(anonymous namespace)::MXVisitorConfig>, indexer::(anonymous namespace)::MXDefaultCodeGenVisitor, vast::cg::unsup_visitor, vast::cg::unreach_visitor>::Visit(this=0x0000000393389698, stmt=0x0000000169366220) at FallBackVisitor.hpp:19:59
    frame #15: 0x0000000102f12868 mx-index`decltype(auto) vast::cg::visitor_lens<vast::cg::visitor_instance<indexer::(anonymous namespace)::MXVisitorConfig>, vast::cg::default_stmt_visitor>::visit<clang::Stmt*>(this=0x0000000393389698, token=0x0000000169366220) at CodeGenVisitorLens.hpp:106:62
    frame #16: 0x0000000102f12824 mx-index`vast::cg::default_stmt_visitor<vast::cg::visitor_instance<indexer::(anonymous namespace)::MXVisitorConfig>>::VisitCompoundStmt(this=0x0000000393387580)::'lambda'()::operator()() const at CodeGenStmtVisitor.hpp:72:21
    frame #17: 0x0000000102f12718 mx-index`auto vast::cg::builder_t<vast::cg::visitor_instance<indexer::(anonymous namespace)::MXVisitorConfig>>::make_scoped<vast::cg::scope_generator<vast::core::ScopeOp>, vast::cg::default_stmt_visitor<vast::cg::visitor_instance<indexer::(anonymous namespace)::MXVisitorConfig>>::VisitCompoundStmt(clang::CompoundStmt const*)::'lambda'()>(this=0x0000000393389698, loc=vast::loc_t @ 0x0000000393387590, content_builder=(unnamed class) @ 0x0000000393387580) at CodeGenBuilder.hpp:188:13
    frame #18: 0x0000000102ed4020 mx-index`vast::cg::default_stmt_visitor<vast::cg::visitor_instance<indexer::(anonymous namespace)::MXVisitorConfig>>::VisitCompoundStmt(this=0x0000000393389698, stmt=0x0000000169366500) at CodeGenStmtVisitor.hpp:70:39
    frame #19: 0x0000000102ecf0dc mx-index`clang::StmtVisitorBase<llvm::make_const_ptr, vast::cg::default_stmt_visitor<vast::cg::visitor_instance<indexer::(anonymous namespace)::MXVisitorConfig>>, mlir::Operation*>::Visit(this=0x0000000393389698, S=0x0000000169366500) at StmtNodes.inc:1498:1
    frame #20: 0x0000000102ecd9f0 mx-index`auto vast::cg::fallback_visitor<vast::cg::visitor_instance<indexer::(anonymous namespace)::MXVisitorConfig>, indexer::(anonymous namespace)::MXDefaultCodeGenVisitor, vast::cg::unsup_visitor, vast::cg::unreach_visitor>::visit_with_fallback<clang::Stmt const*>(this=0x0000000393389698, token=0x0000000169366500) at FallBackVisitor.hpp:31:47
    frame #21: 0x0000000102ecd9c0 mx-index`vast::cg::fallback_visitor<vast::cg::visitor_instance<indexer::(anonymous namespace)::MXVisitorConfig>, indexer::(anonymous namespace)::MXDefaultCodeGenVisitor, vast::cg::unsup_visitor, vast::cg::unreach_visitor>::Visit(this=0x0000000393389698, stmt=0x0000000169366500) at FallBackVisitor.hpp:19:59
    frame #22: 0x0000000102ee1b00 mx-index`decltype(auto) vast::cg::visitor_lens<vast::cg::visitor_instance<indexer::(anonymous namespace)::MXVisitorConfig>, vast::cg::builder_t>::visit<clang::Stmt const*>(this=0x0000000393389698, token=0x0000000169366500) at CodeGenVisitorLens.hpp:106:62
    frame #23: 0x0000000102ef30d0 mx-index`auto vast::cg::builder_t<vast::cg::visitor_instance<indexer::(anonymous namespace)::MXVisitorConfig>>::make_region_builder(this=0x0000000393387f18, bld=0x00000003933896a8, (null)=Location @ 0x0000000393387708)::'lambda'(auto&, auto)::operator()<mlir::OpBuilder, mlir::Location>(auto&, auto) const at CodeGenBuilder.hpp:297:27

Debugger output that might help

f 6
(lldb) p dst.dump()
%0 = unsup.stmt "DeclRefExpr" : !hl.typedef<"curl_malloc_callback"> 
(lldb) p dst.getType().dump()
!hl.typedef<"curl_malloc_callback">
(lldb)

Steps to Reproduce

Lowering curl codebase

lkorenc commented 3 months ago

If this is an important issue please provide a minimal example we can reproduce on, it will speed up getting it fixed.

pgoodman commented 3 months ago

I suspect this unit test would repro the issue: https://github.com/trailofbits/vast/blob/6668ec5b7ea8f2e054cbf305be23d7f8137c8aba/test/vast/Transform/HL/LowerTypes/fn-ptr-a.c#L4

The difference is that we're not using the ASTConsumer interface, and instead we're doing codegen by visiting a TranslationUnitDecl.

pgoodman commented 3 months ago

@lkorenc this is an important issue. What would help catch these types of issues sooner is if VAST would add a testing harness/mechanism that doesn't use the ASTConsumer interface, and instead operations on a clang::TranslationUnitDecl, / clang::ASTContext.

kumarak commented 3 months ago

@lkorenc, It is related to https://github.com/trailofbits/vast/issues/450 and I can reproduce with the file attached. Attached another source file that can help you reproduce the issue.

global_init.gz