exaloop / codon

A high-performance, zero-overhead, extensible Python compiler using LLVM
https://docs.exaloop.io/codon
Other
14.94k stars 510 forks source link

How to get AST with pretty dump before simplify #574

Closed CanftIn closed 1 day ago

CanftIn commented 1 month ago

When i debug for frontend infra of codon, i found there is no existing api to dump the raw AST like that SimplifyVisitor can dump a S-expr string. The api like this code comment, or is a better method to dump to a "_dump_ast.sexp"? So there will be intuitively to diff raw ast with simplify ast.

ast::StmtPtr codeStmt = isCode
                            ? ast::parseCode(cache.get(), abspath, code, startLine)
                            : ast::parseFile(cache.get(), abspath);

///
// api like: codeStmt->dump();
// or:       ast::SomeVisitor::apply(std::move(codeStmt));
///

cache->module0 = file;

Timer t2("simplify");
t2.logged = true;
auto transformed =
    ast::SimplifyVisitor::apply(cache.get(), std::move(codeStmt), abspath, defines,
                                getEarlyDefines(), (testFlags > 1));
LOG_TIME("[T] parse = {:.1f}", totalPeg);
LOG_TIME("[T] simplify = {:.1f}", t2.elapsed() - totalPeg);

if (codon::getLogger().flags & codon::Logger::FLAG_USER) {
  auto fo = fopen("_dump_simplify.sexp", "w");
  fmt::print(fo, "{}\n", transformed->toString(0));
  fclose(fo);
}

And i noticed if i dump a test python file to codon simplify sexp or cir, it would load all sexpr/cir produced by stdlib, and it would cause to produce a large ir file with more information that does not need attention for the moment. Is there a way to avoid this? thank you!

inumanag commented 1 day ago

Hi @CanftIn,

Right now, there is no user-facing AST API (and the whole simplify stage will be removed in the next version). You can print any AST in codon via LOG("{}", node->toString(0));.

You can also run codon with CODON_DEBUG=lt that will produce dump files that you can use for diffing. It's honestly a bit ugly at the moment, but handles well diffing and other simple operations.