codegen-norecursion has merged

[MasonProtter]

https://github.com/JuliaLang/julia/pull/25984 has merged into master so the note in the readme about it should probably be updated, especially because the link is now dead.

Does anything here need to change to support recursion or should StaticCompiler now support recursion on the julia master branch?

[tshort]

StaticCompiler needs to change. It's a welcome change, because we'll (hopefully) be able to compile more code.

[tshort]

For reference, see the comparable changes needed in CUDAnative: https://github.com/JuliaGPU/CUDAnative.jl/pull/162

[tshort]

I've started a branch to adapt to Julia's codegen-restructuring changes:

• https://github.com/tshort/StaticCompiler.jl/tree/recursion

With the changes, StaticCompiler has several issues I'm stuck on. These are:

• Code generation error: The global-variable handling causes errors. _deserialize_globals often has Julia constructs like:

  function _deserialize_globals(Vptr, global1)
    unsafe_store!(global1, unsafe_string(Vptr + 0, 5))
    return
  end

  That is compiled to code that includes the following statements:

  %20 = addrspacecast %jl_value_t addrspace(10)* %18 to %jl_value_t addrspace(11)*, !dbg !31
  %21 = addrspacecast %jl_value_t addrspace(11)* %20 to %jl_value_t*

  That is optimized to:

  %17 = addrspacecast %jl_value_t addrspace(10)* %15 to %jl_value_t*

  That causes an error related to an illegal address space cast. It fails at this check. When that check is bypassed in Julia and recompiled, tests/globals.jl and tests/ccalls.jl pass.

  Something needs to be done here to generate different code.

• LLVM.verify failure: The following test file fails at the verify stage:

  using StaticCompiler, LLVM
  const s = "hello"
  fs() = s
  m = irgen(fs, Tuple{}, optimize_llvm=false);
  LLVM.verify(m)

  Here's the error:

  julia> verify(m)
  ERROR: LLVM error: Global variable initializer type does not match global variable type!
  %jl_value_t** @jl.global
  
  Stacktrace:
  [1] verify(::LLVM.Module) at /home/tshort/.julia/packages/LLVM/pINgj/src/analysis.jl:11

• test/others.jl segfault:

  signal (11): Segmentation fault
  in expression starting at /home/tshort/dev/StaticCompiler/test/others.jl:7
  _ZNK4llvm13AttributeList14getNumAttrSetsEv at /home/tshort/julia/usr/bin/../lib/libLLVM-9jl.so (unknown line)
  _ZNK4llvm13AttributeList13getAttributesEj at /home/tshort/julia/usr/bin/../lib/libLLVM-9jl.so (unknown line)
  _ZNK4llvm13AttributeList12hasAttributeEjNS_9Attribute8AttrKindE at /home/tshort/julia/usr/bin/../lib/libLLVM-9jl.so (unknown line)
  _ZNK4llvm8CallBase25hasFnAttrOnCalledFunctionENS_9Attribute8AttrKindE at /home/tshort/julia/usr/bin/../lib/libLLVM-9jl.so (unknown line)
  _ZN4llvm23removeUnreachableBlocksERNS_8FunctionEPNS_13LazyValueInfoEPNS_14DomTreeUpdaterEPNS_16MemorySSAUpdaterE at /home/tshort/julia/usr/bin/../lib/libLLVM-9jl.so (unknown line)
  _ZL19simplifyFunctionCFGRN4llvm8FunctionERKNS_19TargetTransformInfoERKNS_18SimplifyCFGOptionsE at /home/tshort/julia/usr/bin/../lib/libLLVM-9jl.so (unknown line)
  _ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at /home/tshort/julia/usr/bin/../lib/libLLVM-9jl.so (unknown line)
  _ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at /home/tshort/julia/usr/bin/../lib/libLLVM-9jl.so (unknown line)
  _ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /home/tshort/julia/usr/bin/../lib/libLLVM-9jl.so (unknown line)
  LLVMRunPassManager at /home/tshort/julia/usr/bin/../lib/libLLVM-9jl.so (unknown line)
  macro expansion at /home/tshort/.julia/packages/LLVM/pINgj/src/base.jl:18 [inlined]
  LLVMRunPassManager at /home/tshort/.julia/packages/LLVM/pINgj/lib/9.0/libLLVM_h.jl:2813 [inlined]
  run! at /home/tshort/.julia/packages/LLVM/pINgj/src/passmanager.jl:34 [inlined]
  #54 at /home/tshort/dev/StaticCompiler/src/irgen.jl:143 [inlined]
  ModulePassManager at /home/tshort/.julia/packages/LLVM/pINgj/src/passmanager.jl:28
  optimize! at /home/tshort/dev/StaticCompiler/src/irgen.jl:137
  #irgen#45 at /home/tshort/dev/StaticCompiler/src/irgen.jl:95
  unknown function (ip: 0x7f8dacb5c32b)
  irgen at /home/tshort/dev/StaticCompiler/src/irgen.jl:49 [inlined]
  

• Standalone initialization: To initialize Julia, a minimal blank.ji file is used. With the code generation updates, the following no longer works (blank.jl is an empty file):

  julia --output-ji blank.ji --startup-file=no -g0 -O0 blank.jl

  Initialization will have to be handled another way, or some other method will be needed to create a blank.ji file.

[tshort]

It's a depressing amount of issues.

[aminya]

Does LLVM.jl need to change to accomodate the latest update? I think some of these are related to LLVM.jl.

[tshort]

Not that I'm aware of. CUDAnative changes have merged.

[tshort]

I figured out one of the issues. To create a blank.ji file, the following works:

ccall(:jl_save_incremental, Cint, (Cstring, Array), "blank.ji", [])

It's a small thing, but it's nice to finally push the ball forward a little bit.

[tshort]

Spoke too soon. It creates a blank.ji, but jl_init_with_image fails to finish with that file, so it must need more content than that.