search

JuliaIPU / IPUToolkit.jl

IPU programming in Julia
https://juliaipu.github.io/IPUToolkit.jl/
MIT License
30 stars 3 forks source link

[CI] Test with Poplar 3.3 #45

Open giordano opened 7 months ago

giordano commented 7 months ago

Clang.jl is currently not compatible with Julia v1.11, so we can't test this at the moment. Fixing #18 would let us reduce the number of dependencies, including Clang.jl itself (it's only needed to generate the wrappers, not at runtime).

giordano commented 6 months ago

With #62 we now have a version of Clang.jl compatible with Julia v1.11, but building the bindings is failing because the Poplar header files can't be found. The problem is that instead of explicitly setting the environment variables like CPATH and LD_LIBRARY_PATH, in the graphcore/poplar:3.3.0 container they now set BASH_ENV to run a script which sets those variables, compare graphcore/poplar:3.2.0-ubuntu-20.04-20230314 and graphcore/poplar:3.3.0-ubuntu-20.04-20230703. I'm trying to figure out how to make CPATH and LD_LIBRARY_PATH persist throughout the job.

For what is worth, building of the package works locally for me inside

docker run -ti --rm graphcore/poplar:3.3.0-ubuntu-20.04-20230703 bash

so there's hope this will eventually work, but the problem is that here we aren't going through the exact same path as we aren't using the bash entrypoint.

giordano commented 6 months ago

I believe there's a bug in Clang.jl v0.18.0 with llvm 16, because compilation of the automatically generated wrappers fails here with julia v1.11 with errors like

In file included from /__w/IPUToolkit.jl/IPUToolkit.jl/deps/wrapper/template.cpp:116:
/__w/IPUToolkit.jl/IPUToolkit.jl/deps/wrapper/gen_inline.cpp: In function ‘void define_julia_module(jlcxx::Module&)’:
/__w/IPUToolkit.jl/IPUToolkit.jl/deps/wrapper/gen_inline.cpp:329:41: error: ISO C++ forbids declaration of ‘type name’ with no type [-fpermissive]
  329 | JLOptionFlagsiterator.constructor<const iterator &>();
      |                                         ^~~~~~~~
/__w/IPUToolkit.jl/IPUToolkit.jl/deps/wrapper/gen_inline.cpp:329:23: error: parse error in template argument list
  329 | JLOptionFlagsiterator.constructor<const iterator &>();
      |                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/__w/IPUToolkit.jl/IPUToolkit.jl/deps/wrapper/gen_inline.cpp:329:53: error: no matching function for call to ‘jlcxx::TypeWrapper<poplar::OptionFlags::iterator>::constructor<<expression error> >()’
  329 | JLOptionFlagsiterator.constructor<const iterator &>();
      |                                                     ^

while it works fine in #48 with the same version of Clang.jl but a different version of Julia/llvm (there we are using julia v1.7/llvm 12). I'll need to compare the generate wrappers with the two different versions of Julia/llvm. Edit: issue opened at https://github.com/JuliaInterop/Clang.jl/issues/482.

Gnimuc commented 2 months ago

The tricky part is that we need to upgrade both polar (3.2->3.3) and the wrapper generator script, probably CxxWrap as well.

giordano commented 2 months ago

Yes, each upgrade of this package is tricky 🥲 CxxWrap was upgraded in #66, I probably need to rebase this branch on main