Closed sjsprecious closed 1 year ago
What is your need date?
I ask because our current plan is to update the operating system on Casper in the early 2023 timeframe, and include a totally fresh user software stack based off of Spack. This would be much easier to include in that update, we estimate around March 2023 for user access.
Thanks @benkirk. I plan to test the LLVM module before the beginning of NCAR/NOAA Open Hackathon, which will be the end of February 2023. But if you think it is easier to include it during the update of Casper in March 2023, that is fine and I can try to find a way to work around.
What features & dependent libraries might you need? Is it simply the compiler with CUDA support?
I'll play with this and see if we can meet your need date with just the compiler, not fully integrated with the existing Casper module tree.
Thanks @benkirk for your information. I would let @mattldawson answer your questions since he is more familiar with the LLVM requirement for the JIT compiling.
hi @benkirk - we've been using just the core LLVM project to explore JIT-compiling some of the model code (CPU only). We don't need clang or any of the other additional projects (but I guess it wouldn't hurt to have them, and I'm not sure what Jian might need for GPU stuff once we get to that point.) We're using these libraries from the LLVM project: support core orcjit native irreader
I've just been testing things locally, but yesterday I built the LLVM v15.0.6 project on Casper here: /glade/scratch/mattdawson/llvm-15
from source here: /glade/scratch/mattdawson/llvm-project-15.0.6.src
, to try out our tests on Casper, if this is useful. (We might be able to just use this for the Hack-a-thon if that works better with the Casper upgrade plans.)
I will try to figure out with Jian if there is anything we might need in addition for the Hack-a-thon
@mattldawson: Wow, thanks for your update and I did not realize that you were trying to build LLVM on Casper already. I could try to use your LLVM path and run your JIT example on Casper to examine whether it works on my end. If yes, probably it is enough for the Hackathon but we could always add more libraries later if they are required for GPU.
@mattldawson & @sjsprecious
Thanks, looks like you are set for a little bit anyway!
I've installed llvm@15.0.4 with CUDA support on Casper as well:
/glade/work/benkirk/my_spack_playground/deploy/view/CentOS-compilers/llvm/15.0.4
I've not yet kicked the tires much with the CUDA support, other than getting it to compile with a consistent cuda
and cuda_arch
.
==> Installed packages
-- linux-centos7-x86_64 / gcc@11.3.0 ----------------------------
zk5vnam llvm@15.0.4+clang+compiler-rt+cuda+flang+gold+internal_unwind~ipo+libcxx~link_llvm_dylib+lld+lldb+llvm_dylib~mlir+omp_as_runtime~omp_debug~omp_tsan+polly~python~split_dwarf~z3 build_system=cmake build_type=Release cuda_arch=80 patches=3dc9bc1 shlib_symbol_version=none targets=none version_suffix=none
vzv5l7r binutils@2.38~gas+gold~headers~interwork+ld~libiberty~lto+nls+plugins build_system=autotools libs=shared,static
zl7eejh diffutils@3.8 build_system=autotools
7i6nsqc libiconv@1.16 build_system=autotools libs=shared,static
o5yhcaq gettext@0.21.1+bzip2+curses+git~libunistring+libxml2+tar+xz build_system=autotools
2yjvplf tar@1.34 build_system=autotools zip=pigz
ua2dk5n pigz@2.7 build_system=makefile
ougyety zstd@1.5.2+programs build_system=makefile compression=none libs=shared,static
qiv4z2x cmake@3.24.3~doc+ncurses+ownlibs~qt build_system=generic build_type=Release
gxlv7fe openssl@1.1.1s~docs~shared build_system=generic certs=mozilla
kw6oooa ca-certificates-mozilla@2022-10-11 build_system=generic
jqtrzzw cuda@11.8.0~allow-unsupported-compilers~dev build_system=generic
3qzwztv libxml2@2.10.3~python build_system=autotools
6ip3hzq hwloc@2.8.0~cairo~cuda~gl~libudev+libxml2~netloc~nvml~oneapi-level-zero~opencl+pci~rocm build_system=autotools libs=shared,static
fwljxhm libpciaccess@0.16 build_system=autotools
qkwvmfy libtool@2.4.7 build_system=autotools
nsvf4s2 m4@1.4.19+sigsegv build_system=autotools patches=9dc5fbd,bfdffa7
llzrwrn libsigsegv@2.13 build_system=autotools
bf3yj4s util-macros@1.19.3 build_system=autotools
5kvx74h libedit@3.1-20210216 build_system=autotools
efdk47z libelf@0.8.13 build_system=autotools
s7ljr3d libffi@3.4.2 build_system=autotools
3a5alko ncurses@6.3~symlinks+termlib abi=none build_system=autotools
aqb4ow7 ninja@1.11.1 build_system=generic
4kpdbiy perl-data-dumper@2.173 build_system=perl
6qo6hp5 perl@5.36.0+cpanm+shared+threads build_system=generic
abgzjy5 berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches=26090f4,b231fcc
rzt66kl pkgconf@1.8.0 build_system=autotools
qz37yfl python@3.10.8+bz2+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4+uuid+zlib build_system=generic patches=0d98e93,7d40923,f2fd060
2auw3wr bzip2@1.0.8~debug~pic+shared build_system=generic
cvodpnn expat@2.4.8+libbsd build_system=autotools
ovf6sww libbsd@0.11.5 build_system=autotools
xtaemos libmd@1.0.4 build_system=autotools
kbqlols gdbm@1.23 build_system=autotools
lbxcvam readline@8.1.2 build_system=autotools
6ixd27k sqlite@3.39.4+column_metadata+dynamic_extensions+fts~functions+rtree build_system=autotools
sujcpgq util-linux-uuid@2.38.1 build_system=autotools
dn2w46o xz@5.2.7~pic build_system=autotools libs=shared,static
apr7omr zlib@1.2.13+optimize+pic+shared build_system=makefile
Thanks @benkirk. I could confirm that I am able to use Matt's LLVM build on Casper to run a test successfully. I could try your build for the Hackathon since we will need the GPU support and let you know if I run into any issue later. Thank you for your help!
@sjsprecious - are you okay with this being closed, since it sounds like you have a solution and this isn't relevant to Gust at present anyway?
Yes, Brian. It could be closed as complete. Thanks Ben's help on this issue.
Sorry if I post this request at the wrong place but I would like the latest LLVM library to be installed on Casper so that we could explore some just-in-time compiling features later.