compilation/build now works

[kostrzewa]

@Marcogarofalo I've now included https://github.com/matejak/argbash/releases/tag/2.10.0 in this repository. Running build in the compilation directory now correctly generates bootstrap-chroma from the jinja template.

[kostrzewa]

Note that for smearing you do not need QPhiX, so you might want to simply disable it when compiling on Marconi-SKL for smearing.

[Marcogarofalo]

thank you

[kostrzewa]

if you hit some roadblocks, please let me know

[Marcogarofalo]

I got an error when installing QPhiX and I didn't manage to disable it. If I comment all the lines with QPhiX in the bootstrap script I still get

/marconi/home/userexternal/mgarofal/chroma/sources/chroma/lib/actions/ferm/invert/qphix/syssolver_linop_clover_qphix_w.h(30): catastrophic error: cannot open source file "qphix/geometry.h"
  #include "qphix/geometry.h"

[kostrzewa]

It seems to me that you're still compiling with QPhiX as a possible solver library for chroma. The relevant lines to disable are everything to do with QPhiX in the final section bootstrap-chroma where chroma itself is configured:

1315 mkdir -p "$build/$repo"
1316 pushd "$build/$repo"
1317 if ! [[ -f Makefile ]]; then
1318   $sourcedir/$repo/configure $base_configure \                                                                                        
1319     --enable-openmp \
1320     --enable-parallel-arch=parscalar \
1321     --enable-parallel-io \
1322     --enable-precision=$_arg_precision \
1323     --enable-qdp-alignment=128 \
1324     --enable-sse2 \
1325     --with-gmp="$gmp" \
1326     --with-libxml2="$libxml" \
1327     --with-qdp="$prefix" \
1328     --with-qphix-solver="$prefix" \
1329     --enable-qphix-solver-compress12 \
1330     --enable-qphix-solver-arch=$isa \
1331     --enable-qphix-solver-soalen=$soalen \
1332     --enable-qphix-solver-inner-soalen=$inner_soalen \
1333     --enable-qphix-solver-inner-type=$_arg_precision_inner \
1334     CFLAGS="$cflags" CXXFLAGS="$cxxflags"
1335 fi
1336 make-make-install
1337 popd

[Marcogarofalo]

I removed

1328     --with-qphix-solver="$prefix" \
1329     --enable-qphix-solver-compress12 \
1330     --enable-qphix-solver-arch=$isa \
1331     --enable-qphix-solver-soalen=$soalen \
1332     --enable-qphix-solver-inner-soalen=$inner_soalen \
1333     --enable-qphix-solver-inner-type=$_arg_precision_inner \

but still

/marconi/home/userexternal/mgarofal/chroma/sources/chroma/lib/actions/ferm/invert/qphix/syssolver_linop_clover_qphix_w.h(30): catastrophic error: cannot open source file "qphix/geometry.h"
  #include "qphix/geometry.h"

[kostrzewa]

Can you point me to your build directory?

[kostrzewa]

oh, it's right there :)

[kostrzewa]

Your chroma build /marconi/home/userexternal/mgarofal/chroma/build-icc/chroma is still configured to use qphix.

[kostrzewa]

What's the difference between your chroma-tools and chroma-auxiliary-scripts directories? Where did the former come from?

[Marcogarofalo]

chroma-tools is were I was working before you commit the argbash. It was my way of adding argbash. Please look at chroma-auxiliary-scripts

[kostrzewa]

Did you delete the chroma directory after making the changes? Keep in mind the if ! [[ -f Makefile ]]; then

[Marcogarofalo]

originally no, but now I delete the directory /marconi/home/userexternal/mgarofal/chroma/sources/chroma but still the same error

[kostrzewa]

Not the sources, you need to delete the build directory in build-icc or wherever you're building.

[kostrzewa]

Yes, build-icc. local-icc is the installation directory.

[kostrzewa]

When the Makefile has already been generated, the build driver script (bootstrap-chroma) skips the configure step as it assumes that it has already happened.

[Marcogarofalo]

It seems that it compile now but when I try to run I ger

srun  /marconi/home/userexternal/mgarofal/chroma/local-icc/bin/chroma -i "0454.stout6.chroma-ini.xml" -by 1 -bz 1 -c 12 -sy 1 -sz 1 -pxy 1 -pxyz 0 -minct 2
srun: error: r161c01s01: task 0: Illegal instruction
srun: Terminating job step 8029610.3

I am in /marconi_scratch/userexternal/mgarofal/smearing_chroma

[kostrzewa]

You've compiled with CXXFLAGS=-xMIC-AVX512, which is only supported on KNC/KNL.

[kostrzewa]

I see that you've added a marconi-a3 case statement. For simplicity, just swtich down to isa=avx2. It won't make much of a difference here anyway.

[kostrzewa]

When this was written back in 2017/2018, general purpose avx512 did not exist yet, hence the confusing codename of avx512 for the special-purpose MIC-AVX512.

[Marcogarofalo]

ok I had to delete all build-icc and recompile but now seems to work.

Thanks

[kostrzewa]

Awesome, thanks!