search

mratsim / weave

A state-of-the-art multithreading runtime: message-passing based, fast, scalable, ultra-low overhead
Other
532 stars 22 forks source link

Recent Nim makes Weave crash #171

Open HJarausch opened 3 years ago

HJarausch commented 3 years ago

This is with Nim devel from today

Nim Compiler Version 1.5.1 [Linux: amd64]
Compiled at 2021-03-14
Copyright (c) 2006-2021 by Andreas Rumpf

git hash: 7937abab4439ef2cd317d4ef70551327f55711e1
active boot switches: -d:release

nim --threads:on c -r Max_Rand_atomic.nim           
Hint: used config file '/etc/nim/nim.cfg' [Conf]
Hint: used config file '/etc/nim/config.nims' [Conf]
Hint: used config file '/home/jarausch/.config/nim/nim.cfg' [Conf]
.......................................
/home/jarausch/.nimble/pkgs/weave-#master/weave/cross_thread_com/scoped_barriers.nim(66, 12) Warning: Moving a shared resource (an atomic type). [User]
..................................fatal.nim(53)            sysFatal
Error: unhandled exception: 'sym' is not accessible using discriminant 'kind' of type 'TNode' [FieldDefect]
HJarausch commented 3 years ago

It seems the other way round. Weave uncovers a bug in the recent Nim compiler. I've tried a bit to isolate the problem. I've found that state_machine/decline_thief.nim triggers the error and only in the very last to lines

synthesize(declineReqFSA):
  proc decline*(req: sink StealRequest) {.gcsafe, raises: [].}

I hope this helps a bit. Unfortunately, I haven't understood synthesis yet, so it's hard for me to isolate the code which provokes the compiler error further.

ringabout commented 3 years ago

I think this issue should be fixed by https://github.com/nim-lang/Nim/pull/17657

It would be great if someone can think of a reproducible issue without weave. Anyway weave should be added to important packages. https://github.com/nim-lang/Nim/pull/17665

HJarausch commented 3 years ago

Thanks!

ringabout commented 3 years ago

@mratsim

BTW CI failed(https://github.com/nim-lang/Nim/pull/17665) https://github.com/nim-lang/Nim/pull/17665/checks?check_run_id=2286001568 

  /home/runner/work/Nim/Nim/pkgstemp/weave/weave/parallel_for.nim(417, 19) template/generic instantiation of `parallelFor` from here
  /home/runner/work/Nim/Nim/pkgstemp/weave/weave/parallel_for.nim(172, 16) Error: redefinition of 'CapturedTy'; previous declaration here: /home/runner/work/Nim/Nim/pkgstemp/weave/weave/parallel_for.nim(172, 16)
  stack trace: (most recent call last)
HJarausch commented 3 years ago

I can reproduce this error iff I try to compile weave/parallel_for.nim itself. But, when I compile a nim source file which uses parallelFor there are no problems at all ( checked with several source files) Unfortunately, I don't understand the logic of Weave but I think it must compile weave/parallel_for.nim on the fly when I use import weave which in turn imports 

  weave/[
    parallel_tasks, parallel_for, parallel_for_staged,
    runtime],
  weave/state_machines/[sync_root, sync, sync_scope],
  weave/datatypes/flowvars,
  weave/cross_thread_com/flow_events,
  weave/contexts,
  weave/[executor, parallel_jobs]
fengkehh commented 3 years ago

This bug is a bit of a shame since I just built a general high level interface for embarassingly parallel problems based on Weave and it effectively kept all of my components that will make use of it on nim <= 1.4.4.

As a side note, on 1.4.4 the interface (and therefore Weave) works like a charm and the performance beats the pants off things like Numba. Color me impressed! :)

HJarausch commented 3 years ago

The bug has been fixed in the current developing version of Nim.

ringabout commented 2 years ago

latest error messages:

========================================================================================
2022-01-08T04:23:39.3361153Z In file included from /usr/lib/gcc/x86_64-linux-gnu/9/include/immintrin.h:55,
2022-01-08T04:23:39.3362046Z                  from /usr/lib/gcc/x86_64-linux-gnu/9/include/x86intrin.h:32,
2022-01-08T04:23:39.3362802Z                  from /home/runner/.cache/nim/gemm_weave_nestable_r/@mgemm_weave_nestable.nim.c:21:
2022-01-08T04:23:39.3365163Z /home/runner/.cache/nim/gemm_weave_nestable_r/@mgemm_weave_nestable.nim.c: In function ‘gebb_ukernel_float64_x86_AVX512__gemm95weave95nestable_1817’:
2022-01-08T04:23:39.3367566Z /usr/lib/gcc/x86_64-linux-gnu/9/include/avx512fintrin.h:325:1: error: inlining failed in call to always_inline ‘_mm512_setzero_pd’: target specific option mismatch
2022-01-08T04:23:39.3368640Z   325 | _mm512_setzero_pd (void)
2022-01-08T04:23:39.3369040Z       | ^~~~~~~~~~~~~~~~~
2022-01-08T04:23:39.3370382Z /home/runner/.cache/nim/gemm_weave_nestable_r/@mgemm_weave_nestable.nim.c:7599:11: note: called from here
2022-01-08T04:23:39.3371252Z  7599 |  AB13_1 = _mm512_setzero_pd();
2022-01-08T04:23:39.3372125Z       |           ^~~~~~~~~~~~~~~~~~~
2022-01-08T04:23:39.3372924Z In file included from /usr/lib/gcc/x86_64-linux-gnu/9/include/immintrin.h:55,
2022-01-08T04:23:39.3373789Z                  from /usr/lib/gcc/x86_64-linux-gnu/9/include/x86intrin.h:32,
2022-01-08T04:23:39.3374535Z                  from /home/runner/.cache/nim/gemm_weave_nestable_r/@mgemm_weave_nestable.nim.c:21:
2022-01-08T04:23:39.3375973Z /usr/lib/gcc/x86_64-linux-gnu/9/include/avx512fintrin.h:325:1: error: inlining failed in call to always_inline ‘_mm512_setzero_pd’: target specific option mismatch
2022-01-08T04:23:39.3376742Z   325 | _mm512_setzero_pd (void)
2022-01-08T04:23:39.3377097Z       | ^~~~~~~~~~~~~~~~~
2022-01-08T04:23:39.3377834Z /home/runner/.cache/nim/gemm_weave_nestable_r/@mgemm_weave_nestable.nim.c:7596:11: note: called from here
2022-01-08T04:23:39.3378487Z  7596 |  AB13_0 = _mm512_setzero_pd();
2022-01-08T04:23:39.3378862Z       |           ^~~~~~~~~~~~~~~~~~~
2022-01-08T04:23:39.3379800Z In file included from /usr/lib/gcc/x86_64-linux-gnu/9/include/immintrin.h:55,
2022-01-08T04:23:39.3380675Z                  from /usr/lib/gcc/x86_64-linux-gnu/9/include/x86intrin.h:32,
2022-01-08T04:23:39.3381429Z                  from /home/runner/.cache/nim/gemm_weave_nestable_r/@mgemm_weave_nestable.nim.c:21:
2022-01-08T04:23:39.3383222Z /usr/lib/gcc/x86_64-linux-gnu/9/include/avx512fintrin.h:325:1: error: inlining failed in call to always_inline ‘_mm512_setzero_pd’: target specific option mismatch
2022-01-08T04:23:39.3384051Z   325 | _mm512_setzero_pd (void)
2022-01-08T04:23:39.3384391Z       | ^~~~~~~~~~~~~~~~~
2022-01-08T04:23:39.3386232Z /home/runner/.cache/nim/gemm_weave_nestable_r/@mgemm_weave_nestable.nim.c:7593:11: note: called from here
2022-01-08T04:23:39.3387109Z  7593 |  AB12_1 = _mm512_setzero_pd();
2022-01-08T04:23:39.3387575Z       |           ^~~~~~~~~~~~~~~~~~~
2022-01-08T04:23:39.3388882Z compilation terminated due to -fmax-errors=3.
2022-01-08T04:23:39.3395635Z Error: execution of an external compiler program 'gcc -c  -w -fmax-errors=3 -pthread -O3 -fno-strict-aliasing -fno-ident   -I/home/runner/work/Nim/Nim/lib -I/home/runner/work/Nim/Nim/pkgstemp/weave/benchmarks/matmul_gemm_blas/gemm_pure_nim -o /home/runner/.cache/nim/gemm_weave_nestable_r/@mgemm_weave_nestable.nim.c.o /home/runner/.cache/nim/gemm_weave_nestable_r/@mgemm_weave_nestable.nim.c' failed with exit code: 1

ref https://github.com/mratsim/Arraymancer/issues/505 ref https://github.com/mratsim/Arraymancer/pull/542