search

mratsim / Arraymancer

A fast, ergonomic and portable tensor library in Nim with a deep learning focus for CPU, GPU and embedded devices via OpenMP, Cuda and OpenCL backends
https://mratsim.github.io/Arraymancer/
Apache License 2.0
1.34k stars 96 forks source link

error with openmp + debug macros #407

Closed brentp closed 4 years ago

brentp commented 4 years ago

Note you can recreate the problem below by adding --lineDir:on to Arraymancer's nim.cfg

in somalier, I get:

Error: execution of an external compiler program 'gcc -c  -w -mpopcnt -fopenmp   -I/home/brentp/src/nim/lib -I/home/brentp/src/somalier/src -o /home/brentp/.cache/nim/somalier_d/@m..@s..@s..@s.nimble@spkgs@sarraymancer-0.5.2@snn_primitives@snnp_softmax_cross_entropy.nim.c.o /home/brentp/.cache/nim/somalier_d/@m..@s..@s..@s.nimble@spkgs@sarraymancer-0.5.2@snn_primitives@snnp_softmax_cross_entropy.nim.c' failed with exit code: 1

/home/brentp/.cache/nim/somalier_d/@m..@s..@s..@s.nimble@spkgs@sarraymancer-0.5.2@snn_primitives@snnp_softmax_cross_entropy.nim.c: In function ‘sparse_softmax_cross_entropy__U2kN4wTjyBGH8VUyHptJTg’:
/home/brentp/.cache/nim/somalier_d/@m..@s..@s..@s.nimble@spkgs@sarraymancer-0.5.2@snn_primitives@snnp_softmax_cross_entropy.nim.c:35:19: error: invalid form of ‘#pragma omp atomic’ before ‘;’ token
   35 |       FR_.line = n; FR_.filename = file;
      |                   ^
/home/brentp/.cache/nim/somalier_d/@m..@s..@s..@s.nimble@spkgs@sarraymancer-0.5.2@snn_primitives@snnp_softmax_cross_entropy.nim.c:904:3: note: in expansion of macro ‘nimln_’
  904 |   nimln_(109, "/home/brentp/.nimble/pkgs/arraymancer-0.5.2/nn_primitives/nnp_s"

the code from 903-905 is:

                  #pragma omp atomic
                  nimln_(109, "/home/brentp/.nimble/pkgs/arraymancer-0.5.2/nn_primitives/nnp_s"
  "oftmax_cross_entropy.nim");
                  result += tmp;

I can bypass the error by compiling in release mode.

mratsim commented 4 years ago

I need to sprinkle some {.push stacktrace:off.} with OpenMP or going next line when emit

mratsim commented 4 years ago

Strange what is nimln_? On my platform I get #line and those compiles properly with linedir

DeepinScreenshot_select-area_20191207005454

brentp commented 4 years ago

Thanks for taking a look. I have these in the C file:

  #  define nimfr_(proc, file) \
      TFrame FR_; \
      FR_.procname = proc; FR_.filename = file; FR_.line = 0; FR_.len = 0; nimFrame(&FR_);

  #  define nimfrs_(proc, file, slots, length) \
      struct {TFrame* prev;NCSTRING procname;NI line;NCSTRING filename; NI len; VarSlot s[slots];} FR_; \
      FR_.procname = proc; FR_.filename = file; FR_.line = 0; FR_.len = length; nimFrame((TFrame*)&FR_);

  #  define nimln_(n, file) \
      FR_.line = n; FR_.filename = file;
brentp commented 4 years ago

this happened on 1.0.4 and 1.1.1 (dev).

mratsim commented 4 years ago

Can you try with the new commit, On my side it doesn't change the codegen but I had no nimframe inserted for starters

It might have to do with nim.cfg stacktrace:off not propagating properly on nimble installs https://github.com/mratsim/Arraymancer/blob/master/nim.cfg#L60

brentp commented 4 years ago

now in Arraymancer dir with --lineDir:on in nim.cfg, I see:

Error: execution of an external compiler program 'gcc -c  -w   -I/home/brentp/src/nim/lib -I/home/brentp/src/Arraymancer/tests/_split_tests -o /home/brentp/.cache/nim/tests_tensor_part02_d/@m..@s..@ssrc@slaser@sprimitives@smatrix_multiplication@sgemm_packing.nim.c.o /home/brentp/.cache/nim/tests_tensor_part02_d/@m..@s..@ssrc@slaser@sprimitives@smatrix_multiplication@sgemm_packing.nim.c' failed with exit code: 1

/home/brentp/src/Arraymancer/src/laser/primitives/matrix_multiplication/gemm_packing.nim: In function ‘pack_B_kc_nc__11bn39ac1x5110D79cGLGBkQ’:
/home/brentp/src/Arraymancer/src/laser/primitives/matrix_multiplication/gemm_packing.nim:79:37: error: stray ‘#’ in program
   79 |       #pragma omp parallel for
      |                                     ^
/home/brentp/src/Arraymancer/src/laser/primitives/matrix_multiplication/gemm_packing.nim:79:38: error: unknown type name ‘pragma’; did you mean ‘_Pragma’?
   79 |       #pragma omp parallel for
      |                                      ^     
      |                                      _Pragma
/home/brentp/src/Arraymancer/src/laser/primitives/matrix_multiplication/gemm_packing.nim:79:49: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘parallel’
   79 |       #pragma omp parallel for
      |                                                 ^       
/home/brentp/src/Arraymancer/src/laser/primitives/matrix_multiplication/gemm_packing.nim:80:23: error: ‘j’ undeclared (first use in this function)
   80 |       for (int j = 0; j < `unroll_stop`; j+=`NR`)
      |                       ^
/home/brentp/src/Arraymancer/src/laser/primitives/matrix_multiplication/gemm_packing.nim:80:23: note: each undeclared identifier is reported only once for each function it appears in
/home/brentp/src/Arraymancer/src/laser/primitives/matrix_multiplication/gemm_packing.nim:80:51: error: expected ‘;’ before ‘)’ token
   80 |       for (int j = 0; j < `unroll_stop`; j+=`NR`)
      |                                                   ^
      |                                                   ;
/home/brentp/src/Arraymancer/src/laser/primitives/matrix_multiplication/gemm_packing.nim:80:51: error: expected statement before ‘)’ token
/home/brentp/src/Arraymancer/src/laser/primitives/matrix_multiplication/gemm_packing.nim: In function ‘pack_B_kc_nc__11bn39ac1x5110D79cGLGBkQ_2’:
/home/brentp/src/Arraymancer/src/laser/primitives/matrix_multiplication/gemm_packing.nim:79:37: error: stray ‘#’ in program
   79 |       #pragma omp parallel for
      |                                     ^
/home/brentp/src/Arraymancer/src/laser/primitives/matrix_multiplication/gemm_packing.nim:79:38: error: unknown type name ‘pragma’; did you mean ‘_Pragma’?
   79 |       #pragma omp parallel for
      |                                      ^     
      |                                      _Pragma
/home/brentp/src/Arraymancer/src/laser/primitives/matrix_multiplication/gemm_packing.nim:79:49: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘parallel’
   79 |       #pragma omp parallel for
      |                                                 ^       
/home/brentp/src/Arraymancer/src/laser/primitives/matrix_multiplication/gemm_packing.nim:80:23: error: ‘j’ undeclared (first use in this function)
   80 |       for (int j = 0; j < `unroll_stop`; j+=`NR`)
      |                       ^
/home/brentp/src/Arraymancer/src/laser/primitives/matrix_multiplication/gemm_packing.nim:80:51: error: expected ‘;’ before ‘)’ token
   80 |       for (int j = 0; j < `unroll_stop`; j+=`NR`)
      |                                                   ^
      |                                                   ;
/home/brentp/src/Arraymancer/src/laser/primitives/matrix_multiplication/gemm_packing.nim:80:51: error: expected statement before ‘)’ token
stack trace: (most recent call last)
/tmp/nimblecache/nimscriptapi.nim(165, 16)
/home/brentp/src/Arraymancer/arraymancer_7004.nims(155, 8) testTask
/home/brentp/src/Arraymancer/arraymancer_7004.nims(146, 10) test
/home/brentp/src/nim/lib/system/nimscript.nim(252, 7) exec
/home/brentp/src/nim/lib/system/nimscript.nim(252, 7) Error: unhandled exception: FAILED: nim c -o:build/tests_tensor_part02 -r tests/_split_tests/tests_tensor_part02.nim [OSError]
     Error: Exception raised during nimble script execution

and in my project, I see:

Error: execution of an external compiler program 'gcc -c  -w -mpopcnt -fopenmp   -I/home/brentp/src/nim/lib -I/home/brentp/src/somalier/src -o /home/brentp/.cache/nim/somalier_d/@m..@s..@s..@s.nimble@spkgs@sarraymancer-0.5.2@snn_primitives@snnp_softmax_cross_entropy.nim.c.o /home/brentp/.cache/nim/somalier_d/@m..@s..@s..@s.nimble@spkgs@sarraymancer-0.5.2@snn_primitives@snnp_softmax_cross_entropy.nim.c' failed with exit code: 1

/home/brentp/.cache/nim/somalier_d/@m..@s..@s..@s.nimble@spkgs@sarraymancer-0.5.2@snn_primitives@snnp_softmax_cross_entropy.nim.c: In function ‘sparse_softmax_cross_entropy__U2kN4wTjyBGH8VUyHptJTg’:
/home/brentp/.cache/nim/somalier_d/@m..@s..@s..@s.nimble@spkgs@sarraymancer-0.5.2@snn_primitives@snnp_softmax_cross_entropy.nim.c:35:19: error: invalid form of ‘#pragma omp atomic’ before ‘;’ token
   35 |       FR_.line = n; FR_.filename = file;
      |                   ^
/home/brentp/.cache/nim/somalier_d/@m..@s..@s..@s.nimble@spkgs@sarraymancer-0.5.2@snn_primitives@snnp_softmax_cross_entropy.nim.c:904:3: note: in expansion of macro ‘nimln_’
  904 |   nimln_(111, "/home/brentp/.nimble/pkgs/arraymancer-0.5.2/nn_primitives/nnp_s"
      |   ^~~~~~
mratsim commented 4 years ago

I made another tentative. Still can't reproduce.

brentp commented 4 years ago

thanks for having a look. with f06c43d, if I add --lineDir:on to the top of arraymancer's nim.cfg then run

nim c --lineDir:on -r tests/_split_tests/tests_tensor_part02.nim

I see:

...
CC: stdlib_hashes.nim
Error: execution of an external compiler program 'gcc -c  -w   -I/home/brentp/src/nim/lib -I/home/brentp/src/Arraymancer/tests/_split_tests -o /home/brentp/.cache/nim/tests_tensor_part02_d/@m..@s..@ssrc@slaser@sprimitives@smatrix_multiplication@sgemm.nim.c.o /home/brentp/.cache/nim/tests_tensor_part02_d/@m..@s..@ssrc@slaser@sprimitives@smatrix_multiplication@sgemm.nim.c' failed with exit code: 1

/home/brentp/src/Arraymancer/src/laser/openmp.nim: In function ‘gemm_impl__bx5PNxdYQ3GkKKa7m9cGDkg’:
/home/brentp/src/Arraymancer/src/laser/openmp.nim:124:74: error: stray ‘#’ in program
  124 |   {.emit: ["#pragma omp parallel if (", predicate, ")"].}
      |                                                                          ^
/home/brentp/src/Arraymancer/src/laser/openmp.nim:124:75: error: unknown type name ‘pragma’; did you mean ‘_Pragma’?
  124 |   {.emit: ["#pragma omp parallel if (", predicate, ")"].}
      |                                                                           ^     
      |                                                                           _Pragma
/home/brentp/src/Arraymancer/src/laser/openmp.nim:124:86: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘parallel’
  124 |   {.emit: ["#pragma omp parallel if (", predicate, ")"].}
      |                                                                                      ^       
/home/brentp/src/Arraymancer/src/laser/openmp.nim: In function ‘gemm_impl__bx5PNxdYQ3GkKKa7m9cGDkg_2’:
/home/brentp/src/Arraymancer/src/laser/openmp.nim:124:74: error: stray ‘#’ in program
  124 |   {.emit: ["#pragma omp parallel if (", predicate, ")"].}
      |                                                                          ^
/home/brentp/src/Arraymancer/src/laser/openmp.nim:124:75: error: unknown type name ‘pragma’; did you mean ‘_Pragma’?
  124 |   {.emit: ["#pragma omp parallel if (", predicate, ")"].}
      |                                                                           ^     
      |                                                                           _Pragma
/home/brentp/src/Arraymancer/src/laser/openmp.nim:124:86: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘parallel’
  124 |   {.emit: ["#pragma omp parallel if (", predicate, ")"].}

these tests complete without --lineDir:on. I am on 

$ nim --version
Nim Compiler Version 1.1.1 [Linux: amd64]
Compiled at 2019-12-06
Copyright (c) 2006-2019 by Andreas Rumpf

git hash: 7213969901a4232013a6e6732d7b434970922510
active boot switches: -d:release
mratsim commented 4 years ago

Hopefully there are no more OpenMP errors left now.

brentp commented 4 years ago

now in somalier, I get:

Error: execution of an external compiler program 'gcc -c  -w -mpopcnt -fopenmp -g3 -Og  -I/home/brentp/src/nim/lib -I/home/brentp/src/somalier/src -o /home/brentp/.cache/nim/somalier_d/@m..@s..@s..@s.nimble@spkgs@sarraymancer-0.5.2@snn_primitives@snnp_softmax_cross_entropy.nim.c.o /home/brentp/.cache/nim/somalier_d/@m..@s..@s..@s.nimble@spkgs@sarraymancer-0.5.2@snn_primitives@snnp_softmax_cross_entropy.nim.c' failed with exit code: 1

/home/brentp/.nimble/pkgs/arraymancer-0.5.2/nn_primitives/nnp_softmax_cross_entropy.nim: In function ‘sparse_softmax_cross_entropy__U2kN4wTjyBGH8VUyHptJTg’:
/home/brentp/.nimble/pkgs/arraymancer-0.5.2/nn_primitives/nnp_softmax_cross_entropy.nim:112:37: error: stray ‘#’ in program
  112 |       #pragma omp atomic""".}
      |                                     ^
/home/brentp/.nimble/pkgs/arraymancer-0.5.2/nn_primitives/nnp_softmax_cross_entropy.nim:112:38: error: unknown type name ‘pragma’; did you mean ‘_Pragma’?
  112 |       #pragma omp atomic""".}
      |                                      ^     
      |                                      _Pragma
/home/brentp/.nimble/pkgs/arraymancer-0.5.2/nn_primitives/nnp_softmax_cross_entropy.nim:112:49: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘atomic’
  112 |       #pragma omp atomic""".}
      |                                                 ^

I am attaching the C file in case it helps. o.c.gz

mratsim commented 4 years ago

I manage to reproduce the issue by cloning somalier and running the tests It should be fixed for good now.

I'll definitely move away from OpenMP ASAP.

brentp commented 4 years ago

yes, this is fixed. thanks very much!