search

animetosho / ParPar

High performance PAR2 create client for NodeJS
195 stars 21 forks source link

Error when trying to run Parpar from the command line on Debian #29

Closed tommyboy24 closed 3 years ago

tommyboy24 commented 3 years ago

I have successfully installed Parpar with no errors. But When running in the terminal I get this error

`tommyboy@storm :~$ parpar internal/modules/cjs/loader.js:1187 return process.dlopen(module, path.toNamespacedPath(filename)); ^

Error: /home200/tommy/lib/node_modules/@animetosho/parpar/build/Release/parpar_gf.node: undefined symbol: gf_w16_xor_final_avx512 at Object.Module._extensions..node (internal/modules/cjs/loader.js:1187:18) at Module.load (internal/modules/cjs/loader.js:985:32) at Function.Module._load (internal/modules/cjs/loader.js:878:14) at Module.require (internal/modules/cjs/loader.js:1025:19) at require (internal/modules/cjs/helpers.js:72:18) at Object. (/home23/usb668/lib/node_modules/@animetosho/parpar/lib/par2.js:4:10) at Module._compile (internal/modules/cjs/loader.js:1137:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1157:10) at Module.load (internal/modules/cjs/loader.js:985:32) at Function.Module._load (internal/modules/cjs/loader.js:878:14) `

animetosho commented 3 years ago

Thanks, are you able to give more info on how you installed it?

Can you try going into the ParPar folder (/home200/tommy/lib/node_modules/@animetosho/parpar in your case) and issuing a node-gyp rebuild command to see if there's any warnings/errors during build?

tommyboy24 commented 3 years ago

I tried node-gyp rebuild with no errors during build. But then I tried running parpar command again with the same error.

Off topic - I installed Nyuu the same way and it runs great

I presume you installed v0.3.2 via NPM. I also assume this is an amd64 setup? Yes

What version of Debian are you running? Debian 9

What compiler and version are you using? Latest version

What version of NodeJS are you running? Latest version

animetosho commented 3 years ago

Thanks for that. Unfortunately "Latest version" can mean all sorts of things, and I'm unable to figure out what the issue is from the info given.

Would you be able to run the following commands and post the output?

cat /etc/debian_version
c++ -v  # only need the version info
node -v || nodejs -v
echo|c++ -dM -E -march=native -|grep tune
echo|c++ -dM -E -march=native -|grep AVX512
grep model /proc/cpuinfo|head -n4
nm /home200/tommy/lib/node_modules/@animetosho/parpar/build/Release/parpar_gf.node

Thanks

tommyboy24 commented 3 years ago

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. Last login: Fri Mar 12 12:21:59 2021 from 185.69.144.247 usb842@lw851:~$ cat /etc/debian_version c++ -v # only need the version info node -v || nodejs -v echo|c++ -dM -E -march=native -|grep tune echo|c++ -dM -E -march=native -|grep AVX512 grep model /proc/cpuinfo|head -n4 nm /lib/node_modules/@animetosho/parpar/build/Release/parpar_gf.node9.13 usb842@lw851:~$ c++ -v # only need the version info Using built-in specs. COLLECT_GCC=c++ COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/6/lto-wrapper Target: x86_64-linux-gnu Configured with: ../src/configure -v --with-pkgversion='Debian 6.3.0-18+deb9u1' --with-bugurl=file:///usr/share/doc/gcc-6/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-6 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-6-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-6-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-6-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu Thread model: posix gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1) usb842@lw851:~$ node -v || nodejs -v v12.18.3 usb842@lw851:~$ echo|c++ -dM -E -march=native -|grep tune usb842@lw851:~$ echo|c++ -dM -E -march=native -|grep AVX512

define AVX512F 1

define AVX512BW 1

define AVX512CD 1

define AVX512DQ 1

usb842@lw851:~$ grep model /proc/cpuinfo|head -n4 model : 85 model name : Intel(R) Xeon(R) Gold 6139 CPU @ 2.30GHz model : 85 model name : Intel(R) Xeon(R) Gold 6139 CPU @ 2.30GHz usb842@lw851:~$ nm /lib/node_modules/@animetosho/parpar/build/Release/parpar_gf.node nm: '/lib/node_modules/@animetosho/parpar/build/Release/parpar_gf.node': No such file usb842@lw851:~$ ^C usb842@lw851:~$ clear usb842@lw851:~$ cat /etc/debian_version 9.13 usb842@lw851:~$ c++ -v # only need the version info grep model /proc/cpuinfo|head -n4 nm ~/lib/node_modules/@animetosho/parpar/build/Release/parpar_gf.nodeUsing built-in specs. COLLECT_GCC=c++ COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/6/lto-wrapper Target: x86_64-linux-gnu Configured with: ../src/configure -v --with-pkgversion='Debian 6.3.0-18+deb9u1' --with-bugurl=file:///usr/share/doc/gcc-6/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-6 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-6-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-6-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-6-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu Thread model: posix gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1) usb842@lw851:~$ node -v || nodejs -v v12.18.3 usb842@lw851:~$ echo|c++ -dM -E -march=native -|grep tune usb842@lw851:~$ echo|c++ -dM -E -march=native -|grep AVX512

define AVX512F 1

define AVX512BW 1

define AVX512CD 1

define AVX512DQ 1

usb842@lw851:~$ grep model /proc/cpuinfo|head -n4 model : 85 model name : Intel(R) Xeon(R) Gold 6139 CPU @ 2.30GHz model : 85 model name : Intel(R) Xeon(R) Gold 6139 CPU @ 2.30GHz usb842@lw851:~$ nm ~/lib/node_modules/@animetosho/parpar/build/Release/parpar_gf.node U aligned_alloc@@GLIBC_2.16 U assert_fail@@GLIBC_2.2.5 0000000000225350 B bss_start 00000000002553c0 B CHUNK_SIZE 0000000000225380 b completed.6972 00000000002553fc B cpu_detect_run w cxa_finalize@@GLIBC_2.2.5 0000000000225340 D defaultNumThreads 0000000000003e80 t deregister_tm_clones 000000000000b310 T detect_cpu 0000000000003f10 t __do_global_dtors_aux 0000000000224bb0 t do_global_dtors_aux_fini_array_entry 00000000002252e0 d dso_handle 0000000000224bc0 d _DYNAMIC 0000000000225350 D _edata 0000000000257c40 B _end U exit@@GLIBC_2.2.5 00000000000207f4 T _fini U fprintf@@GLIBC_2.2.5 0000000000003f50 t frame_dummy 0000000000224ba0 t frame_dummy_init_array_entry 0000000000023e30 r FRAME_END U free@@GLIBC_2.2.5 U fwrite@@GLIBC_2.2.5 00000000002253b8 B gf 00000000002253a4 B gfCount 00000000000083a0 T gf_do_final_region_alignment 0000000000008380 T gf_do_initial_region_alignment 00000000002253c0 B gf_exp 0000000000008020 T gf_free 0000000000007ff0 T gf_init_easy 0000000000007ea0 T gf_init_hard 0000000000008940 T gf_memcpy 00000000002553cc B GF_METHOD 00000000002553c8 B GF_METHOD_ARG1 00000000002553c4 B GF_METHOD_ARG2 00000000002253a8 B gf_method_wordsize 00000000000083e0 T gf_multby_one 00000000000083c0 T gf_multby_zero 0000000000007e40 T gf_scratch_size 0000000000008070 T gf_set_region_data 0000000000007e70 T gf_size 0000000000007d60 t gf_slow_multiply_region 00000000000088a0 T gf_w16_default_regionX 000000000000b570 T gf_w16_init 000000000000b560 T gf_w16_scratch_size 000000000000d760 T gf_w16_split_4_16_lazy_altmap_multiply_region_avx2 000000000000f290 T gf_w16_split_4_16_lazy_altmap_multiply_region_avx512 000000000000b9c0 T gf_w16_split_4_16_lazy_altmap_multiply_region_sse 000000000000dd70 T gf_w16_split_4_16_lazy_altmap_multiply_regionX_avx2 000000000000f8e0 T gf_w16_split_4_16_lazy_altmap_multiply_regionX_avx512 000000000000c000 T gf_w16_split_4_16_lazy_altmap_multiply_regionX_sse 000000000000d120 T gf_w16_split_4_16_lazy_sse_multiply_region 000000000000af80 t gf_w16_split_8_16_lazy_multiply_region 0000000000008d80 t gf_w16_split_8_16_lazy_multiply_region.part.5 000000000000d5f0 T gf_w16_split_final_avx2 000000000000f0f0 T gf_w16_split_final_avx512 000000000000b850 T gf_w16_split_final_sse 00000000000089b0 t gf_w16_split_init 0000000000008980 t gf_w16_split_null 000000000000d450 T gf_w16_split_start_avx2 000000000000ef10 T gf_w16_split_start_avx512 000000000000b6b0 T gf_w16_split_start_sse 000000000001bbf0 T gf_w16_xor_create_jit_lut_avx2 000000000001e320 T gf_w16_xor_create_jit_lut_avx512 0000000000010c70 T gf_w16_xor_create_jit_lut_sse 000000000001dbd0 T gf_w16_xor_final_avx2 U gf_w16_xor_final_avx512 000000000001b700 T gf_w16_xor_final_sse 000000000000afd0 t gf_w16_xor_init 000000000001dfb0 T gf_w16_xor_init_jit_avx2 000000000001e0e0 T gf_w16_xor_init_jit_avx512 000000000001bad0 T gf_w16_xor_init_jit_sse 000000000001be20 T gf_w16_xor_lazy_jit_altmap_multiply_region_avx2 000000000001e0d0 T gf_w16_xor_lazy_jit_altmap_multiply_region_avx512 00000000000110c0 T gf_w16_xor_lazy_jit_altmap_multiply_region_sse 0000000000012c90 T gf_w16_xor_lazy_sse_altmap_multiply_region 000000000001d770 T gf_w16_xor_start_avx2 U gf_w16_xor_start_avx512 000000000001b300 T gf_w16_xor_start_sse 0000000000225000 d _GLOBAL_OFFSETTABLE w gmon_start 0000000000022850 r GNU_EH_FRAME_HDR U GOMP_parallel@@GOMP_4.0 00000000002553e4 B has_avx2 00000000002553dc B has_avx512bw 00000000002553e0 B has_avxslow 00000000002553d8 B has_gfni 00000000002553d4 B has_htt 00000000002553d0 B has_neon 00000000002553e8 B has_pclmul 00000000002553f0 B has_slow_shuffle 00000000002553f8 B has_ssse3 0000000000003898 T _init 00000000002453c0 B input_lookup w _ITM_deregisterTMCloneTable w _ITM_registerTMCloneTable 0000000000224bb8 d __JCR_END 0000000000224bb8 d JCR_LIST__ w _Jv_RegisterClasses U malloc@@GLIBC_2.2.5 0000000000225344 D maxNumThreads 000000000001f850 T md5_final 000000000001eb30 T md5_init 000000000001eb70 T md5_multi_update 000000000001e330 t md5_update_block 000000000001eaf0 T md5_update_single 000000000001f970 T md5_update_sse 000000000001ef50 T md5_update_zeroes 000000000022534c D MEM_ALIGN U memcpy@@GLIBC_2.14 U memset@@GLIBC_2.2.5 0000000000225348 D MEM_WALIGN 0000000000225384 B mmActiveTasks U mmap64@@GLIBC_2.2.5 U munmap@@GLIBC_2.2.5 U node_module_register U omp_get_max_threads@@OMP_1.0 U omp_get_num_procs@@OMP_1.0 U omp_get_num_threads@@OMP_1.0 U omp_get_thread_num@@OMP_1.0 U omp_set_num_threads@@OMP_1.0 U realloc@@GLIBC_2.2.5 0000000000003e60 t _register_parpar_gf 0000000000003ec0 t register_tm_clones 00000000002253b0 B size_hint U stderr@@GLIBC_2.2.5 0000000000225350 d TMC_END 00000000002253a0 B using_altmap U uv_default_loop U uv_queue_work 0000000000257400 B xor256_jit_clut_code1 0000000000257800 B xor256_jit_clut_info_mem 0000000000255404 B xor256_jit_created 0000000000022260 r xor256_jit_len 0000000000257000 B xor256_jit_nums 0000000000257840 B xor256_jit_rmask 00000000002567c0 B xor_jit_clut_code1 0000000000256c00 B xor_jit_clut_code2 0000000000255440 B xor_jit_clut_code3 0000000000255540 B xor_jit_clut_code4 0000000000256140 B xor_jit_clut_code5 0000000000256540 B xor_jit_clut_code6 0000000000256700 B xor_jit_clut_info_mem 0000000000256640 B xor_jit_clut_info_reg 00000000002566c0 B xor_jit_clut_ncinfo_mem 0000000000256780 B xor_jit_clut_ncinfo_reg 0000000000256bc0 B xor_jit_clut_ncinfo_rm 0000000000255940 B xor_jit_clut_nocomm 0000000000255400 B xor_jit_created 0000000000006750 T _Z14parpar_gf_initN2v85LocalINS_6ObjectEEENS0_INS_5ValueEEEPv 00000000000079d0 T _Z15ppgf_get_methodPiSPPKcS 00000000000077e0 T _Z15ppgf_prepinputmmPcS 0000000000007c80 T _Z15ppgf_set_methodiii 0000000000007950 T _Z17ppgf_finish_inputjPPtm 00000000000076b0 T _Z17ppgf_multiply_matPPtPmjmS0_S1_ji 00000000000073c0 t _Z17ppgf_multiply_matPPtPmjmS0_S1_ji._omp_fn.0 0000000000007560 T _Z19ppgf_init_constantsv 0000000000007c00 T _Z19ppgf_init_gf_modulev 0000000000007690 T _Z19ppgf_maybe_setup_gfv 0000000000007ba0 T _Z20ppgf_get_num_threadsv 0000000000007bd0 T _Z20ppgf_set_num_threadsi 0000000000007660 T _Z26ppgf_omp_check_num_threadsv U _ZdaPv@@GLIBCXX_3.4 U _ZdlPvm@@CXXABI_1.3.9 0000000000004700 t _ZL10MD5Update2RKN2v820FunctionCallbackInfoINS_5ValueEEE 00000000000043f0 t _ZL13GetNumThreadsRKN2v820FunctionCallbackInfoINS_5ValueEEE 0000000000005630 t _ZL13MultiplyMultiRKN2v820FunctionCallbackInfoINS_5ValueEEE 0000000000004460 t _ZL13SetMaxThreadsRKN2v820FunctionCallbackInfoINS_5ValueEEE 00000000000045d0 t _ZL15AlignmentOffsetRKN2v820FunctionCallbackInfoINS_5ValueEEE 00000000000061e0 t _ZL15MD5UpdateZeroesRKN2v820FunctionCallbackInfoINS_5ValueEEE 0000000000004a50 t _ZL6FinishRKN2v820FunctionCallbackInfoINS_5ValueEEE 0000000000004530 t _ZL6FinishRKN2v820FunctionCallbackInfoINS_5ValueEEE._omp_fn.0 0000000000005490 t _ZL6MMWorkP9uv_work_s 00000000000054e0 t _ZL7MMAfterP9uv_work_si 0000000000225300 d _ZL7_module 00000000000070f0 t _ZL8alloc_gfv 0000000000006540 t _ZL8MD5StartRKN2v820FunctionCallbackInfoINS_5ValueEEE 0000000000007270 t _ZL8setup_gfv 00000000000063a0 t _ZL9MD5FinishRKN2v820FunctionCallbackInfoINS_5ValueEEE 0000000000005290 t _ZL9PrepInputRKN2v820FunctionCallbackInfoINS_5ValueEEE 0000000000003f80 t _ZL9SetMethodRKN2v820FunctionCallbackInfoINS_5ValueEEE U _ZN2v811HandleScope12CreateHandleEPNS_8internal7IsolateEm U _ZN2v811HandleScopeC1EPNS_7IsolateE U _ZN2v811HandleScopeD1Ev U _ZN2v816FunctionTemplate11GetFunctionENS_5LocalINS_7ContextEEE U _ZN2v816FunctionTemplate3NewEPNS_7IsolateEPFvRKNS_20FunctionCallbackInfoINS_5ValueEEEENS_5LocalIS4_EENSA_INS_9SignatureEEEiNS_19ConstructorBehaviorENS_14SideEffectTypeE U _ZN2v82V812ToLocalEmptyEv U _ZN2v82V813DisposeGlobalEPm U _ZN2v82V817FromJustIsNothingEv U _ZN2v82V818GlobalizeReferenceEPNS_8internal7IsolateEPm U _ZN2v86Object3GetENS_5LocalINS_7ContextEEEj U _ZN2v86Object3NewEPNS_7IsolateE U _ZN2v86Object3SetENS_5LocalINS_7ContextEEENS1_INS5ValueEEES5 U _ZN2v86String11NewFromUtf8EPNS_7IsolateEPKcNS_13NewStringTypeEi U _ZN2v86String14NewFromOneByteEPNS_7IsolateEPKhNS_13NewStringTypeEi U _ZN2v87Integer3NewEPNS_7IsolateEi U _ZN2v87Isolate10GetCurrentEv U _ZN2v87Isolate14ThrowExceptionENS_5LocalINS_5ValueEEE U _ZN2v87Isolate17GetCurrentContextEv U _ZN2v88Function7SetNameENS_5LocalINS_6StringEEE U _ZN2v89Exception5ErrorENS_5LocalINS_6StringEEE U _ZN4node12MakeCallbackEPN2v87IsolateENS0_5LocalINS0_6ObjectEEEPKciPNS3_INS0_5ValueEEENS_13async_contextE U _ZN4node6Buffer11HasInstanceEN2v85LocalINS1_5ValueEEE U _ZN4node6Buffer3NewEPN2v87IsolateEm U _ZN4node6Buffer4DataEN2v85LocalINS1_5ValueEEE U _ZN4node6Buffer4DataEN2v85LocalINS1_6ObjectEEE U _ZN4node6Buffer6LengthEN2v85LocalINS1_5ValueEEE U _Znam@@GLIBCXX_3.4 U _ZNK2v85Array6LengthEv U _ZNK2v85Value10IsFunctionEv U _ZNK2v85Value7IsArrayEv U _ZNK2v87Boolean5ValueEv U _ZNK2v87Integer5ValueEv U _Znwm@@GLIBCXX_3.4 0000000000020d60 r _ZZL7MMAfterP9uv_work_siE19__PRETTY_FUNCTION :~$

animetosho commented 3 years ago

Ah, I see what's going on - something weird with compiler CPU detection (doesn't pick up AVX512VL). Might be something to do with old GCC compiler and relatively new CPU.

Can you try going into the ParPar directory and issuing the following command, then seeing if it works?

CFLAGS=-mavx512vl node-gyp rebuild

(there's arguably a bug on ParPar's side as well, but I've changed the relevant code so much since v0.3.2 that I'm not sure it's worth a fix for a unique case if the above command fixes it)

tommyboy24 commented 3 years ago

Thanks its all working now

animetosho commented 3 years ago

Thanks for confirming!