search

animetosho / ParPar

High performance PAR2 create client for NodeJS
195 stars 21 forks source link

Error compiling on RPI (armhf) #21

Closed mbruel closed 4 years ago

mbruel commented 4 years ago

Hi, I've just tried to install ParPar on my Raspberry PI 4 running Raspbian and I'm getting a compilation error:

In file included from ../gf-complete/gf_w16.c:25:
../gf-complete/gf_w16_additions.c: In function ‘detect_cpu’:
../gf-complete/gf_w16_additions.c:124:35: error: ‘HWCAP_ASIMD’ undeclared (first use in this function); did you mean ‘HWCAP_THUMB’?
  has_neon = getauxval(AT_HWCAP) & HWCAP_ASIMD;
                                   ^~~~~~~~~~~
                                   HWCAP_THUMB
../gf-complete/gf_w16_additions.c:124:35: note: each undeclared identifier is reported only once for each function it appears in
make: *** [gf-complete.target.mk:115: Release/obj.target/gf-complete/gf-complete/gf_w16.o] Error 1
make: Leaving directory '/home/pi/apps/ParPar/build'
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/share/node-gyp/lib/build.js:262:23)
gyp ERR! stack     at ChildProcess.emit (events.js:189:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
gyp ERR! System Linux 4.19.75-v7l+
gyp ERR! command "/usr/bin/node" "/usr/bin/node-gyp" "rebuild"
gyp ERR! cwd /home/pi/apps/ParPar
gyp ERR! node -v v10.15.2
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok 
pi@raspberrypi:~/apps/ParPar $

Can it be compiled on armhf architecture or is it not supported?

Uukrull commented 4 years ago

Check this

    #ifndef HWCAP_ASIMD // This is not def'd on pre-ASIMD platforms
        const unsigned long HWCAP_ASIMD = 0;
    #endif
mbruel commented 4 years ago

hum thanks, but correcting this leads to other errors on wrong type assignments :$

 ../gf-complete/gf_w16/shuffle128_neon.c: In function ‘gf_w16_split_4_16_lazy_multiply_region_neon’:
../gf-complete/gf_w16/shuffle128_neon.c:100:14: warning: implicit declaration of function ‘vuzp1q_u8’; did you mean ‘vuzpq_u8’? [-Wimplicit-function-declaration]
   tbl_l[0] = vuzp1q_u8(rl, rh);
              ^~~~~~~~~
              vuzpq_u8
../gf-complete/gf_w16/shuffle128_neon.c:100:12: error: incompatible types when assigning to type ‘uint8x16_t’ from type ‘int’
   tbl_l[0] = vuzp1q_u8(rl, rh);
            ^
../gf-complete/gf_w16/shuffle128_neon.c:101:14: warning: implicit declaration of function ‘vuzp2q_u8’; did you mean ‘vuzpq_u8’? [-Wimplicit-function-declaration]
   tbl_h[0] = vuzp2q_u8(rl, rh);
              ^~~~~~~~~
              vuzpq_u8
../gf-complete/gf_w16/shuffle128_neon.c:101:12: error: incompatible types when assigning to type ‘uint8x16_t’ from type ‘int’
   tbl_h[0] = vuzp2q_u8(rl, rh);
            ^
../gf-complete/gf_w16/shuffle128_neon.c:108:29: warning: implicit declaration of function ‘vqtbl1q_u8’; did you mean ‘vtbl1_u8’? [-Wimplicit-function-declaration]
     tbl_l[c] = veorq_u8(rl, vqtbl1q_u8(ltd->poly->val[0], ri)); \
                             ^~~~~~~~~~
../gf-complete/gf_w16/shuffle128_neon.c:134:3: note: in expansion of macro ‘MUL16’
   MUL16(0, 1);
   ^~~~~
../gf-complete/gf_w16/shuffle128_neon.c:108:29: error: incompatible type for argument 2 of ‘veorq_u8’
     tbl_l[c] = veorq_u8(rl, vqtbl1q_u8(ltd->poly->val[0], ri)); \
                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../gf-complete/gf_w16/shuffle128_neon.c:134:3: note: in expansion of macro ‘MUL16’
   MUL16(0, 1);
   ^~~~~
In file included from ../gf-complete/gf_w16/../gf_complete.h:40,
                 from ../gf-complete/gf_w16/../gf_int.h:14,
                 from ../gf-complete/gf_w16/shuffle128_neon.c:44:
/usr/lib/gcc/arm-linux-gnueabihf/8/include/arm_neon.h:14044:38: note: expected ‘uint8x16_t’ but argument is of type ‘int’
 veorq_u8 (uint8x16_t __a, uint8x16_t __b)
                           ~~~~~~~~~~~^~~
../gf-complete/gf_w16/shuffle128_neon.c:109:29: error: incompatible type for argument 2 of ‘veorq_u8’
     tbl_h[c] = veorq_u8(rh, vqtbl1q_u8(ltd->poly->val[1], ri))
                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../gf-complete/gf_w16/shuffle128_neon.c:134:3: note: in expansion of macro ‘MUL16’
   MUL16(0, 1);
   ^~~~~
In file included from ../gf-complete/gf_w16/../gf_complete.h:40,
                 from ../gf-complete/gf_w16/../gf_int.h:14,
                 from ../gf-complete/gf_w16/shuffle128_neon.c:44:
/usr/lib/gcc/arm-linux-gnueabihf/8/include/arm_neon.h:14044:38: note: expected ‘uint8x16_t’ but argument is of type ‘int’
 veorq_u8 (uint8x16_t __a, uint8x16_t __b)
                           ~~~~~~~~~~~^~~
../gf-complete/gf_w16/shuffle128_neon.c:108:29: error: incompatible type for argument 2 of ‘veorq_u8’
     tbl_l[c] = veorq_u8(rl, vqtbl1q_u8(ltd->poly->val[0], ri)); \
                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../gf-complete/gf_w16/shuffle128_neon.c:135:3: note: in expansion of macro ‘MUL16’
   MUL16(1, 2);
   ^~~~~
In file included from ../gf-complete/gf_w16/../gf_complete.h:40,
                 from ../gf-complete/gf_w16/../gf_int.h:14,
                 from ../gf-complete/gf_w16/shuffle128_neon.c:44:
/usr/lib/gcc/arm-linux-gnueabihf/8/include/arm_neon.h:14044:38: note: expected ‘uint8x16_t’ but argument is of type ‘int’
 veorq_u8 (uint8x16_t __a, uint8x16_t __b)
                           ~~~~~~~~~~~^~~
../gf-complete/gf_w16/shuffle128_neon.c:109:29: error: incompatible type for argument 2 of ‘veorq_u8’
     tbl_h[c] = veorq_u8(rh, vqtbl1q_u8(ltd->poly->val[1], ri))
                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../gf-complete/gf_w16/shuffle128_neon.c:135:3: note: in expansion of macro ‘MUL16’
   MUL16(1, 2);
   ^~~~~
In file included from ../gf-complete/gf_w16/../gf_complete.h:40,
                 from ../gf-complete/gf_w16/../gf_int.h:14,
                 from ../gf-complete/gf_w16/shuffle128_neon.c:44:
/usr/lib/gcc/arm-linux-gnueabihf/8/include/arm_neon.h:14044:38: note: expected ‘uint8x16_t’ but argument is of type ‘int’
 veorq_u8 (uint8x16_t __a, uint8x16_t __b)
                           ~~~~~~~~~~~^~~
../gf-complete/gf_w16/shuffle128_neon.c:108:29: error: incompatible type for argument 2 of ‘veorq_u8’
     tbl_l[c] = veorq_u8(rl, vqtbl1q_u8(ltd->poly->val[0], ri)); \
                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../gf-complete/gf_w16/shuffle128_neon.c:136:3: note: in expansion of macro ‘MUL16’
   MUL16(2, 3);
   ^~~~~
In file included from ../gf-complete/gf_w16/../gf_complete.h:40,
                 from ../gf-complete/gf_w16/../gf_int.h:14,
                 from ../gf-complete/gf_w16/shuffle128_neon.c:44:
/usr/lib/gcc/arm-linux-gnueabihf/8/include/arm_neon.h:14044:38: note: expected ‘uint8x16_t’ but argument is of type ‘int’
 veorq_u8 (uint8x16_t __a, uint8x16_t __b)
                           ~~~~~~~~~~~^~~
../gf-complete/gf_w16/shuffle128_neon.c:109:29: error: incompatible type for argument 2 of ‘veorq_u8’
     tbl_h[c] = veorq_u8(rh, vqtbl1q_u8(ltd->poly->val[1], ri))
                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../gf-complete/gf_w16/shuffle128_neon.c:136:3: note: in expansion of macro ‘MUL16’
   MUL16(2, 3);
   ^~~~~
In file included from ../gf-complete/gf_w16/../gf_complete.h:40,
                 from ../gf-complete/gf_w16/../gf_int.h:14,
                 from ../gf-complete/gf_w16/shuffle128_neon.c:44:
/usr/lib/gcc/arm-linux-gnueabihf/8/include/arm_neon.h:14044:38: note: expected ‘uint8x16_t’ but argument is of type ‘int’
 veorq_u8 (uint8x16_t __a, uint8x16_t __b)
                           ~~~~~~~~~~~^~~
../gf-complete/gf_w16/shuffle128_neon.c:155:10: error: incompatible types when assigning to type ‘uint8x16_t’ from type ‘int’
       rl = vqtbl1q_u8(tbl_l[0], vandq_u8(va.val[0], loset));
          ^
../gf-complete/gf_w16/shuffle128_neon.c:156:10: error: incompatible types when assigning to type ‘uint8x16_t’ from type ‘int’
       rh = vqtbl1q_u8(tbl_h[0], vandq_u8(va.val[0], loset));
          ^
../gf-complete/gf_w16/shuffle128_neon.c:157:25: error: incompatible type for argument 2 of ‘veorq_u8’
       rl = veorq_u8(rl, vqtbl1q_u8(tbl_l[2], vandq_u8(va.val[1], loset)));
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../gf-complete/gf_w16/../gf_complete.h:40,
                 from ../gf-complete/gf_w16/../gf_int.h:14,
                 from ../gf-complete/gf_w16/shuffle128_neon.c:44:
/usr/lib/gcc/arm-linux-gnueabihf/8/include/arm_neon.h:14044:38: note: expected ‘uint8x16_t’ but argument is of type ‘int’
 veorq_u8 (uint8x16_t __a, uint8x16_t __b)
                           ~~~~~~~~~~~^~~
../gf-complete/gf_w16/shuffle128_neon.c:158:25: error: incompatible type for argument 2 of ‘veorq_u8’
       rh = veorq_u8(rh, vqtbl1q_u8(tbl_h[2], vandq_u8(va.val[1], loset)));
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../gf-complete/gf_w16/../gf_complete.h:40,
                 from ../gf-complete/gf_w16/../gf_int.h:14,
                 from ../gf-complete/gf_w16/shuffle128_neon.c:44:
/usr/lib/gcc/arm-linux-gnueabihf/8/include/arm_neon.h:14044:38: note: expected ‘uint8x16_t’ but argument is of type ‘int’
 veorq_u8 (uint8x16_t __a, uint8x16_t __b)
                           ~~~~~~~~~~~^~~
../gf-complete/gf_w16/shuffle128_neon.c:163:25: error: incompatible type for argument 2 of ‘veorq_u8’
       rl = veorq_u8(rl, vqtbl1q_u8(tbl_l[1], va.val[0]));
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../gf-complete/gf_w16/../gf_complete.h:40,
                 from ../gf-complete/gf_w16/../gf_int.h:14,
                 from ../gf-complete/gf_w16/shuffle128_neon.c:44:
/usr/lib/gcc/arm-linux-gnueabihf/8/include/arm_neon.h:14044:38: note: expected ‘uint8x16_t’ but argument is of type ‘int’
 veorq_u8 (uint8x16_t __a, uint8x16_t __b)
                           ~~~~~~~~~~~^~~
../gf-complete/gf_w16/shuffle128_neon.c:164:25: error: incompatible type for argument 2 of ‘veorq_u8’
       rh = veorq_u8(rh, vqtbl1q_u8(tbl_h[1], va.val[0]));
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../gf-complete/gf_w16/../gf_complete.h:40,
                 from ../gf-complete/gf_w16/../gf_int.h:14,
                 from ../gf-complete/gf_w16/shuffle128_neon.c:44:
/usr/lib/gcc/arm-linux-gnueabihf/8/include/arm_neon.h:14044:38: note: expected ‘uint8x16_t’ but argument is of type ‘int’
 veorq_u8 (uint8x16_t __a, uint8x16_t __b)
                           ~~~~~~~~~~~^~~
../gf-complete/gf_w16/shuffle128_neon.c:165:32: error: incompatible type for argument 2 of ‘veorq_u8’
       va.val[0] = veorq_u8(rl, vqtbl1q_u8(tbl_l[3], va.val[1]));
                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../gf-complete/gf_w16/../gf_complete.h:40,
                 from ../gf-complete/gf_w16/../gf_int.h:14,
                 from ../gf-complete/gf_w16/shuffle128_neon.c:44:
/usr/lib/gcc/arm-linux-gnueabihf/8/include/arm_neon.h:14044:38: note: expected ‘uint8x16_t’ but argument is of type ‘int’
 veorq_u8 (uint8x16_t __a, uint8x16_t __b)
                           ~~~~~~~~~~~^~~
../gf-complete/gf_w16/shuffle128_neon.c:166:32: error: incompatible type for argument 2 of ‘veorq_u8’
       va.val[1] = veorq_u8(rh, vqtbl1q_u8(tbl_h[3], va.val[1]));
                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../gf-complete/gf_w16/../gf_complete.h:40,
                 from ../gf-complete/gf_w16/../gf_int.h:14,
                 from ../gf-complete/gf_w16/shuffle128_neon.c:44:
/usr/lib/gcc/arm-linux-gnueabihf/8/include/arm_neon.h:14044:38: note: expected ‘uint8x16_t’ but argument is of type ‘int’
 veorq_u8 (uint8x16_t __a, uint8x16_t __b)
                           ~~~~~~~~~~~^~~
../gf-complete/gf_w16/shuffle128_neon.c:179:10: error: incompatible types when assigning to type ‘uint8x16_t’ from type ‘int’
       rl = vqtbl1q_u8(tbl_l[0], vandq_u8(va.val[0], loset));
          ^
../gf-complete/gf_w16/shuffle128_neon.c:180:10: error: incompatible types when assigning to type ‘uint8x16_t’ from type ‘int’
       rh = vqtbl1q_u8(tbl_h[0], vandq_u8(va.val[0], loset));
          ^
../gf-complete/gf_w16/shuffle128_neon.c:181:25: error: incompatible type for argument 2 of ‘veorq_u8’
       rl = veorq_u8(rl, vqtbl1q_u8(tbl_l[2], vandq_u8(va.val[1], loset)));
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../gf-complete/gf_w16/../gf_complete.h:40,
                 from ../gf-complete/gf_w16/../gf_int.h:14,
                 from ../gf-complete/gf_w16/shuffle128_neon.c:44:
/usr/lib/gcc/arm-linux-gnueabihf/8/include/arm_neon.h:14044:38: note: expected ‘uint8x16_t’ but argument is of type ‘int’
 veorq_u8 (uint8x16_t __a, uint8x16_t __b)
                           ~~~~~~~~~~~^~~
../gf-complete/gf_w16/shuffle128_neon.c:182:25: error: incompatible type for argument 2 of ‘veorq_u8’
       rh = veorq_u8(rh, vqtbl1q_u8(tbl_h[2], vandq_u8(va.val[1], loset)));
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../gf-complete/gf_w16/../gf_complete.h:40,
                 from ../gf-complete/gf_w16/../gf_int.h:14,
                 from ../gf-complete/gf_w16/shuffle128_neon.c:44:
/usr/lib/gcc/arm-linux-gnueabihf/8/include/arm_neon.h:14044:38: note: expected ‘uint8x16_t’ but argument is of type ‘int’
 veorq_u8 (uint8x16_t __a, uint8x16_t __b)
                           ~~~~~~~~~~~^~~
../gf-complete/gf_w16/shuffle128_neon.c:187:25: error: incompatible type for argument 2 of ‘veorq_u8’
       rl = veorq_u8(rl, vqtbl1q_u8(tbl_l[1], va.val[0]));
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../gf-complete/gf_w16/../gf_complete.h:40,
                 from ../gf-complete/gf_w16/../gf_int.h:14,
                 from ../gf-complete/gf_w16/shuffle128_neon.c:44:
/usr/lib/gcc/arm-linux-gnueabihf/8/include/arm_neon.h:14044:38: note: expected ‘uint8x16_t’ but argument is of type ‘int’
 veorq_u8 (uint8x16_t __a, uint8x16_t __b)
                           ~~~~~~~~~~~^~~
../gf-complete/gf_w16/shuffle128_neon.c:188:25: error: incompatible type for argument 2 of ‘veorq_u8’
       rh = veorq_u8(rh, vqtbl1q_u8(tbl_h[1], va.val[0]));
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../gf-complete/gf_w16/../gf_complete.h:40,
                 from ../gf-complete/gf_w16/../gf_int.h:14,
                 from ../gf-complete/gf_w16/shuffle128_neon.c:44:
/usr/lib/gcc/arm-linux-gnueabihf/8/include/arm_neon.h:14044:38: note: expected ‘uint8x16_t’ but argument is of type ‘int’
 veorq_u8 (uint8x16_t __a, uint8x16_t __b)
                           ~~~~~~~~~~~^~~
../gf-complete/gf_w16/shuffle128_neon.c:189:32: error: incompatible type for argument 2 of ‘veorq_u8’
       va.val[0] = veorq_u8(rl, vqtbl1q_u8(tbl_l[3], va.val[1]));
                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../gf-complete/gf_w16/../gf_complete.h:40,
                 from ../gf-complete/gf_w16/../gf_int.h:14,
                 from ../gf-complete/gf_w16/shuffle128_neon.c:44:
/usr/lib/gcc/arm-linux-gnueabihf/8/include/arm_neon.h:14044:38: note: expected ‘uint8x16_t’ but argument is of type ‘int’
 veorq_u8 (uint8x16_t __a, uint8x16_t __b)
                           ~~~~~~~~~~~^~~
../gf-complete/gf_w16/shuffle128_neon.c:190:32: error: incompatible type for argument 2 of ‘veorq_u8’
       va.val[1] = veorq_u8(rh, vqtbl1q_u8(tbl_h[3], va.val[1]));
                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../gf-complete/gf_w16/../gf_complete.h:40,
                 from ../gf-complete/gf_w16/../gf_int.h:14,
                 from ../gf-complete/gf_w16/shuffle128_neon.c:44:
/usr/lib/gcc/arm-linux-gnueabihf/8/include/arm_neon.h:14044:38: note: expected ‘uint8x16_t’ but argument is of type ‘int’
 veorq_u8 (uint8x16_t __a, uint8x16_t __b)
                           ~~~~~~~~~~~^~~
make: *** [gf-complete.target.mk:115: Release/obj.target/gf-complete/gf-complete/gf_w16/shuffle128_neon.o] Error 1
make: Leaving directory '/home/pi/apps/ParPar/build'
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/share/node-gyp/lib/build.js:262:23)
gyp ERR! stack     at ChildProcess.emit (events.js:189:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
gyp ERR! System Linux 4.19.75-v7l+
gyp ERR! command "/usr/bin/node" "/usr/bin/node-gyp" "rebuild"
gyp ERR! cwd /home/pi/apps/ParPar
gyp ERR! node -v v10.15.2
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok
animetosho commented 4 years ago

Thanks for the report. Interesting that it doesn't happen on my system.

Could you try the current Git version to see if it builds now?

mbruel commented 4 years ago

Working like a charm. Thanks! I would have thought RPI arch would be to slow to use Parpar but it's alright.

pi@raspberrypi:~/ngPost $ time ~/apps/ParPar/bin/parpar.js -s1M -r8% -o test.par2 file1 file2 
Multiply method used: Shuffle (64 bit), 4 threads
Generating 94 MiB recovery data (94 slices) from 1172.77 MiB of data
Calculating: 100.00%
PAR2 created. Time taken: 57.365 second(s)

real    1m1.066s
user    2m24.232s
sys 0m3.319s

with only 3 cores available as I'm packaging ParPar and thus building nodejs at the same time. building node is bloody slow...

animetosho commented 4 years ago

Thanks for confirming!
Cortex A72 isn't that bad of a core, in terms of performance. You may find performance isn't as good on earlier RPi models.

Note that I've just noticed a serious bug in ARM64, which I'm looking to fix soon. I'd recommend not building anything for ARM64 prior to v0.3.2.

animetosho commented 4 years ago

Note that I've just noticed a serious bug in ARM64, which I'm looking to fix soon. I'd recommend not building anything for ARM64 prior to v0.3.2.

I've been investigating this issue for a while now. I believe it's actually a hardware/power fault causing the output corruption I noticed, so the above would likely be a false alarm, and code should be fine.