search

concurrencykit / ck

Concurrency primitives, safe memory reclamation mechanisms and non-blocking (including lock-free) data structures designed to aid in the research, design and implementation of high performance concurrent systems developed in C99+.
http://concurrencykit.org/
Other
2.38k stars 313 forks source link

ck tests fail if default gcc/ck_pr.h is used on MIPS64 #38

Closed juricast closed 8 years ago

juricast commented 9 years ago

While building package ck from latest source on MIPS64 (little endian) we have got following errors:

----[ Testing bytelock....
make[2]: Entering directory '/home/jurica/SBUILD/ck-mips64/NEW_GIT/git/ck/regressions/ck_bytelock/validate'
./validate 16 1
Creating threads (mutual exclusion)...done
Waiting for threads to finish correctness regression...ERROR [RD:113]: 3 != 0
Makefile:11: recipe for target 'check' failed
make[2]: *** [check] Error 1
make[2]: Leaving directory '/home/jurica/SBUILD/ck-mips64/NEW_GIT/git/ck/regressions/ck_bytelock/validate'

----[ Testing fifo....
make[2]: Entering directory '/home/jurica/SBUILD/ck-mips64/NEW_GIT/git/ck/regressions/ck_fifo/validate'
./ck_fifo_spsc 16 1 64000
T [3:0xffec014a70] 4 != 2
Makefile:8: recipe for target 'check' failed
make[2]: *** [check] Error 1
make[2]: Leaving directory '/home/jurica/SBUILD/ck-mips64/NEW_GIT/git/ck/regressions/ck_fifo/validate'

----[ Testing ring....
make[2]: Entering directory '/home/jurica/SBUILD/ck-mips64/NEW_GIT/git/ck/regressions/ck_ring/validate'
./ck_ring_spsc 16 1 16384
Makefile:9: recipe for target 'check' failed
make[2]: *** [check] Segmentation fault
make[2]: Leaving directory '/home/jurica/SBUILD/ck-mips64/NEW_GIT/git/ck/regressions/ck_ring/validate'

----[ Testing sequence....
make[2]: Entering directory '/home/jurica/SBUILD/ck-mips64/NEW_GIT/git/ck/regressions/ck_sequence/validate'
./ck_sequence 16 1
ERROR: Failed regression: copy->c (13064 != 6030 + 7030 / 13060)
Makefile:11: recipe for target 'check' failed
make[2]: *** [check] Error 1
make[2]: Leaving directory '/home/jurica/SBUILD/ck-mips64/NEW_GIT/git/ck/regressions/ck_sequence/validate'

Those failures does not occur on amd64 when gcc/ck_pr.h is used.

I do understand that gcc/ck_pr.h is only meant to serve as a lay-over until a native port is developed. I would like, if possible, to have working lay-over before native port is done.

Later this package could be ported to different 32bit and 64bit MIPS architectures.

Any help is welcomed!

Previous patch: https://github.com/concurrencykit/ck/issues/35 did resolve previous problem on MIPS64 also.

Thank you!

Regards, Jurica

sbahra commented 9 years ago

Interesting. Any chance we can get objdump -d output of the failed tests?

juricast commented 9 years ago

bytelock - validate https://pastee.org/36m6w

fifo - ck_fifo_spsc https://pastee.org/sjubc

ring - ck_ring_spsc https://pastee.org/xq65y

sequence - ck_sequence https://pastee.org/rbkhj

Regards, Jurica

sbahra commented 9 years ago

jurica, any way we could get access to a MIPS64 box? Perhaps we'll also port CK to it.

juricast commented 9 years ago

Sorry,

I have asked for MIPS64 box access. I was told that there is no publicly available MIPS64 box right now. If it does help, I can test changes (debug) on box that I have access to and send you a report.

There's also QEMU (which will work with 64-bit as well): http://mipsdebian.imgtec.com/debian/qemu/

If it helps maybe we could get access to one CI20 board (MIPS32 little endian board)? I will ask if there is CI20 access if you think that this could help package build on MIPS.

Regards, Jurica

cognet commented 9 years ago

I tried to use qemu to reproduce this, but was unsuccessful in getting linux booting on an SMP VM, neither 32 nor 64bits, and the problem won't happen on UP, of course. Can you reproduce the problem on the 32bits board as well ?

Regards,

Olivier

juricast commented 9 years ago

There was failiures on MIPS32 as well. I will check again and post results here.

There is additional problem on MIPS32. MIPS32 does not support 8B __sync operations. I have to use libatomic to resolve this.

Regards, Jurica

cognet commented 9 years ago

I don't know MIPS well, but it seems it doesn't support 64BITS ll/sc for 32bits systems, so we're kinda out of luck here. I don't think libatomic is an option for us, as it seems to use pthread for what is not supported natively by gcc.

Regards,

Olivier

juricast commented 9 years ago

Yes, you are right. MIPS32 does not have lld/scd implemented. I was looking at ck_f_pr.h and ck_pr.h Could we avoid using 8B operations on MIPS32?

Regards, Jurica

cognet commented 9 years ago

Sure, we just have to provide a separate ck_f_pr.h for it, without the various _64 defines.

Regards,

Olivier

juricast commented 9 years ago

Hello,

I did rerun package on MIPS32 (on same box). More tests fail on MIPS32.

I would like to recheck results on other build machine. I will be back with results.

Regards, Jurica

sbahra commented 9 years ago

The GCC fallback relies on GCC to do the right thing juricast. I think the best route here is to develop a native port to MIPS, which should be fairly straight-forward. As far as MIPS64 is concerned, we still need to do further investigation. However, we still don't have access to a multicore MIPS box.

MIPS32 failures are expected if GCC is not emitting the right sync instruction on there.

I didn't see anything obvious in the disassembly you gave me, for MIPS64. Could you compile the tests with -O0 and try again and give us disassembly of ck_sequence assuming it still fails?

juricast commented 9 years ago

I have excluded functionality that require 8B __sync for MIPS32. Package still fail. Interesting is that on different MIPS box package fails but less tests fail. That box has less cores and does not have FPU. And I was able to once completely build package without failure on loongson box.

I need to look at this more, and I will provide disassembly from tests compiled with -O0 shortly.

I suppose that you are right and it would be nice to have MIPS port for this package.

Regards, Jurica

sbahra commented 9 years ago

@juricast - could you also paste us your include/ck_md.h file (post-configure) for both MIPS32 and MIPS64?

juricast commented 9 years ago

Hello,

I have tried to build upstream source on different MIPS boards, with -O0 and with -O2.

On Loongson based board with 4 threads package builds successfully on MIPS32 and MIPS64. For MIPS32 I have removed _64 defines from gcc/ck_pr.h and gcc/ck_f_pr.h

On box with 16 threads (NetLogic-XLP) results are inconsistent. Make check takes a lot of time to execute. If there are no other tasks on this box package sometimes execute tests successfully. When package build fails, if I independently run tests that have failed previously, they execute successfully.

include/ck_md.h for MIPS32:

#ifndef CK_MD_H
#define CK_MD_H
#ifndef CK_MD_CACHELINE
#define CK_MD_CACHELINE (64)
#endif
#ifndef CK_MD_PAGESIZE
#define CK_MD_PAGESIZE (4096)
#endif
#ifndef CK_MD_RTM_DISABLE
#define CK_MD_RTM_DISABLE
#endif /* CK_MD_RTM_DISABLE */
#ifndef CK_MD_POINTER_PACK_DISABLE
#define CK_MD_POINTER_PACK_DISABLE
#endif /* CK_MD_POINTER_PACK_DISABLE */
#ifndef CK_MD_VMA_BITS_UNKNOWN
#define CK_MD_VMA_BITS_UNKNOWN
#endif /* CK_MD_VMA_BITS_UNKNOWN */
#ifndef CK_MD_TSO
#define CK_MD_TSO
#endif /* CK_MD_TSO */
#endif /* CK_MD_H */

include/ck_md.h for MIPS64:

#ifndef CK_MD_H
#define CK_MD_H
#ifndef CK_MD_CACHELINE
#define CK_MD_CACHELINE (64)
#endif
#ifndef CK_MD_PAGESIZE
#define CK_MD_PAGESIZE (4096)
#endif
#ifndef CK_MD_RTM_DISABLE
#define CK_MD_RTM_DISABLE
#endif /* CK_MD_RTM_DISABLE */
#ifndef CK_MD_POINTER_PACK_DISABLE
#define CK_MD_POINTER_PACK_DISABLE
#endif /* CK_MD_POINTER_PACK_DISABLE */
#ifndef CK_MD_VMA_BITS_UNKNOWN 
#define CK_MD_VMA_BITS_UNKNOWN 
#endif /* CK_MD_VMA_BITS_UNKNOWN */
#ifndef CK_MD_TSO
#define CK_MD_TSO
#endif /* CK_MD_TSO */
#endif /* CK_MD_H */

Regards, Jurica

sbahra commented 9 years ago

Ok, this is likely your problem. Try changing the CK_MD_TSO to CK_MD_RMO (don't re-run configure) and then try make clean. Does it pass now? Alternatively, pass --memory-model=rmo to configure.

sbahra commented 9 years ago

@juricast - Have you had a chance to test this yet?

juricast commented 9 years ago

Yes I did. Sorry for taking so long.

I still have problems for MIPS32 on box with 16 threads. For MIPS64 it seams to be okay (with --memory-model=rmo and -O0).

Thank you!

Regards, Jurica

sbahra commented 9 years ago

Terrific. -O1 should work as well. How many cores are on the MIPS32 box? When you say problem, do you mean tests take forever to run / appear to hang or failures? The 8b/4b sync isn't addressed yet.

juricast commented 9 years ago

MIPS32 16 threads is failing following tests now (with --memory-model=rmo and -O0):

----[ Testing epoch....
make[2]: Entering directory '/build/git/ck/regressions/ck_epoch/validate'
./ck_stack 16 1
Makefile:9: recipe for target 'check' failed
make[2]: *** [check] Killed
make[2]: Leaving directory '/build/git/ck/regressions/ck_epoch/validate'
----[ Testing hp....
make[2]: Entering directory '/build/git/ck/regressions/ck_hp/validate'
./serial
Free 0x7f0018
Free 0x7f0018
Free 0x7f0018
Allocating entry and freeing in other HP record...
Free 0x7f0018
Free 0x7f0040
./ck_hp_stack 16 100 1
Makefile:8: recipe for target 'check' failed
make[2]: *** [check] Killed
make[2]: Leaving directory '/build/git/ck/regressions/ck_hp/validate'

I will look into it again. For MIPS32 I have just removed _64 defines from gcc/ck_pr.h and gcc/ck_f_pr.h

sbahra commented 9 years ago

It seems like OOM killer is kicking in here.

sbahra commented 8 years ago

CK_MD_RMO should have been used, folks can specify memory model with --memory-model=X.