search

rdicosmo / parmap

Parmap is a minimalistic library allowing to exploit multicore architecture for OCaml programs with minimal modifications.
http://rdicosmo.github.io/parmap/
Other
94 stars 20 forks source link

Infinite loop if core pinning fails #103

Open IagoAbal opened 3 years ago

IagoAbal commented 3 years ago

Core pinning does not always work on Apple M1 and as a result Parmap enters an infinite loop. See returntocorp/semgrep#2432 and facebook/infer#1410. We at @returntocorp have only observed this behavior with HomeBrew builds, presumably because in HomeBrew's build environment HAVE_MACH_THREAD_POLICY_H is set? Even then, this only occurs under certain values of ncores and chunksize. So running on 8 files using 8 cores and chunk size of 1 works, but if there are 9 files it hangs. Running on 9 files with 4 cores and a chunk size of 2 it does not hang.

Our current work around is to simply disable core pinning.

Independently of why core pinning is failing in this case, could it make sense to change while (finished==0) { ... } to do { ... } while (finished==0 && w>0) or bound the number of iterations of that loop in some other way ? I am happy to open a PR if we agree on this.

UnixJunkie commented 3 years ago

This file could also benefit from a complete rewrite. A 100% Linux part, another 100% Mac OS X part. Then just including the right one.

You can already send your PR I think.