windows

[JuliaLawall]

A question rather than an issue: Does Parmap work on Windows?

[rdicosmo]

Parmap relies heavily on the properties of the fork system call on *nix systems, which is not present in the standard Windows OS, so as a first answer I would say a clear "no".

The recent evolutions of the Windows OS to support Linux executables natively may be worth checking, see for example https://www.notebookcheck.net/How-to-run-Linux-binaries-natively-in-Windows-10.282676.0.html but unless a fork system call is made available, there is no way to actually have Parmap running natively on Windows without a great deal of reengineering.

On Sat, Sep 01, 2018 at 11:34:45AM -0700, JuliaLawall wrote:

A question rather than an issue: Does Parmap work on Windows?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.*

-- Roberto Di Cosmo

---

Computer Science Professor (on leave at INRIA from IRIF/University Paris Diderot)

Software Heritage E-mail : roberto@dicosmo.org
INRIA Web : http://www.dicosmo.org
Bureau C123 Twitter : http://twitter.com/rdicosmo 2, Rue Simone Iff Tel : +33 1 80 49 44 42 CS 42112 75589 Paris Cedex 12

---

GPG fingerprint 2931 20CE 3A5A 5390 98EC 8BFC FCCA C3BE 39CB 12D3

[JuliaLawall]

On Sat, 1 Sep 2018, Roberto Di Cosmo wrote:

Parmap relies heavily on the properties of the fork system call on *nix systems, which is not present in the standard Windows OS, so as a first answer I would say a clear "no".

The recent evolutions of the Windows OS to support Linux executables natively may be worth checking, see for example https://www.notebookcheck.net/How-to-run-Linux-binaries-natively-in-Windows -10.282676.0.html but unless a fork system call is made available, there is no way to actually have Parmap running natively on Windows without a great deal of reengineering.

Thanks for the information!

julia

On Sat, Sep 01, 2018 at 11:34:45AM -0700, JuliaLawall wrote:

A question rather than an issue: Does Parmap work on Windows?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.*

-- Roberto Di Cosmo

---

Computer Science Professor (on leave at INRIA from IRIF/University Paris Diderot)

Software Heritage E-mail : roberto@dicosmo.org INRIA Web : http://www.dicosmo.org Bureau C123 Twitter : http://twitter.com/rdicosmo 2, Rue Simone Iff Tel : +33 1 80 49 44 42 CS 42112 75589 Paris Cedex 12

GPG fingerprint 2931 20CE 3A5A 5390 98EC 8BFC FCCA C3BE 39CB 12D3

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.[AAesmo9rC4c2Snfki2zeETzIi1VEVjfrks5uWty2gaJpZM4WWU0s.gif]

[UnixJunkie]

I think there are other ocaml parallelization libraries said to work on windows, though I cannot find back the name.

[UnixJunkie]

https://github.com/cryptosense/procord

I think this library was made to work on windows.

[fasiha]

I'd like to use parmap on Windows with Cygwin, via OCaml for Windows, which implements fork —my ulterior goal being to get another program, which depends on parmap, to run on Windows.

When I run opam install parmap in Cygwin, I see the following error:

$ opam install parmap
[ERROR] parmap unmet availability conditions, e.g. os != "win32"

My ignorance of OCaml conventions will show but I'd like to understand where in the parmap project configuration an OS requirement is encoded? I checked

• dune-project and
• parmap.opam

but neither of these seem to say anything OS-specific?

I can install all four of parmap's dependencies:

$ opam install dune-configurator dune base-bigarray base-unix
[NOTE] Package base-unix is already installed (current version is base).
[NOTE] Package base-bigarray is already installed (current version is base).
[NOTE] Package dune is already installed (current version is 2.8.5).
[NOTE] Package dune-configurator is already installed (current version is
       2.8.5).

and the only remaining requirement is "ocaml" {>= "4.02.3"}. I have 4.10.2+mingw64c: I ran

opam switch create . 4.10.2+mingw64c

whose full name seems to be "ocaml-variants.4.10.2+mingw64c". This should satisfy that compiler requirement.

Any tips on what's telling opam that parmap can't be installed in my environment? Thank you 🙏!

(Edit: I'm using the Win10 evaluation image that Microsoft makes available for testing this.)

[UnixJunkie]

Maybe as a first check, check Unix.fork allows you to create two processes on windows. If that's the case, then in principles parmap should work. Personally, since I never use windows, that will be all the help I can provide though.

[fasiha]

@UnixJunkie Ah, I see what you're saying! I'm curious if you have guidance on what might be preventing opam from even installing parmap, since none of its dependencies seem to exclude win32 and since nothing in parmap seems to exclude it.

Is it possible that that error message, parmap unmet availability conditions, e.g. os != "win32", is caused by issues with Unix.fork? It seems that opam stops even before it tries to build anything though.

[UnixJunkie]

No idea where something is saying os must not be win32 (not in parmap's source code apparently). Try to install parmap from a source checkout instead of via opam; maybe you'll know more then.

[fasiha]

Per @mjambon's kind guidance, I cloned parmap directly and tried to build it, and ran into a definite build error:

$ opam install --deps-only parmap
$ git clone https://github.com/rdicosmo/parmap
$ cd parmap
$ make
# ...
dune build @install
x86_64-w64-mingw32-gcc src/setcore_stubs.o (exit 1)
(cd _build/default/src && C:\OCaml64\bin\x86_64-w64-mingw32-gcc.exe -march=x86-64 -mtune=generic -O2 -mms-bitfields -march=x86-64 -mtune=generic -O2 -mms-bitfields -g -I Z:/D
ownloads/todoist/_opam/lib/ocaml -o setcore_stubs.o -c setcore_stubs.c)
setcore_stubs.c: In function ‘numcores’:
setcore_stubs.c:17:18: warning: implicit declaration of function ‘sysconf’; did you mean ‘swscanf’? [-Wimplicit-function-declaration]
   17 |   int numcores = sysconf( _SC_NPROCESSORS_ONLN );
      |                  ^~~~~~~
      |                  swscanf
setcore_stubs.c:17:27: error: ‘_SC_NPROCESSORS_ONLN’ undeclared (first use in this function)
   17 |   int numcores = sysconf( _SC_NPROCESSORS_ONLN );
      |                           ^~~~~~~~~~~~~~~~~~~~
setcore_stubs.c:17:27: note: each undeclared identifier is reported only once for each function it appears in
setcore_stubs.c: In function ‘setcore’:
setcore_stubs.c:22:27: error: ‘_SC_NPROCESSORS_ONLN’ undeclared (first use in this function)
   22 |   int numcores = sysconf( _SC_NPROCESSORS_ONLN );
      |                           ^~~~~~~~~~~~~~~~~~~~
make: *** [Makefile:9: default] Error 1

I see this Stack Overflow question about this very snippet, but in MinGW, not Cygwin. I will continue debugging further but wanted to post this finding. Apologies for the noise I'm causing to subscribed folks!

[UnixJunkie]

The setcore thing is an optimization related to core pinning; parmap should still be able to run fine without it (on not too dumb an OS). If you create a version of parmap where the setcore thing has been removed, you should be able to compile parmap further.

[fasiha]

Thanks @UnixJunkie for your support and encouragement! The error above happens because MinGW (and thus OCaml for Windows) does not provide sysconf or _SC_NPROCESSORS_ONLN in its unistd.h. Only full Cygwin gcc does: parmap will build fine on Windows with full Cygwin (depending on cygwin1.dll). Sorry, I was unclear on this distinction!

[UnixJunkie]

If you manage to make parmap work on windows, please do share about the accelerations you observe. Then, other fork-based libraries (e.g. parany) might also benefit from your discoveries.

On Wed, Apr 21, 2021 at 5:20 AM Ahmed Fasih @.***> wrote:

Thanks @UnixJunkie https://github.com/UnixJunkie for your support and encouragement! The error above happens because MinGW (and thus OCaml for Windows) does not provide sysconf or _SC_NPROCESSORS_ONLN in its unistd.h. Only full Cygwin gcc does: parmap will build fine on Windows with full Cygwin (depending on cygwin1.dll). Sorry, I was unclear on this distinction!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rdicosmo/parmap/issues/76#issuecomment-823574144, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFUFAFMSEYALXVEMWZVF23TJXOXBANCNFSM4FSZJUWA .