search

nim-lang / Nim

Nim is a statically typed compiled systems programming language. It combines successful concepts from mature languages like Python, Ada and Modula. Its design focuses on efficiency, expressiveness, and elegance (in that order of priority).
https://nim-lang.org
Other
16.47k stars 1.47k forks source link

Nim compiler looses track of sub-processes on Linux running MinGW #5796

Open mjfh opened 7 years ago

mjfh commented 7 years ago

This is a follow up from the NIM forum trying to avoid polluting it with listings. This is my environment

jordan@black:~/Desktop/problem$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 9.0 (stretch)
Release:    9.0
Codename:   stretch

jordan@black:~/Desktop/problem$ i686-w64-mingw32-gcc --version
i686-w64-mingw32-gcc (GCC) 6.3.0 20170425
[..]

jordan@black:~/Desktop/problem$ nim --version
Nim Compiler Version 0.16.1 (2017-05-09) [Linux: amd64]
[..]
git hash: fa3436fb657141127038d88431b4aad113c27cf6
active boot switches: -d:release

jordan@black:~/Desktop/problem$ cat nim.cfg 
cpu = i386
i386.windows.gcc.path = "/usr/bin"
i386.windows.gcc.exe = "i686-w64-mingw32-gcc"
i386.windows.gcc.linkerexe = "i686-w64-mingw32-gcc"

The problem is reproducible on my system when I dispatch NIM with a clean environment as in

env -i PATH=/usr/local/bin:/usr/bin:/bin nim cc --os:windows --verbosity:3 --listCmd problem.nim

where the source looks like

# nim test file: problem.nim

# C source equivalent of:
#  #define WIN32_LEAN_AND_MEAN
#  #include <windows.h>
#  #include <wincrypt.h>
#  printf(">>>  PROV_RSA_FULL=%d\n", PROV_RSA_FULL);

{.passC: "-DWIN32_LEAN_AND_MEAN".}
var PROV_RSA_FULL {.importc,
                    header: "<windows.h>",
                    header: "<wincrypt.h>".}: int

echo ">>>  PROV_RSA_FULL=", PROV_RSA_FULL

The NIM compiler just hangs at

[..]
problem.nim(8, 6) Hint: 104005 [Processing]
/usr/bin/i686-w64-mingw32-gcc -c  -w  [..] /home/jordan/Desktop/problem/nimcache/problem.c
/usr/bin/i686-w64-mingw32-gcc -c  -w  [..] /home/jordan/Desktop/problem/nimcache/stdlib_system.c

which results in a a process table like (some line feeds added)

jordan@black:~/Desktop/problem$ ps xafe
[..]
11369 pts/12   S+     0:00  |   \_ /usr/local/share/nim/bin/nim cc --os:windows --verbosity:3 --listCmd problem.nim
                                       PATH=/usr/local/share/nim/bin:/usr/local/bin:/usr/bin:/bin
                                       PWD=/home/jordan/Desktop/problem
11372 pts/12   S+     0:00  |       \_ /bin/sh -c /usr/bin/i686-w64-mingw32-gcc -c  -w -DWIN32_LEAN_AND_MEAN  -I/usr/local/share/nim/lib -o /home/jordan/Desktop/problem/nimcache/problem.o /home/jordan/Desktop/problem/nimcache/problem.c
                                           PATH=/usr/local/share/nim/bin:/usr/local/bin:/usr/bin:/bin
                                           PWD=/home/jordan/Desktop/problem
11374 pts/12   S+     0:00  |       |   \_ /usr/bin/i686-w64-mingw32-gcc -c -w -DWIN32_LEAN_AND_MEAN -I/usr/local/share/nim/lib -o /home/jordan/Desktop/problem/nimcache/problem.o /home/jordan/Desktop/problem/nimcache/problem.c
                                               PATH=/usr/local/share/nim/bin:/usr/local/bin:/usr/bin:/bin
                                               PWD=/home/jordan/Desktop/problem
11376 pts/12   S+     0:00  |       |       \_ /usr/lib/gcc/i686-w64-mingw32/6.3-win32/cc1 -quiet -I /usr/local/share/nim/lib -U_REENTRANT -D WIN32_LEAN_AND_MEAN /home/jordan/Desktop/problem/nimcache/problem.c -quiet -dumpbase problem.c -mtune=generic -march=pentiumpro -auxbase-strip /home/jordan/Desktop/problem/nimcache/problem.o -w -o /tmp/ccMA30uS.s
                                                   PATH=/usr/local/share/nim/bin:/usr/local/bin:/usr/bin:/bin
                                                   PWD=/home/jordan/Desktop/problem COLLECT_GCC=/usr/bin/i686-w64-mingw32-gcc
                                                   COLLECT_GCC_OPTIONS='-c' '-w' '-D' 'WIN32_LEAN_AND_MEAN' '-I' '/usr/local/share/nim/lib' '-o' '/home/jordan/Desktop/problem/nimcache/problem.o' '-mtune=generic' '-march=pentiumpro'
11373 pts/12   Z+     0:00  |       \_ [sh] <defunct>
[..]

Just to make sure the context is all right I have a look at the cgroups

jordan@black:~/Desktop/problem$ systemd-cgls
Control group /:
-.slice
[..]
├─user.slice
│ └─user-1000.slice
│   ├─session-2.scope
│   │ ├─ 3044 lightdm --session-child 12 21
[..]
│   │ ├─11369 /usr/local/share/nim/bin/nim cc --os:windows --verbosity:3 --li...
│   │ ├─11372 /bin/sh -c /usr/bin/i686-w64-mingw32-gcc -c  -w -DWIN32_LEAN_AN...
│   │ ├─11374 /usr/bin/i686-w64-mingw32-gcc -c -w -DWIN32_LEAN_AND_MEAN -I/us...
│   │ ├─11376 /usr/lib/gcc/i686-w64-mingw32/6.3-win32/cc1 -quiet -I /usr/loca...

Tracing the last MinGW command I get

jordan@black:~/Desktop/problem$ strace -p 11376
strace: Process 11376 attached
write(2, "/usr/share/mingw-w64/include/bcr"..., 367
^Cstrace: Process 11376 detached
 <detached ...>

and the open file streams look like

jordan@black:~/Desktop/problem$ lsof | grep 11376
lsof: WARNING: can't stat() aufs [..]
lsof: WARNING: can't stat() tmpfs [..]
lsof: WARNING: can't stat() nsfs [..]
cc1       11376              jordan  cwd       DIR               0,56     4096    3410577 /home/jordan/Desktop/problem
cc1       11376              jordan  rtd       DIR              253,0     4096          2 /
cc1       11376              jordan  txt       REG              253,7 22042016     940171 /usr/lib/gcc/i686-w64-mingw32/6.3-win32/cc1
cc1       11376              jordan  mem       REG              253,0  1685264      58073 /lib/x86_64-linux-gnu/libc-2.24.so
cc1       11376              jordan  mem       REG              253,0  1063328      58094 /lib/x86_64-linux-gnu/libm-2.24.so
cc1       11376              jordan  mem       REG              253,0   105088      57708 /lib/x86_64-linux-gnu/libz.so.1.2.8
cc1       11376              jordan  mem       REG              253,0    14640      58089 /lib/x86_64-linux-gnu/libdl-2.24.so
cc1       11376              jordan  mem       REG              253,7   537448     131530 /usr/lib/x86_64-linux-gnu/libgmp.so.10.3.2
cc1       11376              jordan  mem       REG              253,7   422200     131324 /usr/lib/x86_64-linux-gnu/libmpfr.so.4.1.5
cc1       11376              jordan  mem       REG              253,7    97736     136991 /usr/lib/x86_64-linux-gnu/libmpc.so.3.0.0
cc1       11376              jordan  mem       REG              253,7  1738240     138496 /usr/lib/x86_64-linux-gnu/libisl.so.15.3.0
cc1       11376              jordan  mem       REG              253,0   153288      57359 /lib/x86_64-linux-gnu/ld-2.24.so
cc1       11376              jordan    0r     FIFO               0,10      0t0    7995392 pipe
cc1       11376              jordan    1w     FIFO               0,10      0t0    8003585 pipe
cc1       11376              jordan    2w     FIFO               0,10      0t0    8003585 pipe
cc1       11376              jordan    3r     FIFO               0,10      0t0    7995392 pipe
cc1       11376              jordan    4w      REG               0,37        0    7996413 /tmp/ccMA30uS.s
cc1       11376              jordan    5r      REG              253,7   237724     719406 /usr/share/mingw-w64/include/wincrypt.h
cc1       11376              jordan    6w     FIFO               0,10      0t0    8003585 pipe
cc1       11376              jordan    7r      REG              253,7    23915     709325 /usr/share/mingw-w64/include/bcrypt.h
cc1       11376              jordan    8w     FIFO               0,10      0t0    8003586 pipe

All the other commands below the NIM compiler are explicitly waiting for the child (wait4(<pid>,..)) except the shell which waits for any child (wait4(-1,..)).

I have no clue what is happening here.

ghost commented 7 years ago

@mjfh in my project's nim.cfg I'm using this:

# Windows cross-compile via mingw
@if crosswin:
  gcc.linkerexe = "x86_64-w64-mingw32-gcc"
  gcc.exe = "x86_64-w64-mingw32-gcc"
  gcc.path = "/usr/bin"
  gcc.options.linker = ""
  os = "windows"
  define:windows
@end
ghost commented 7 years ago

@mjfh oh, yeah, your example hangs forever on compilation even on my PC with mingw 64bits...

mjfh commented 7 years ago

lol - this was my surprise as well.

Problem is that the MinGW compiler aborts with errors (see the NIM forum thread).

My solution is basically here.

And NIM correctly reports errors on native NimGW but this has a different process mangement. I do not see this annoyance a high priority as it is easily circumvented. It is just a bit annoying. So people should just know about.

Jordan

perpetual-hydrofoil commented 6 years ago

Same issue on Debian. Running command via --listCmd:

gcc -c  -w -mno-ms-bitfields -I/usr/x86_64-w64-mingw32/include/ \
    -static  -I/home/jamieson/nim-0.17.2/lib -o /home/jamieson/nim-0.17.2/nimcache/stdlib_md5.o \
    /home/jamieson/nim-0.17.2/nimcache/stdlib_md5.c'

results in (tail end):


/home/jamieson/nim-0.17.2/nimcache/stdlib_md5.c:1247:1: error: expected ‘{’ at end of input
 }
 ^
In file included from /usr/x86_64-w64-mingw32/include/minwindef.h:146:0,
                 from /usr/x86_64-w64-mingw32/include/windef.h:8,
                 from /usr/x86_64-w64-mingw32/include/windows.h:69,
                 from /home/jamieson/nim-0.17.2/nimcache/stdlib_md5.c:12:
/usr/x86_64-w64-mingw32/include/winnt.h:138:25: note: the ABI of passing struct with a flexible array member has changed in GCC 4.4
 #define DECLSPEC_IMPORT __declspec (dllimport)
                         ^
/usr/x86_64-w64-mingw32/include/winnt.h:251:18: note: in expansion of macro ‘DECLSPEC_IMPORT’
 #define NTSYSAPI DECLSPEC_IMPORT
                  ^
/usr/x86_64-w64-mingw32/include/winnt.h:6895:5: note: in expansion of macro ‘NTSYSAPI’
     NTSYSAPI WORD NTAPI RtlCaptureStackBackTrace (DWORD FramesToSkip, DWORD FramesToCapture, PVOID *BackTrace, PDWORD BackTraceHash);
     ^

Maybe it was just md5, so took out md5; ten it hung on osproc. Also took that out and now strace says it's hanging on 31002.

write(1, "gcc -c  -w -mno-ms-bitfields -st"..., 213gcc -c  -w -mno-ms-bitfields -static -I/usr/x86_64-w64-mingw32/include/  -I/home/jamieson/nim-0.17.2/lib -o /home/jamieson/nim-0.17.2/nimcache/stdlib_unicode.o /home/jamieson/nim-0.17.2/nimcache/stdlib_unicode.c 
) = 213
close(85)                               = 0
close(86)                               = 0
close(3)                                = 0
close(84)                               = 0
wait4(31002, 
jamieson 31002  0.0  0.0   4336   808 pts/5    S+   21:26   0:00                          \_ /bin/sh -c gcc -c  -w -mno-ms-bitfields -static -I/usr/x86_64-w64-mingw32/include/  -I/home/jamieson/nim-0.17.2/lib -o /home/jamieson/nim-0.17.2/nimcache/compiler_shim.o /home/jamieson/nim-0.17.2/nimcache/compiler_shim.c
~ strace -p 31002
Process 31002 attached
wait4(-1,
yatesco commented 6 years ago

Also hangs on macOS:

  1. brew install nim mingw-64
  2. mkdir problem && cd problem
  3. cat problem.nim: 
    
# nim test file: problem.nim

C source equivalent of:

define WIN32_LEAN_AND_MEAN

include

include

printf(">>> PROV_RSA_FULL=%d\n", PROV_RSA_FULL);

{.passC: "-DWIN32_LEAN_AND_MEAN".} var PROV_RSA_FULL {.importc, header: "", header: "".}: int

echo ">>> PROV_RSA_FULL=", PROV_RSA_FULL

4. `cat nim.cfg`:

cpu = i386 i386.windows.gcc.path = "/usr/local/bin/" i386.windows.gcc.exe = "i686-w64-mingw32-gcc" i386.windows.gcc.linkerexe = "i686-w64-mingw32-gcc"


5. `env -i PATH=/usr/local/bin:/usr/bin:/bin nim cc --os:windows --verbosity:3 --listCmd problem.nim` hangs.

Unfortunately, I am not skilled enough in `nim` to even call myself a `nim` newbie! so I haven't the first clue of how to debug this.
dom96 commented 6 years ago

Possibly an osproc issue?

yatesco commented 6 years ago

@dom96 - if that was to me, the fact I don't even know what osproc means should tell you at what level I am at with nim :-). I have spent too many decades being an "enterprise" developer (Java, Spring, Clojure) etc., I am absolutely bowled over with joy as I read about nim, but it is going to take a while before I can contribute.

dom96 commented 6 years ago

@yatesco no worries. I wasn't directing at anyone in particular, just saying it for anyone that wishes to look further into this issue. The osproc module is the likely culprit.

Glad you're enjoying Nim :)