GRAPE fails to load in 64-bit Windows (ok in 32-bit)

[ignatsoroko]

Observed behaviour

 ┌───────┐   GAP 4.8.7, 24-Mar-2017, build of 2017-03-24 22:12:47 (GMTST)
 │  GAP  │   https://www.gap-system.org
 └───────┘   Architecture: **x86_64-unknown-cygwin-gcc-default64**
 Libs used:  gmp, readline
 Loading the library and packages ...
 Components: trans 1.0, prim 2.1, small* 1.0, id* 1.0
 Packages:   AClib 1.2, Alnuth 3.0.0, AtlasRep 1.5.1, AutPGrp 1.8, Browse 1.8.6, CRISP 1.4.4, Cryst 4.1.12,
             CrystCat 1.1.6, CTblLib 1.2.2, FactInt 1.5.4, FGA 1.3.1, GAPDoc 1.5.1, IO 4.4.6, IRREDSOL 1.3.1,
             LAGUNA 3.7.0, Polenta 1.3.7, Polycyclic 2.11, RadiRoot 2.7, ResClasses 4.6.0, Sophus 1.23, SpinSym 1.5,
             TomLib 1.2.6, Utils 0.46
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'
gap> LoadPackage("grape");
Error, no method found! For debugging hints type ?Recovery from NoMethodFound
Error, no 1st choice method found for `Length' on 1 arguments called from
Length( string ) at /proc/cygdrive/C/gap4r8_x86_64/lib/string.gi:738 called from
StartsWith( name, s ) at /proc/cygdrive/C/gap4r8_x86_64/lib/files.gi:146 called from
func( elm ) at /proc/cygdrive/C/gap4r8_x86_64/lib/list.gi:2628 called from
First( [ "/proc/cygdrive/", "/cygdrive/" ], function ( s )
      return StartsWith( name, s );
  end ) at /proc/cygdrive/C/gap4r8_x86_64/lib/files.gi:146 called from
MakeExternalFilename( Filename( d, s ) ) at /proc/cygdrive/C/gap4r8_x86_64/lib/files.gi:168 called from
...  at line 45 of /proc/cygdrive/C/gap4r8_x86_64/pkg/grape/lib/grape.g
you can 'quit;' to quit to outer loop, or
you can 'return;' to continue
brk>

Expected behaviour

 ┌───────┐   GAP 4.8.7, 24-Mar-2017, build of 2017-03-24 22:43:07 (GMTST)
 │  GAP  │   https://www.gap-system.org
 └───────┘   Architecture: **i686-pc-cygwin-gcc-default32**
 Libs used:  gmp, readline
 Loading the library and packages ...
 Components: trans 1.0, prim 2.1, small* 1.0, id* 1.0
 Packages:   AClib 1.2, Alnuth 3.0.0, AtlasRep 1.5.1, AutPGrp 1.8, Browse 1.8.6, CRISP 1.4.4, Cryst 4.1.12,
             CrystCat 1.1.6, CTblLib 1.2.2, FactInt 1.5.4, FGA 1.3.1, GAPDoc 1.5.1, IO 4.4.6, IRREDSOL 1.3.1,
             LAGUNA 3.7.0, Polenta 1.3.7, Polycyclic 2.11, RadiRoot 2.7, ResClasses 4.6.0, Sophus 1.23, SpinSym 1.5,
             TomLib 1.2.6, Utils 0.46
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'
gap> LoadPackage("grape");
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loading  GRAPE 4.7 (GRaph Algorithms using PErmutation groups)
by Leonard H. Soicher (http://www.maths.qmul.ac.uk/~leonard/).
Homepage: http://www.maths.qmul.ac.uk/~leonard/grape/
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
true
gap>

Comment

It is written in the documentation that only 32-bit windows binaries of nauty and drednaut are supplied with the GRAPE installation, however the issue above seems to be with file names, not binaries.

Edit (07/22/17): Temporary workaround:

Assume that 64-bit GAP is installed into folder c:\gap4r8_x86_64\ and GRAPE is in c:\gap4r8_x86_64\pkg\grape\. Assume also that 32-bit GAP is in c:\gap4r8\.

1. Locate the folder c:\gap4r8_x86_64\pkg\grape\bin\x86_64-unknown-cygwin-gcc-default64\ (it is empty by default)

2. Copy file c:\gap4r8_x86_64\pkg\grape\bin\i686-pc-cygwin-gcc-default32\dreadnautB.exe to c:\gap4r8_x86_64\pkg\grape\bin\x86_64-unknown-cygwin-gcc-default64\

3. Get the version of cygwin1.dll from the 32-bit instatllation of GAP: c:\gap4r8\bin\i686-pc-cygwin-gcc-default32\cygwin1.dll and copy it to the folder above: c:\gap4r8_x86_64\pkg\grape\bin\x86_64-unknown-cygwin-gcc-default64\

After that, GRAPE will load and work in 64-bit version of GAP.

[james-d-mitchell]

This is something that should be reported to the author of GRAPE (Leonard Soicher) since this is an issue with the GRAPE package and not strictly speaking an issue with GAP itself. I don't think GRAPE has an issue tracker, so you might want to email Leonard directly.

You might also try using the Digraphs package, which ought to work on 64-bit Windows.

[ignatsoroko]

Ok, I will send him an email. It seemed to me like a system issue, not package per se, but I may be mistaken.

[james-d-mitchell]

You might be right, maybe someone else will comment on this, but anyway pointing this out to Leonard is probably a good idea.

[markuspf]

This could well be an issue with MakeExternalFilename.

[fingolfin]

Actually, a likely guess is that an invalid filename was passed to ExternalFilename. It then does Filename(d,s), which (for invalid input) returns fail; and then MakeExternalFilename treats this like a string, resulting in this error.

So, on the one hand, I think MakeExternalFilename could include a check for whether its input is a string.

On the other hand, this is most likely the code in grape triggering the error:

GRAPE_DREADNAUT_EXE := 
   ExternalFilename(DirectoriesPackagePrograms("grape"),"dreadnautB"); 

So... I would hazard to guess that dreadnaut was not compiled.

[fingolfin]

@ignatsoroko how did you install GAP resp. the grape package?

[fingolfin]

Aha, you write: "... only 32-bit windows binaries of nauty and drednaut are supplied ..." -- OK, we can work with that: Actually, it doesn't matter to GAP whether nauty and dreadnaut are 32 or 64 bit. So in principle, it should be possible to use grape either way.

So, you could try this workaround: Change into the grape directory, and there into the bin subdirectory. Create a new directory i686-pc-cygwin-gcc-default64if it does not already exist. Then, copy the content of the i686-pc-cygwin-gcc-default32 directory into it. Try again. Does that help?

[ignatsoroko]

I installed both 64- and 32-bit versions of GAP with packages as suggested on GAP webpage: by downloading .exe file and running it in default configuration.

The directory grape/bin directory contains two folders: i686-pc-cygwin-gcc-default32 x86_64-unknown-cygwin-gcc-default64

The first one contains file drednautB.exe while the second one is empty. After I copied this .exe file into the second one, the error above disappeared! Now it loads properly. So that "fixes" the bad filename issue.

However, running the test resulted in errors for all methods that use drednaut:

gap> Test(Filename(DirectoriesPackageLibrary("grape","tst"),"testall.tst")); ########> Diff in /proc/cygdrive/C/gap4r8_x86_64/pkg/grape/tst/testall.tst, line 212:

Input is:

aut := AutGroupGraph(gamma);;

Expected output:

But found:

Error, cannot find output produced by `dreadnaut' ######## ########> Diff in /proc/cygdrive/C/gap4r8_x86_64/pkg/grape/tst/testall.tst, line 215:

Input is:

Size(aut);

Expected output:

48

But found:

Error, Variable: 'aut' must have a value ######## ########> Diff in /proc/cygdrive/C/gap4r8_x86_64/pkg/grape/tst/testall.tst, line 217:

Input is:

delta := NewGroupGraph( aut, gamma );;

Expected output:

But found:

Error, Variable: 'aut' must have a value ######## ... [many more lines]

[olexandr-konovalov]

As I said at http://mail.gap-system.org/pipermail/forum/2016/005278.html,

Grape uses a stand-alone binary, so it can interact with 64-bit GAP version
even being 32-bit application. The problem is that GAP looks for the binary
for GRAPE using the architecture-dependent path which it assumes to be
'x86_64-unknown-cygwin-gcc-default64'. As a quick workaround, I suggest to
try this: in the 'bin' directory of your GRAPE installation, you should
have the directory 'i686-pc-cygwin-gcc-default32'. Copy it to the directory
'x86_64-unknown-cygwin-gcc-default64' and try to load GRAPE from 64-bit 
version of GAP now.

@ignatsoroko is Test(Filename(DirectoriesPackageLibrary("grape","tst"),"testall.tst")); failing in 64-bit or in 32-bit GAP, or in both?

[ignatsoroko]

@ignatsoroko is Test(Filename(DirectoriesPackageLibrary("grape","tst"),"testall.tst")); failing in 64-bit or in 32-bit GAP, or in both?

It fails only in 64-bit GAP, and only in methods involving dreadnaut. The 32-bit version works fine. I am attaching the log file of a 64-bit session. grapetest.log.txt

[ignatsoroko]

Ok, I think I figured it out.

First, I tried to run: c:\gap4r8_x86_64\pkg\grape\bin\i686-pc-cygwin-gcc-default32\dreadnautB.exe which resulted in opening a window with a message Dreadnaut version 2.2 (32 bits) [BIG]. and a prompt, which looks like a successful run.

Then I tried to run c:\gap4r8_x86_64\pkg\grape\bin\x86_64-unknown-cygwin-gcc-default64\dreadnautB.exe which resulted in an error message saying that file cygwin1.dll cannot be found. So I guessed that the file drednautB.exe is searching for a specific location for cygwin1.dll and cannot find it there.

So I copied the file c:\gap4r8_x86_64\bin\x86_64-unknown-cygwin-gcc-default64\cygwin1.dll to c:\gap4r8_x86_64\pkg\grape\bin\x86_64-unknown-cygwin-gcc-default64\ location and tried to run drednautB.exe again, but this time another error occurred: The application was unable to start correctly (0xc0000007b). Click OK to close the application.

My next guess was that it needs a version of cygwin1.dll which is compatible with the one it was compiled with, i.e. a 32-bit version one. And indeed, when I copied c:\gap4r8\bin\i686-pc-cygwin-gcc-default32\cygwin1.dll to c:\gap4r8_x86_64\pkg\grape\bin\x86_64-unknown-cygwin-gcc-default64\ the file drednautB.exe was able to run smoothly, and moreover, GRAPE in 64-bit GAP loaded fine, and all tests passed with 'true' result.

[ignatsoroko]

I edited the original post above and added the workaround.

[fingolfin]

@alex-konovalov @ChrisJefferson do you know if this issue still relevant in GAP 4.10? Don't we know provide a 64bit binary on Windows, too?

[ignatsoroko]

@alex-konovalov @ChrisJefferson do you know if this issue still relevant in GAP 4.10? Don't we know provide a 64bit binary on Windows, too?

Yes, this issue is still there, in slightly different form. Namely, the package loads with a true result, but the tests fail, please see the log file attached. My workaround above needs modification:

To enable grape in 64-bit Windows, one must

1) install the 32-bit version of GAP in addition to the 64-bit one; 2) locate folder c:\gap-4.10.0-x86_64\pkg\grape-4.8.1\bin\x86_64-unknown-cygwin-default64\ (it is empty by default); 3) copy into it two files from the 32-bit version: c:\gap-4.10.0\pkg\grape-4.8.1\bin\i686-pc-cygwin-default32\dreadnautB.exe c:\gap-4.10.0\bin\i686-pc-cygwin-default32\cygwin1.dll

Then grape works well:

gap> Test(Filename(DirectoriesPackageLibrary("grape","tst"),"testall.tst"));
GRAPE package: testall.tst
msecs: 2434
true

log4.10.0.log

[olexandr-konovalov]

GRAPE is now on GitHub - reported this at https://github.com/gap-packages/grape/issues/10

[olexandr-konovalov]

I had a look at the logs. When GRAPE is built in 32-bit Cygwin, it all works:

Running './configure' '/cygdrive/c/gap-4.10.2' 
Running 'make' 
mkdir -p bin/i686-pc-cygwin-default32-kv3/
( cd nauty22; rm -f *.o config.log config.cache config.status makefile; ./configure; make dreadnautB; mv dreadnautB ../bin/i686-pc-cygwin-default32-kv3; chmod 755 ../bin/i686-pc-cygwin-default32-kv3/dreadnautB; rm -f *.o )
creating cache ./config.cache
checking host system type... pentium3-pc-cygwin
checking for gcc... gcc
checking whether the C compiler (gcc  ) works... yes
checking whether the C compiler (gcc  ) is a cross-compiler... no
checking whether we are using GNU C... yes
checking whether gcc accepts -g... yes
checking if gcc supports -O4 flags... yes
checking if gcc supports -march=i686 flags... yes
CFLAGS= -O4 -march=i686
checking for sys/wait.h that is POSIX.1 compatible... yes
checking how to run the C preprocessor... gcc -E
checking for stddef.h... yes
checking for unistd.h... yes
checking for sys/types.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for errno.h... yes
checking for ANSI C header files... yes
checking for pid_t... yes
checking if malloc is declared in stdlib.h or malloc.h... 1
checking if ftell is declared in stdio.h... 1
checking if fdopen is declared in stdio.h... 1
checking if popen is declared in stdio.h... 1
checking if INFINITY is declared in math.h... 1
checking size of int... 4
checking size of long... 4
checking size of long long... 8
checking for working const... yes
checking for isatty... yes
checking for times... yes
checking for time... yes
checking for gettimeofday... yes
checking for times... (cached) yes
checking for getrusage... yes
checking for perror... yes
checking for pipe... yes
checking for wait... yes
checking for popen... yes
checking for gsort... no
checking for sort... sort
updating cache ./config.cache
creating ./config.status
creating makefile
creating nauty.h
creating naututil.h
creating gtools.h
make[1]: Entering directory '/cygdrive/c/gap-4.10.2/pkg/grape-4.8.2/nauty22'
gcc -c -O4 -march=i686  -DBIGNAUTY -o naututilB.o naututil.c
gcc -c -O4 -march=i686  -DBIGNAUTY -o nautyB.o nauty.c
gcc -c -O4 -march=i686  -DBIGNAUTY -o nautilB.o nautil.c
gcc -c -O4 -march=i686  -DBIGNAUTY -o nautinvB.o nautinv.c
gcc -c -O4 -march=i686  -DBIGNAUTY -o naugraphB.o naugraph.c
gcc -c -O4 -march=i686  rng.c
gcc -o dreadnautB -O4 -march=i686 -DBIGNAUTY dreadnaut.c naututilB.o \
    nautyB.o nautilB.o nautinvB.o naugraphB.o rng.o 
make[1]: Leaving directory '/cygdrive/c/gap-4.10.2/pkg/grape-4.8.2/nauty22'

but an attempt to build it on a 64-bit machine fails:

Running './configure' '/cygdrive/c/gap-4.10.2-x86_64' 
Running 'make' 
mkdir -p bin/x86_64-unknown-cygwin-default64-kv3/
( cd nauty22; rm -f *.o config.log config.cache config.status makefile; ./configure; make dreadnautB; mv dreadnautB ../bin/x86_64-unknown-cygwin-default64-kv3; chmod 755 ../bin/x86_64-unknown-cygwin-default64-kv3/dreadnautB; rm -f *.o )
creating cache ./config.cache
checking host system type... configure: error: can not guess host type; you must specify one
make[1]: Entering directory '/cygdrive/c/gap-4.10.2-x86_64/pkg/grape-4.8.2/nauty22'
make[1]: *** No rule to make target 'dreadnautB'.  Stop.
make[1]: Leaving directory '/cygdrive/c/gap-4.10.2-x86_64/pkg/grape-4.8.2/nauty22'
mv: cannot stat ‘dreadnautB’: No such file or directory
chmod: cannot access ‘../bin/x86_64-unknown-cygwin-default64-kv3/dreadnautB’: No such file or directory

Is there a way to enforce bin/BuildPackages.sh to build GRAPE in 32-bit mode on a 64-bit machine?

[olexandr-konovalov]

I will check what happens on the new VM with Cygwin64 that I've set up in June.

[fingolfin]

This PR might fix the issue: https://github.com/gap-packages/grape/pull/11

[fingolfin]

This should be fixed by/in the next GRAPE release.

[olexandr-konovalov]

GRAPE is now loadable in 64-bit GAP, which is the default Windows build in GAP 4.11.0 (see #2112).