emcc fails to compile fc-solve properly.

[shlomif]

Hi all,

and thanks for emcc.

I've decided to try out emcc so I can see if I can easily create a JS version Freecell Solver ( http://fc-solve.shlomifish.org/ ) which is written in -std=gnu99 . However, I ran into a few problems. Here is a way to reproduce it:

git clone https://shlomif@bitbucket.org/shlomif/fc-solve.git
cd fc-solve/fc-solve-source
bash ~/fc-solve-emcc-test.bash | less

Where fc-solve-emcc-test.bash is:

#!/bin/bash

if ! test -d build ; then
    mkdir build
fi

cd build
cmake ..
make boards
OBJS="../app_str.c ../scans.c ../main.c ../lib.c ../preset.c ../instance.c ../move_funcs_order.c  ../move_funcs_maps.c ../meta_alloc.c ../cmd_line.c ../card.c ../state.c ../check_and_add_state.c ../fcs_hash.c ../split_cmd_line.c ../simpsim.c ../freecell.c ../move.c ../fc_pro_iface.c ../rate_state.c"

emcc -m32 -std=gnu99 -g -o fc-solve.js -I. $OBJS --embed-file 24.board
gcc -std=gnu99 -g -o fc-solve.exe -I. $OBJS

echo "Running node.js fc-solve"
echo "[[[[["
node fc-solve.js -s -i -p -t 24.board
echo "]]]]]"

echo "Running the gcc-built fc-solve"
echo "[[[[["
./fc-solve.exe -s -i -p -t 24.board
echo "]]]]]"

As you can see, the output of node.js is incomplete vs. fc-solve.exe.

Another problem I noticed was that specifying only part of the C modules (with links to other routines) did not fail. Is there a way to make the compilation self-contained?

I'm on Mageia Linux 3/Cauldron x86-64 using clang-3.2 and llvm-3.2.

Regards,

— Shlomi Fish

[kripken]

cmake fails for me with

CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
LIBGMP_LIB
[..]
LIBTAP_LIB

[shlomif]

Hi kripken,

thanks for trying to help. Re the LIBGMP_LIB error, you need to install the GMP (GNU Multiple Precision) library. It is "apt-get install libgmp-dev" on Debian/Ubuntu/Mint ; "urpmi gmp-devel" on Mageia/Mandriva ; and "yum install gmp-devel" on Fedora/RHEL/CentOS/etc. . You can also build it from source: http://gmplib.org/ . Maybe I should make the error message easier to understand.

[kripken]

If this needs to link with that, then libgmp would need to be compiled with emcc as well, linking against the native one that apt-get provides won't work. Or does it just need the headers perhaps?

And what is libtap?

[shlomif]

Hi kripken,

sorry for giving hard-to-follow instructions. I guess every new user of the system uncovers new class of bugs, but it's great because I can improve the usability and make it more “idiot-proof”.

fc-solve does not need or link against libgmp - it is just needed by the CMake build for a different part of Freecell Solver (dbm_fc_solver). Furthermore, we only need the CMake process to generate the "config.h". To avoid requiring libtap (which is only needed by parts of the test suite, please use the following script instead of what I gave you:

#!/bin/bash

if ! test -d build ; then
    mkdir build
fi

cd build
cmake -DFCS_WITH_TEST_SUITE= ..
make boards
OBJS="../app_str.c ../scans.c ../main.c ../lib.c ../preset.c ../instance.c ../move_funcs_order.c  ../move_funcs_maps.c ../meta_alloc.c ../cmd_line.c ../card.c ../state.c ../check_and_add_state.c ../fcs_hash.c ../split_cmd_line.c ../simpsim.c ../freecell.c ../move.c ../fc_pro_iface.c ../rate_state.c"

emcc -m32 -std=gnu99 -g -o fc-solve.js -I. $OBJS --embed-file 24.board
gcc -std=gnu99 -g -o fc-solve.exe -I. $OBJS

echo "Running node.js fc-solve"
echo "[[[[["
node fc-solve.js -s -i -p -t 24.board
echo "]]]]]"

echo "Running the gcc-built fc-solve"
echo "[[[[["
./fc-solve.exe -s -i -p -t 24.board
echo "]]]]]"

As you can see, after generating config.h I'm not really using cmake either for "fc-solve.exe" or for "fc-solve.js" and GMP is not required either.

Thanks, and happy Holidays.

Regards,

— Shlomi Fish

[shlomif]

OK, the build process in the master of the git repository on Bitbucket ( https://shlomif@bitbucket.org/shlomif/fc-solve.git ) has been made more user-friendly idiot-proof. You can git pull if you want.

[shlomif]

@kripken : BTW, your name has a very Israeli ring to it. Are you Israeli, by any chance? I am Israeli as well, in case you didn't guess.

[kripken]

Ok, that helps and I pass cmake, but now I hit

emcc: ../rate_state.c: error: No such file or directory
gcc: ../rate_state.c: No such file or directory

and I do not see that file (there is a header with that name though).

P.S. yeah, my dad is israeli, good guess ;)

[shlomif]

@kripken: hi. OK, I was able to reproduce this problem in a fresh repository too and fixed it. The problem was that rate_state.c was only generated by "make" and not by the cmake process, so I fixed it to avoid problems in the future.

Here is the commit, and after I did this change (in a fresh checkout) all the commands yielded the results I demonstrated here. Thanks again and please look into it:

shlomif@telaviv1:~/progs/freecell/more-git/fc-solve/fc-solve/source$ git show
commit b47591cad849a6f3398a4b3f9d3fe93aa6130eb1
Author: Shlomi Fish <shlomif@shlomifish.org>
Date:   Mon Dec 31 10:16:47 2012 +0200

    Add the EXECUTE_PROCESS for the rate_state_gen.

diff --git a/fc-solve/source/CMakeLists.txt b/fc-solve/source/CMakeLists.txt
index 633b168..84911dd 100644
--- a/fc-solve/source/CMakeLists.txt
+++ b/fc-solve/source/CMakeLists.txt
@@ -186,13 +186,18 @@ IF (FCS_ENABLE_RCS_STATES)
 ENDIF (FCS_ENABLE_RCS_STATES)

 SET (rate_state_gen "${CMAKE_CURRENT_SOURCE_DIR}/gen_rate_state_c.pl")
+SET (rate_state_args "${rate_state_gen}" "${CMAKE_CURRENT_SOURCE_DIR}")
+# We do this so there will be no need to run "make" first.
+# See: http://github.com/kripken/emscripten/issues/766#issuecomment-11771150
 ADD_CUSTOM_COMMAND(
     OUTPUT "${CMAKE_CURRENT_SOURCE_DIR}/rate_state.c"
     COMMAND "perl"
-    ARGS "${rate_state_gen}" "${CMAKE_CURRENT_SOURCE_DIR}"
+    ARGS ${rate_state_args}
     DEPENDS "${rate_state_gen}" "${CMAKE_CURRENT_SOURCE_DIR}/rate_state.h"
 )

+EXECUTE_PROCESS(COMMAND "perl" ${rate_state_args})
+
 SET (FREECELL_SOLVER_LIB_MODULES
     app_str.c
     card.c

[kripken]

Ok, when I run it now it seems to work fine. The output from the native build is 100% identical to the js build,

https://gist.github.com/4424060

What error do you get?

[shlomif]

@kripken : hi and thanks for the reply. After I did "git pull --ff-only origin master" and reran the tests, everything seems fine now and the node.js program is running fine. Perhaps this bug (or at least its symptom) was fixed in one of the subsequent commits. As far as I'm concerned, this bug can be fixed, but you may wish to add a regression test.

shlomif@telaviv1:~/Download/unpack/prog/llvm-to-js/emscripten$ git reflog
ada59f0 HEAD@{0}: pull --ff-only origin master: Fast-forward
2f4401a HEAD@{1}: clone: from https://github.com/kripken/emscripten.git
shlomif@telaviv1:~/Download/unpack/prog/llvm-to-js/emscripten$ 

[kripken]

If you can still reproduce the problem on an older commit, let me know, I'd like to add a regression test based on that (I try to make the tests be specific to the issue, so without seeing the problem myself I can't add one).

[shlomif]

@kripken: hi! I can reproduce it with

shlomif@telaviv1:~/Download/unpack/prog/llvm-to-js/emscripten$ git show
commit 2f4401a79e96917f78876c3e09b8d31754a26f70
Merge: 2c8c19f 160cc72
Author: Alon Zakai <alonzakai@gmail.com>
Date:   Fri Dec 21 15:54:59 2012 -0800

    Merge branch 'incoming'

shlomif@telaviv1:~/Download/unpack/prog/llvm-to-js/emscripten$ 

I can try to bisect.

[kripken]

I did a quick bisect, looks like this was fixed in

commit 5dbcaabf71b885926a105db3ab725d4bd76c8f41
Author: Alon Zakai <alonzakai@gmail.com>
Date:   Mon Dec 24 15:51:49 2012 -0800

    fix llvm.expect and add test

No need to add a new test then if that was the cause. Glad we already fixed this.

[shlomif]

@kripken : thanks for the investigation and the fix.

That put aside, is there any way I can invoke the emcc compiler incrementally on each .c file from which I want to prepare an executable to speed up the compilation? This would be similar to "gcc -c" or "clang -c" (compile and not link).

Regards,

— Shlomi Fish

[kripken]

If you have a Makefile that tracks dependencies (instead of passing all .c files to emcc as in that shell script), it will compile only the modified .c files into bitcode and save time that way. That won't help for compiling bitcode to JS though, for that use the jcache option in emcc (run it with --help for info). The bitcode to JS step is mostly parallel btw, so more CPU cores will scale it almost linearly.

edit: -c works in emcc just like gcc or clang

[shlomif]

@kripken : sorry for the late response and thanks for the tip. After a little STFWing and RTFMing I ended up with this makefile . It is working now liked it worked before but I have still to figure out how to either get STDIN to be done asynchronously inside the .html file by allowing the user to fill in a text area (rather than the message box popup that only accepts a line at a time), or alternatively how to do something like “demos/python.html” and “demos/python.cc.js” which I see no explanations in the code how they were generated / written. I think I can manage from here by pursuing the wiki and the files in the distribution, but it will take some time.

Thanks again for a great project and for your help.

[shlomif]

@kripken : I already managed on my own based on reading the documentation on the wiki, and some tinkering. Thanks anyway!