Problem with running tests in ppl and logol

[dogsleg]

Please, see the following Debian bugs:

916942 -- https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=916942

916952 -- https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=916952

916953 -- https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=916953

Trying to recompile logol package, tests are failing with error:

/usr/lib/swi-prolog/library/process.pl:57: Initialization goal raised

In process.pl this is related to :

:-use_foreign_library(foreign(process))."

Stack error:

process: cannot open shared object file: No such file or directory In: [stderr] ERROR: [18] throw(error(shared_object(open,'process: cannot open shared object file: No such file or directory'),context(...,_918)))

[osallou]

For bugs 916942 and 916953, I tried to recompile swi-prolog from scratch on Debian system following http://www.swi-prolog.org/build/unix.html

I tested it again, and get same error, so this is not related to Debian package itself.

I compile with swipl-ld a prolog file that executes a process and process raise the error:

ERROR: /usr/lib/swi-prolog/library/process.pl:57: Initialization goal raised exception:
ERROR: process: cannot open shared object file: No such file or directory
ERROR: In:
ERROR:   [18] throw(error(shared_object(open,'process: cannot open shared object file: No such file or directory'),context(...,_918)))
ERROR:   [16] <meta call>
ERROR:   [15] with_mutex('$foreign',load_foreign_library(foreign(process),process,default(install))) <foreign>
ERROR:   [12] '$run_init_goal'(shlib:load_foreign_library(...)) at /usr/lib/swi-prolog/boot/init.pl:616
ERROR:   [11] catch(system:'$run_init_goal'(...),_1044,system:'$initialization_error'(_1066,...,...)) at /usr/lib/swi-prolog/boot/init.pl:384
ERROR:   [10] catch_with_backtrace(system:'$run_init_goal'(...),_1096,system:'$initialization_error'(_1118,...,...)) at /usr/lib/swi-prolog/boot/init.pl:434
ERROR: 
ERROR: Note: some frames are missing due to last-call optimization.
ERROR: Re-run your program in debug mode (:- debug.) to get more detail.

With simple example (see example fiels at https://groups.google.com/forum/#!topic/swi-prolog/PJTBwNMGmo8):

swipl-ld -o test.exe test.c test.pl
./test.exe pi/2

=> raise the error

Running the test.pl directly in swipl (not compiled version with swipl-ld) works fine:

swipl -s test.pl

?- calc('pi/2').

[erlanger]

@osallou This could be because by default the new build installs libswipl.so.* in /usr/lib/swipl/lib/<arch>/. Normally this is not a problem because the swipl binary has the rpath set so that it can find the libraries.

I think on debian rpath (or runpath) is not allowed, and this could be the cause of the problem.

Try recompiling with cmake -DSWIPL_INSTALL_IN_LIB=ON ..... this will place libswipl.so* in /usr/lib.

Alternatively can run cmake-gui . in the build directory and turn on the SWIPL_INSTALL_IN_LIB option. cmake-gui is useful in so far as it lets you see the different compilation options with a tooltip describing them.

Merry Christmas!

[dogsleg]

For Debian SWI-Prolog is compiled with -DSWIPL_INSTALL_IN_LIB=ON. Moreover, package/jpl is compiled with INSTALL_RPATH "" and INSTALL_RPATH_USE_LINK_PATH FALSE to prevent RPATH in SWI-Prolog's jpl package (see, package/jpl/CMakeLists.txt).

[erlanger]

I see... the problem is that process.so is not found in the list of foreign file_search_path, as called from process.pl line 57

Some more interesting information:

if we run swipl-ld with the -v flag we can see (using the test.pl example above) that it calls:

swipl -f none -F none -g true -t "consult(['test.pl']),qsave_program('pltmp-7729',[goal=version,toplevel=prolog,init_file=none])"

I can reproduce the problem ONLY by running swipl from the source tree (without installing it): [Please note that my swipl below is pointing to the source tree swipl (not the installed one in /usr/bin)]

$ swipl -f none -F none -g true -t "consult(['test.pl']),qsave_program('pltmp-7729' [goal=version,toplevel=prolog,init_file=none])"
ERROR: /home/u/tmp/swipl-devel/build.release/home/library/process.pl:57:
        /home/u/tmp/swipl-devel/build.release/home/library/process.pl:57: Initialization goal raised exception:
        '$open_shared_object'/3: process: cannot open shared object file: No such file or directory
ERROR: /tmp/test.pl:1:
        Exported procedure process:process_set_method/1 is not defined
ERROR: /tmp/test.pl:1:
        Exported procedure process:process_kill/2 is not defined
ERROR: /tmp/test.pl:1:
        Exported procedure process:process_group_kill/2 is not defined
ERROR: /tmp/test.pl:1:
        Exported procedure process:process_wait/3 is not defined
% halt

If we run the without the -F none option everything works fine:

$ swipl -f none  -g true -t "consult(['test.pl']),qsave_program('pltmp-7729',[goal=version,toplevel=prolog,init_file=none
])"
% halt

 $ file ./pltmp-7729
./pltmp-7729: POSIX shell script executable (binary data)

The reason is because -F none prevents loading swipl.rc which sets up the foreign file_search_path which allows it to find process.so.

If I run the installed version everything works fine:

 /usr/bin/swipl -f none -F none -g true -t "consult(['test.pl']),qsave_program('pltmp-7729',[goal=version,toplevel=prolo
g,init_file=none])"
% halt

As you see (on archlinux) everything works fine if I use the installed swipl (which is the same version as the source tree version which I used above to reproduce the problem).

We could just remove -F none from swipl-ld.c but t is better to fix the root cause.

[JanWielemaker]

If running from the source directory is the problem I'm not very tempted to try and find a solution. The possibility to run from the sources is there to simplify development, not for normal use. If a developer is really persistent, setting up the right LD_LIBRARY_PATH probably solves the problem.

[osallou]

When tested on Debian (from package or locally compiled/installed), the problem occured from installed swipl-ld, not from local source dir.

[osallou]

Setting LD_LIBRARY_PATH does not solve the problem.

make install gives:

-- Install configuration: "Release"
-- Set runtime path of "/usr/local/lib/swipl/bin/x86_64-linux/swipl" to "/usr/local/lib/swipl/lib/x86_64-linux"
-- Set runtime path of "/usr/local/lib/swipl/lib/x86_64-linux/libswipl.so.7.7.25" to "/usr/local/lib/swipl/lib/x86_64-linux"
-- Set runtime path of "/usr/local/lib/swipl/bin/x86_64-linux/swipl-ld" to "/usr/local/lib/swipl/lib/x86_64-linux"
-- Set runtime path of "/usr/local/lib/swipl/lib/x86_64-linux/libjpl.so" to "/usr/local/lib/swipl/lib/x86_64-linux:/usr/lib/jvm/java-8-openjdk-amd64/include:/usr/lib/jvm/java-8-openjdk-amd64/include/linux:/usr/lib/jvm/default-java/lib/server"

root@2bcd5323f5a0:~# ls /usr/local/lib/swipl/lib/x86_64-linux/
archive4pl.so        json.so         pcre4pl.so    sgml2pl.so     turtle.so
bdb4pl.so        libedit4pl.so   pdt_console.so    sha4pl.so      uid.so
cgi.so           libjpl.so       pl2xpce.so    snowball.so    unicode4pl.so
crypt.so         libswipl.so     porter_stem.so    socket.so      unix.so
crypto4pl.so         libswipl.so.7   process.so    ssl4pl.so      uri.so
double_metaphone.so  libswipl.so.7.7.25  prolog_stream.so  streaminfo.so  uuid.so
files.so         mallocinfo.so   protobufs.so      syslog.so      websocket.so
hashstream.so        md54pl.so       rdf_db.so     table.so   yaml4pl.so
http_stream.so       memfile.so      readline4pl.so    tex.so     zlib4pl.so
inclpr.so        ntriples.so     readutil.so       time.so
isub.so          odbc4pl.so      rlimit.so     tipc.so

root@2bcd5323f5a0:~# export LD_LIBRARY_PATH=/usr/local/lib/swipl/lib/x86_64-linux/:$LD_LIBRARY_PATH
root@2bcd5323f5a0:~# ./test.exe pi/2
ERROR: /usr/local/lib/swipl/library/process.pl:57: Initialization goal raised exception:
ERROR: process: cannot open shared object file: No such file or directory
....

root@2bcd5323f5a0:~# ldd test.exe 
linux-vdso.so.1 (0x00007fff48558000)
libswipl.so.7 => /usr/local/lib/swipl/lib/x86_64-linux/libswipl.so.7 (0x00007fc74e15e000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc74df95000)
libncurses.so.6 => /lib/x86_64-linux-gnu/libncurses.so.6 (0x00007fc74df6c000)
libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6 (0x00007fc74df3e000)
libform.so.6 => /lib/x86_64-linux-gnu/libform.so.6 (0x00007fc74df2b000)
libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 (0x00007fc74dea8000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fc74dc88000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc74dc67000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fc74dc62000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fc74dadf000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fc74dad5000)
/lib64/ld-linux-x86-64.so.2 (0x00007fc74e2c6000)

[JanWielemaker]

I can reproduce this. As far as I'm concerned the problem is not so much that process.so cannot be found. These are searched for over the foreign library search path (see file_search_path/2) and the user must ensure the foreign libraries required by the application are distributed with the application and the search paths are setup correctly. We could avoid that by including foreign code into the state by default (that probably makes sense).

The real surprise and probably the thing changed though is that it tries to find library(process). It should not need that. I'll have a look, but I'm mostly not behind a computer these days.

[osallou]

Setting search path to process.so location in test.pl works

:- multifile user:file_search_path/2.
user:file_search_path(foreign, Dir) :-
    Dir='/usr/local/lib/swipl/lib/x86_64-linux'.
....

the problem is process.so is on local system directory but executable does not search in /usr/local/lib/swipl/lib/x86_64-linux. Seems executable does not search in "default" install path.

Embedding foreign code would solve the issue and avoid distributing additional foreign code. But if it is available on system libs, it should be able to find them. We did not face this issue on previous release.

[JanWielemaker]

I think the change is that the last fallback to find Prolog files was the directory where the system was intended to be installed. That may not be working ok anymore with all the changes. It works nice for the Linux version. For other OSes it is a little dubious as the binaries are often installed in different locations.

[JanWielemaker]

Yip. Pushed a fix for this. Please close if you can confirm.

[osallou]

I made a test with latest and it works, thanks! @dogsleg we should take package from latest commit

[dogsleg]

@osallou as I understand it is 42a4f815e1280fbe376157906e4c2b899f277eee, right?

[osallou]

I do not know which commit it is, I tested 30mn ago with latest.

[dogsleg]

OK, I'll test it and upload a new package to the Debian archive in the next few days. Probably tomorrow evening.

@JanWielemaker when do you plan to release the next stable version?

[JanWielemaker]

@dogsleg Good question. Work has been delayed by family matters. The current version is in a good shape, except some pending portability issues such as #394 to #397. I think 8.0 should happen somewhere end of next week or the week thereafter. Would that work?

[dogsleg]

@JanWielemaker yes, that works; hope migration from unstable to testing release of Debian will not be delayed by any issue.