'ps.h' file not found

[dreampuf]

Hi, do you know where could I get this ps.h? I guess it's a header of postscript. It was from a clean docker container ubuntu:18.04.

docker run -it --rm --name js -v "$PWD:/opt/work" -w "/opt/work" -p 8080:8080 ubuntu:18.04 bash

And couple instructions.

apt update
./scripts/cpp-install-prerequisites.sh
npm ci
npm run install-build-deps
npm run build

[ 97%] Building C object graphviz/plugin/core/CMakeFiles/gvplugin_core.dir/opt/work/src-graphviz/plugin/core/gvrender_core_tk.c.o
[ 98%] Building C object graphviz/plugin/core/CMakeFiles/gvplugin_core.dir/opt/work/src-graphviz/plugin/core/gvrender_core_vml.c.o
/opt/work/src-graphviz/plugin/core/gvrender_core_ps.c:24:10: fatal error: 'ps.h' file not found
#include "ps.h"
         ^~~~~~
1 error generated.
shared:ERROR: '/opt/work/emsdk/upstream/bin/clang -target wasm32-unknown-emscripten -D__EMSCRIPTEN_major__=1 -D__EMSCRIPTEN_minor__=39 -D__EMSCRIPTEN_tiny__=8 -D_LIBCPP_ABI_VERSION=2 -Dunix -D__unix -D__unix__ -Werror=implicit-function-declaration -Xclang -nostdsysteminc -Xclang -isystem/opt/work/emsdk/upstream/emscripten/system/include/libcxx -Xclang -isystem/opt/work/emsdk/upstream/emscripten/system/lib/libcxxabi/include -Xclang -isystem/opt/work/emsdk/upstream/emscripten/system/include/compat -Xclang -isystem/opt/work/emsdk/upstream/emscripten/system/include -Xclang -isystem/opt/work/emsdk/upstream/emscripten/system/include/libc -Xclang -isystem/opt/work/emsdk/upstream/emscripten/system/lib/libc/musl/arch/emscripten -Xclang -isystem/opt/work/emsdk/upstream/emscripten/system/local/include -Xclang -isystem/root/.emscripten_cache/wasm-obj/include -I/opt/work/cpp/graphviz -I/opt/work/cpp/../src-graphviz/build/plugin/core -I/opt/work/cpp/../src-graphviz/plugin/core -I/opt/work/cpp/../src-graphviz/lib/vpsc -I/opt/work/cpp/../src-graphviz/lib/twopigen -I/opt/work/cpp/../src-graphviz/lib/sparse -I/opt/work/cpp/../src-graphviz/lib/sfdpgen -I/opt/work/cpp/../src-graphviz/lib/pathplan -I/opt/work/cpp/../src-graphviz/lib/common -I/opt/work/cpp/../src-graphviz/lib/gvc -I/opt/work/cpp/../src-graphviz/lib/cdt -I/opt/work/cpp/../src-graphviz/lib/cgraph -I/opt/work/cpp/../src-graphviz/lib/xdot -DNDEBUG -Os -c -DEMSCRIPTEN /opt/work/src-graphviz/plugin/core/gvrender_core_ps.c -fno-inline-functions -Xclang -isystem/opt/work/emsdk/upstream/emscripten/system/include/SDL -c -o CMakeFiles/gvplugin_core.dir/opt/work/src-graphviz/plugin/core/gvrender_core_ps.c.o -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr' failed (1)
graphviz/plugin/core/CMakeFiles/gvplugin_core.dir/build.make:238: recipe for target 'graphviz/plugin/core/CMakeFiles/gvplugin_core.dir/opt/work/src-graphviz/plugin/core/gvrender_core_ps.c.o' failed
make[2]: *** [graphviz/plugin/core/CMakeFiles/gvplugin_core.dir/opt/work/src-graphviz/plugin/core/gvrender_core_ps.c.o] Error 1
make[2]: *** Waiting for unfinished jobs....
CMakeFiles/Makefile2:1333: recipe for target 'graphviz/plugin/core/CMakeFiles/gvplugin_core.dir/all' failed
make[1]: *** [graphviz/plugin/core/CMakeFiles/gvplugin_core.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2

[GordonSmith]

I think its supposed to be auto-created from src-graphviz\plugin\core\ps.txt during the https://github.com/hpcc-systems/hpcc-js-wasm/blob/master/scripts/cpp-install-graphviz.sh script.

Why its not building in you environment and is in mine is going to be down to a missing package I suspect, but offhand I can't say which one...

[GordonSmith]

FWIW on my clean Ubuntu environment I tend to run the following:

sudo apt-get install cmake bison flex build-essential binutils-dev libldap2-dev libcppunit-dev libicu-dev libxslt1-dev zlib1g-dev libboost-regex-dev libarchive-dev python-dev libv8-dev default-jdk libapr1-dev libaprutil1-dev libiberty-dev libhiredis-dev libtbb-dev libxalan-c-dev libnuma-dev nodejs libevent-dev libatlas-base-dev libblas-dev python3-dev default-libmysqlclient-dev libsqlite3-dev r-base-dev r-cran-rcpp r-cran-rinside r-cran-inline libmemcached-dev libcurl4-openssl-dev pkg-config libtool autotools-dev automake libssl-dev

And I suspect you missing dependency is in that list?

[GordonSmith]

This looks like the line that generates it:

$(AWK) -f $(top_srcdir)/awk/stringize.awk $(srcdir)/ps.txt > **ps.h**

Do you have awk installed in your environment? Also look further up your logs to see if there is an error during the configuration or cmake steps?

[GordonSmith]

If it is that then I will add it to: https://github.com/hpcc-systems/hpcc-js-wasm/blob/master/scripts/cpp-install-prerequisites.sh

[dreampuf]

Thank you for the quick response.

I do have awk in my environment.

root@5e80691c263c:/opt/work# awk -W version
mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan

compiled limits:
max NF             32767
sprintf buffer      2040

I tried your suggestion and it will involve a conflict with node-gyp. I found that could be resolved by install some dependencies first.

# after `apt-get install cmake bison flex build-essential binutils-dev libldap2-dev libcppunit-dev libicu-dev libxslt1-dev zlib1g-dev libboost-regex-dev libarchive-dev python-dev libv8-dev default-jdk libapr1-dev libaprutil1-dev libiberty-dev libhiredis-dev libtbb-dev libxalan-c-dev libnuma-dev nodejs libevent-dev libatlas-base-dev libblas-dev python3-dev default-libmysqlclient-dev libsqlite3-dev r-base-dev r-cran-rcpp r-cran-rinside r-cran-inline libmemcached-dev libcurl4-openssl-dev pkg-config libtool autotools-dev automake libssl-dev`

root@5e80691c263c:/opt/work# apt install -y nodejs npm cmake gcc-multilib g++-multilib pkg-config autoconf bison libtool flex zlib1g-dev python2.7 python-pip
Reading package lists... Done
Building dependency tree
Reading state information... Done
autoconf is already the newest version (2.69-11).
autoconf set to manually installed.
bison is already the newest version (2:3.0.4.dfsg-1build1).
flex is already the newest version (2.6.4-6).
libtool is already the newest version (2.4.6-2).
pkg-config is already the newest version (0.29.1-0ubuntu2).
zlib1g-dev is already the newest version (1:1.2.11.dfsg-0ubuntu2).
cmake is already the newest version (3.10.2-1ubuntu2.18.04.1).
python2.7 is already the newest version (2.7.17-1~18.04).
python2.7 set to manually installed.
nodejs is already the newest version (8.10.0~dfsg-2ubuntu0.4).
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 npm : Depends: node-gyp (>= 0.10.9) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

apt install -y nodejs-dev node-gyp libssl1.0-dev
apt install -y nodejs npm cmake gcc-multilib g++-multilib pkg-config autoconf bison libtool flex zlib1g-dev python2.7 python-pip

I'm not sure if that is the problem. I don't have ps.txt.

root@5e80691c263c:/opt/work# ll src-graphviz/ps.txt
ls: cannot access 'src-graphviz/ps.txt': No such file or directory

[GordonSmith]

Delete the src-graphviz folder and try the following (inside Ubuntu and hpcc-js-wasm folder):

sudo ./scripts/cpp-install-prerequisites.sh
./scripts/cpp-install-graphviz.sh

Keeping the logs in case there are any errors...

Once complete check if this path exists: src-graphviz\build\plugin\core\ps.h

[dreampuf]

ubuntu:1804 doesn't have wget by default. It needs to be installed before ./scripts/cpp-install-graphviz.sh. It could compile entire graphviz now. But I got some undefined errors during generating webassembly.

[ 98%] Built target gvplugin_core
[ 98%] Generating main_glue.js, main_glue.cpp
Scanning dependencies of target graphvizlib
[ 99%] Building CXX object graphviz/graphvizlib/CMakeFiles/graphvizlib.dir/main.cpp.o
[ 99%] Building CXX object graphviz/graphvizlib/CMakeFiles/graphvizlib.dir/fs.cpp.o
[100%] Linking CXX executable graphvizlib.js
cache:INFO: generating system library: libcompiler_rt.a... (this will be cached in "/root/.emscripten_cache/wasm-obj/libcompiler_rt.a" for subsequent builds)
cache:INFO:  - ok
cache:INFO: generating system library: libc-wasm.a... (this will be cached in "/root/.emscripten_cache/wasm-obj/libc-wasm.a" for subsequent builds)
cache:INFO:  - ok
cache:INFO: generating system library: libc++-noexcept.a... (this will be cached in "/root/.emscripten_cache/wasm-obj/libc++-noexcept.a" for subsequent builds)
cache:INFO:  - ok
cache:INFO: generating system library: libc++abi-noexcept.a... (this will be cached in "/root/.emscripten_cache/wasm-obj/libc++abi-noexcept.a" for subsequent builds)
cache:INFO:  - ok
cache:INFO: generating system library: libdlmalloc.a... (this will be cached in "/root/.emscripten_cache/wasm-obj/libdlmalloc.a" for subsequent builds)
cache:INFO:  - ok
cache:INFO: generating system library: libpthread_stub.a... (this will be cached in "/root/.emscripten_cache/wasm-obj/libpthread_stub.a" for subsequent builds)
cache:INFO:  - ok
cache:INFO: generating system library: libc_rt_wasm.a... (this will be cached in "/root/.emscripten_cache/wasm-obj/libc_rt_wasm.a" for subsequent builds)
cache:INFO:  - ok
cache:INFO: generating system library: libsockets.a... (this will be cached in "/root/.emscripten_cache/wasm-obj/libsockets.a" for subsequent builds)
cache:INFO:  - ok
wasm-ld: error: ../plugin/core/libgvplugin_core.a(gvrender_core_dot.c.o): undefined symbol: AgIoDisc
wasm-ld: error: ../plugin/core/libgvplugin_core.a(gvrender_core_dot.c.o): undefined symbol: AgIoDisc
wasm-ld: error: ../plugin/neato_layout/libgvplugin_neato_layout.a(gvlayout_neato_layout.c.o): undefined symbol: Nop
wasm-ld: error: ../plugin/neato_layout/libgvplugin_neato_layout.a(gvlayout_neato_layout.c.o): undefined symbol: Nop
wasm-ld: error: ../plugin/neato_layout/libgvplugin_neato_layout.a(gvlayout_neato_layout.c.o): undefined symbol: Nop
wasm-ld: error: ../plugin/neato_layout/libgvplugin_neato_layout.a(gvlayout_neato_layout.c.o): undefined symbol: Nop
wasm-ld: error: ../lib/circogen/libcircogen.a(blocktree.c.o): undefined symbol: Verbose
wasm-ld: error: ../lib/circogen/libcircogen.a(circularinit.c.o): undefined symbol: Ndim
wasm-ld: error: ../lib/circogen/libcircogen.a(circularinit.c.o): undefined symbol: E_weight
wasm-ld: error: ../lib/circogen/libcircogen.a(circularinit.c.o): undefined symbol: Agstrictundirected
wasm-ld: error: ../lib/circogen/libcircogen.a(circularinit.c.o): undefined symbol: Ndim
wasm-ld: error: ../lib/dotgen/libdotgen.a(class2.c.o): undefined symbol: Concentrate
wasm-ld: error: ../lib/dotgen/libdotgen.a(class2.c.o): undefined symbol: Concentrate
wasm-ld: error: ../lib/dotgen/libdotgen.a(class1.c.o): undefined symbol: E_constr
wasm-ld: error: ../lib/circogen/libcircogen.a(blocktree.c.o): undefined symbol: Verbose
wasm-ld: error: ../lib/circogen/libcircogen.a(circularinit.c.o): undefined symbol: E_weight
wasm-ld: error: ../lib/dotgen/libdotgen.a(dotinit.c.o): undefined symbol: N_group
wasm-ld: error: ../lib/dotgen/libdotgen.a(dotinit.c.o): undefined symbol: N_group
wasm-ld: error: ../lib/dotgen/libdotgen.a(dotinit.c.o): undefined symbol: E_showboxes
wasm-ld: error: ../lib/dotgen/libdotgen.a(dotinit.c.o): undefined symbol: E_minlen
wasm-ld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors)
shared:ERROR: '/opt/work/emsdk/upstream/bin/wasm-ld -o /opt/work/emscripten_temp_aajUPp/graphvizlib.wasm --lto-O0 CMakeFiles/graphvizlib.dir/main.cpp.o CMakeFiles/graphvizlib.dir/fs.cpp.o -L/opt/work/emsdk/upstream/emscripten/system/local/lib ../lib/gvc/libgvc.a -L/opt/work/emsdk/upstream/emscripten/system/lib ../plugin/core/libgvplugin_core.a -L/root/.emscripten_cache/wasm-obj ../plugin/dot_layout/libgvplugin_dot_layout.a ../plugin/neato_layout/libgvplugin_neato_layout.a ../lib/circogen/libcircogen.a ../lib/dotgen/libdotgen.a ../lib/osage/libosage.a ../lib/patchwork/libpatchwork.a ../lib/twopigen/libtwopigen.a ../lib/sfdpgen/libsfdpgen.a ../lib/gvc/libgvc.a ../lib/common/libcommon.a ../lib/cgraph/libcgraph.a ../lib/pathplan/libpathplan.a ../lib/sparse/libsparse.a ../lib/xdot/libxdot.a ../lib/neatogen/libneatogen.a ../lib/pack/libpack.a ../lib/ortho/libortho.a ../lib/fdpgen/libfdpgen.a ../lib/label/liblabel.a ../lib/gvc/libgvc.a ../lib/common/libcommon.a ../lib/cgraph/libcgraph.a ../lib/pathplan/libpathplan.a ../lib/sparse/libsparse.a ../lib/xdot/libxdot.a ../lib/neatogen/libneatogen.a ../lib/pack/libpack.a ../lib/ortho/libortho.a ../lib/fdpgen/libfdpgen.a ../lib/label/liblabel.a ../lib/expat/libexpat.a ../lib/vpsc/libvpsc.a ../lib/cdt/libcdt.a /root/.emscripten_cache/wasm-obj/libc.a /root/.emscripten_cache/wasm-obj/libcompiler_rt.a /root/.emscripten_cache/wasm-obj/libc-wasm.a /root/.emscripten_cache/wasm-obj/libc++-noexcept.a /root/.emscripten_cache/wasm-obj/libc++abi-noexcept.a /root/.emscripten_cache/wasm-obj/libdlmalloc.a /root/.emscripten_cache/wasm-obj/libpthread_stub.a /root/.emscripten_cache/wasm-obj/libc_rt_wasm.a /root/.emscripten_cache/wasm-obj/libsockets.a --allow-undefined --import-memory --import-table -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --strip-debug --export __wasm_call_ctors --export __data_end --export main --export malloc --export free --export setThrew --export __errno_location --export emscripten_builtin_memalign --export memalign --export emscripten_builtin_free --export realloc --export _get_environ --export _ZSt18uncaught_exceptionv -z stack-size=5242880 --initial-memory=16777216 --no-entry --global-base=1024' failed (1)
graphviz/graphvizlib/CMakeFiles/graphvizlib.dir/build.make:167: recipe for target 'graphviz/graphvizlib/graphvizlib.js' failed
make[2]: *** [graphviz/graphvizlib/graphvizlib.js] Error 1
CMakeFiles/Makefile2:1520: recipe for target 'graphviz/graphvizlib/CMakeFiles/graphvizlib.dir/all' failed
make[1]: *** [graphviz/graphvizlib/CMakeFiles/graphvizlib.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2

[GordonSmith]

They look like a failed emsdk issue now... Which could be down to the missing wget also! Delete the emsdk folder and re-run ./scripts/cpp-install-emsdk.sh

(might be worth while doing a clean clone and starting from scratch if wget was missing!)

[GordonSmith]

I am adding wget to pre-reqs now.

[GordonSmith]

BTW Why are you building this, rather than simply doing a npm install @hpcc-js/wasm?

Also which docker image are you building in, so I can add notes to the readme once its working?

[dreampuf]

I'm trying to test how to render a huge dot file. Emscripten has some switches to limit the stack size and total memory.

[GordonSmith]

Ahh - unfortunatly there are pathogenic graphs that kill command line "dot" as well (and not that big...)

Once building you will can set those flags here: https://github.com/hpcc-systems/hpcc-js-wasm/blob/master/cpp/graphviz/graphvizlib/CMakeLists.txt#L3-L4

I have already enabled "ALLOW_MEMORY_GROWTH"...

[GordonSmith]

@dreampuf - can you confirm that this is now building in your clean docker container and that the main issue was a missing wget?

If so the following PR should have resolved the issue and I can close: https://github.com/hpcc-systems/hpcc-js-wasm/pull/19

[GordonSmith]

@dreampuf FYI: https://github.com/hpcc-systems/hpcc-js-wasm/blob/master/test/ubuntu-dev.dockerfile

[dreampuf]

It works. Thanks for your efforts.