capnp build error

[xet7]

From @xet7 on February 7, 2018 21:41

@kentonv

When I try to build wordpress-sandstorm .spk file in vagrant with this script: https://github.com/wekan/wordpress-sandstorm/blob/master/build-spk-in-vagrant.sh

I get this error when building capnp. How can this be fixed?

libtool: install: chmod 644 /usr/local/lib/libcapnpc.a
libtool: install: ranlib /usr/local/lib/libcapnpc.a
libtool: finish: PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin" ldconfig -n /usr/local/lib
----------------------------------------------------------------------
Libraries have been installed in:
   /usr/local/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the `LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the `LD_RUN_PATH' environment variable
     during linking
   - use the `-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to `/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
 /bin/mkdir -p '/usr/local/bin'
  /bin/bash ./libtool   --mode=install /usr/bin/install -c capnp capnpc-capnp capnpc-c++ '/usr/local/bin'
libtool: install: /usr/bin/install -c .libs/capnp /usr/local/bin/capnp
libtool: install: /usr/bin/install -c .libs/capnpc-capnp /usr/local/bin/capnpc-capnp
libtool: install: /usr/bin/install -c .libs/capnpc-c++ /usr/local/bin/capnpc-c++
make  install-exec-hook
make[3]: Entering directory '/home/vagrant/capnproto/c++'
ln -sf capnp /usr/local/bin/capnpc
ldconfig < /dev/null > /dev/null 2>&1 || true
make[3]: Leaving directory '/home/vagrant/capnproto/c++'
 /bin/mkdir -p '/usr/local/lib/cmake/CapnProto'
 /usr/bin/install -c -m 644 cmake/CapnProtoConfig.cmake cmake/CapnProtoConfigVersion.cmake cmake/CapnProtoMacros.cmake cmake/CapnProtoTargets.cmake '/usr/local/lib/cmake/CapnProto'
 /bin/mkdir -p '/usr/local/include/capnp'
 /usr/bin/install -c -m 644 src/capnp/c++.capnp src/capnp/schema.capnp src/capnp/rpc.capnp src/capnp/rpc-twoparty.capnp src/capnp/persistent.capnp src/capnp/compat/json.capnp '/usr/local/include/capnp'
 /bin/mkdir -p '/usr/local/include/capnp'
 /usr/bin/install -c -m 644 src/capnp/c++.capnp.h src/capnp/common.h src/capnp/blob.h src/capnp/endian.h src/capnp/layout.h src/capnp/orphan.h src/capnp/list.h src/capnp/any.h src/capnp/message.h src/capnp/capability.h src/capnp/membrane.h src/capnp/schema.capnp.h src/capnp/schema-lite.h src/capnp/schema.h src/capnp/schema-loader.h src/capnp/schema-parser.h src/capnp/dynamic.h src/capnp/pretty-print.h src/capnp/serialize.h src/capnp/serialize-async.h src/capnp/serialize-packed.h src/capnp/serialize-text.h src/capnp/pointer-helpers.h src/capnp/generated-header-support.h src/capnp/raw-schema.h src/capnp/rpc-prelude.h src/capnp/rpc.h src/capnp/rpc-twoparty.h src/capnp/rpc.capnp.h src/capnp/rpc-twoparty.capnp.h src/capnp/persistent.capnp.h src/capnp/ez-rpc.h '/usr/local/include/capnp'
 /bin/mkdir -p '/usr/local/include/capnp/compat'
 /usr/bin/install -c -m 644 src/capnp/compat/json.h src/capnp/compat/json.capnp.h '/usr/local/include/capnp/compat'
 /bin/mkdir -p '/usr/local/include/kj'
 /usr/bin/install -c -m 644 src/kj/common.h src/kj/units.h src/kj/memory.h src/kj/refcount.h src/kj/array.h src/kj/vector.h src/kj/string.h src/kj/string-tree.h src/kj/encoding.h src/kj/exception.h src/kj/debug.h src/kj/arena.h src/kj/io.h src/kj/tuple.h src/kj/one-of.h src/kj/function.h src/kj/mutex.h src/kj/thread.h src/kj/threadlocal.h src/kj/filesystem.h src/kj/async-prelude.h src/kj/async.h src/kj/async-inl.h src/kj/time.h src/kj/timer.h src/kj/async-unix.h src/kj/async-win32.h src/kj/async-io.h src/kj/main.h src/kj/test.h src/kj/windows-sanity.h '/usr/local/include/kj'
 /bin/mkdir -p '/usr/local/include/kj/compat'
 /usr/bin/install -c -m 644 src/kj/compat/gtest.h src/kj/compat/url.h src/kj/compat/http.h '/usr/local/include/kj/compat'
 /bin/mkdir -p '/usr/local/include/kj/parse'
 /usr/bin/install -c -m 644 src/kj/parse/common.h src/kj/parse/char.h '/usr/local/include/kj/parse'
 /bin/mkdir -p '/usr/local/include/kj/std'
 /usr/bin/install -c -m 644 src/kj/std/iostream.h '/usr/local/include/kj/std'
 /bin/mkdir -p '/usr/local/lib/pkgconfig'
 /usr/bin/install -c -m 644 pkgconfig/capnp.pc pkgconfig/capnp-rpc.pc pkgconfig/capnp-json.pc pkgconfig/kj.pc pkgconfig/kj-async.pc pkgconfig/kj-http.pc pkgconfig/kj-test.pc '/usr/local/lib/pkgconfig'
make[2]: Leaving directory '/home/vagrant/capnproto/c++'
make[1]: Leaving directory '/home/vagrant/capnproto/c++'
(cd sandstorm && make CXX=g++)
make[1]: Entering directory '/opt/app/sandstorm'
make[1]: *** No rule to make target '/opt/sandstorm/latest/usr/include/sandstorm/*.capnp', needed by 'tmp/genfiles'.  Stop.
make[1]: Leaving directory '/opt/app/sandstorm'
Makefile:29: recipe for target 'sandstorm/bin/getPublicId' failed
make: *** [sandstorm/bin/getPublicId] Error 2
Connection to 127.0.0.1 closed.
Command failed with a non-zero exit status (2).

BR, xet7

Copied from original issue: wekan/wordpress-sandstorm#1

[xet7]

From @kentonv on February 11, 2018 6:18

It appears as if Sandstorm isn't installed inside the Vagrant image, which is weird. Maybe there's some more initialization that needs to be done by vagrant-spk? TBH I have never used vagrant-spk so I don't really know. Maybe @dwrensha (who created the Wordpress package) has an idea.

[xet7]

From @dwrensha on February 11, 2018 14:3

~Yeah I probably forgot to include the Sandstorm capnp files in the vagrant-spk build: https://github.com/dwrensha/wordpress-sandstorm/commit/17d397180e48282fe37d432aa8c9eb4d2786f4c8~

EDIT: hm. Those sandstorm files really should be there. When I do vagrant-spk up and vagrant-spk ssh, I see them in /opt/sandstorm/

[xet7]

From @dwrensha on February 11, 2018 14:31

When I try to build https://github.com/dwrensha/wordpress-sandstorm, it looks like I get farther than you did, but I still hit an error:

(cd sandstorm && make CXX=g++)
make[1]: Entering directory '/opt/app/sandstorm'
mkdir -p tmp
capnp compile --src-prefix=/opt/sandstorm/latest/usr/include -oc++:tmp /opt/sandstorm/latest/usr/include/sandstorm/*.capnp
*** Uncaught exception ***
kj/filesystem-disk-unix.c++:469: failed: mmap: Invalid argument
stack: 0x405b42 0x43a40e 0x43af2c 0x7f7d19cb38c5 0x7f7d19cb52c6
c++: plugin failed: exit code 1
Makefile:21: recipe for target 'tmp/genfiles' failed

I conjecture that this has to do with the fact that the /vagrant directory is synced between the host and guest.

[xet7]

From @kentonv on February 11, 2018 21:33

Ugh, apparently shared folders don't implement writable mmap(). I'm surprised they can get away with that. I guess maybe the capnp C++ output plugin needs to go back to using write(). Sad.

https://github.com/capnproto/capnproto/issues/635

[xet7]

This is the PR they used to fix their Vagrant 1.9.6 downloading the sandstorm version of the box: github.com/sandstorm-io/vagrant-spk/pull/209

Got that from @ocdtrekkie . I did change previously to probably broken box version.

[xet7]

From @JamborJan on March 6, 2018 19:38

Hi guys,

I have updated my fork of wordpress for Sandstorm (see https://github.com/JamborJan/wordpress-sandstorm/commit/cb93151f8b0901b6db1eb85aec7dbc28e230a4ef). Now I'm running into a different error:

(cd sandstorm && make CXX=g++)
make[1]: Entering directory '/opt/app/sandstorm'
generating capnp files...
/opt/sandstorm/latest/usr/include/sandstorm/payments.capnp:21:14-37: error: Import failed: /sandstorm/util.capnp
/opt/sandstorm/latest/usr/include/sandstorm/payments.capnp:28:29-42: error: Not defined: LocalizedText
/opt/sandstorm/latest/usr/include/sandstorm/payments.capnp:65:25-38: error: Not defined: LocalizedText
/opt/sandstorm/latest/usr/include/sandstorm/payments.capnp:73:17-30: error: Not defined: LocalizedText
/opt/sandstorm/latest/usr/include/sandstorm/payments.capnp:22:14-43: error: Import failed: /sandstorm/supervisor.capnp
/opt/sandstorm/latest/usr/include/sandstorm/payments.capnp:87:58-74: error: Not defined: SystemPersistent
/opt/sandstorm/latest/usr/include/sandstorm/payments.capnp:88:62-78: error: Not defined: SystemPersistent
/opt/sandstorm/latest/usr/include/sandstorm/payments.capnp:89:46-62: error: Not defined: SystemPersistent
Makefile:21: recipe for target 'tmp/genfiles' failed
make[1]: *** [tmp/genfiles] Error 1
make[1]: Leaving directory '/opt/app/sandstorm'
Makefile:26: recipe for target 'sandstorm/bin/getPublicId' failed
make: *** [sandstorm/bin/getPublicId] Error 2

I was not able yet to dig a bit deeper into these error messages but will do so as soon as possible. I wanted to leave a note here just in case someone of you can easily help or solve this.

PS: @xet7 as soon as we have figured out how we deal with the app key I will contribute to your repo. Until then I use my fork for testing reasons. Otherwise I would not be able to build packages and test them.

Thanks

[xet7]

From @ocdtrekkie on March 6, 2018 19:44

@JamborJan payments.capnp is a weird thing to show up here. I know that @kentonv just merged that back into Sandstorm proper, so it's possible something's missing there, and it's never been an issue with building before for that reason: https://github.com/sandstorm-io/sandstorm/pull/3042

[xet7]

@JamborJan

No problem, a moment ago I did fetch and merge changes from your repo to this repo, and also add some more modifications for php5 to php7. I will look at app key when bulding wp-sandstorm works, and there is something worth releasing.

[xet7]

From @JamborJan on March 7, 2018 8:3

@xet7

You also took my sandstorm-pkgdef.capnp. I can send you my private app key (over another secure channel) so that you are able tu build packages. If this is okay for you, I would be able to delete my fork and use yours instead.

[xet7]

@JamborJan

Yes it's OK. You can install desktop or mobile client of https://about.riot.im so we can enable end-to-end encryption, my Riot nickname is xet7wekan

[JamborJan]

Hey @ocdtrekkie, did I get your comment right and the payments.capnp will be analysed or is there something I need to do because I did something wrong when I tried to build the Wordpress app?

[ocdtrekkie]

I am just suspecting it's a new problem cause by the fact that payments.capnp is new to the Sandstorm repo. Kenton should be around this weekend and can probably explain why it's showing up as an error.

[kentonv]

It looks like there's a bug in the makefile:

capnp compile --src-prefix=/opt/sandstorm/latest/usr/include -oc++:tmp /opt/sandstorm/latest/usr/include/sandstorm/*.capnp

This line is missing an argument:

-I/opt/sandstorm/latest/usr/include

It fails on payments.capnp because that file contains absolute imports rather than relative, which is vestigial since it used to live in Blackrock rather than Sandstorm. I will change it to use relative imports in today's release, but it may also make sense to fix the Wordpress app's Makefile.

[xet7]

With latest changes in git I have this progress, after installing vagrant:

git clone git@github.com:wpsandstorm/wordpress-sandstorm.git

cd wordpress-sandstorm

./build-spk-in-vagrant.sh

Then inside VirtualBox VM window, where is username: vagrant password: vagrant

sudo apt install task-lxde-desktop

startx

Then on Firefox ESR http://local.sandstorm.io:6080

[xet7]

Related https://github.com/sandstorm-io/sandstorm/issues/3054

[JamborJan]

I guess it's about the new PHP7 version according to the error log. I tried to run and enter a grain to figure out what is wrong but I ran into another issue (see https://github.com/sandstorm-io/vagrant-spk/issues/213).

I've realised that there are some php config files in the repo. There is one line which seems to be wrong for PHP7, but it seems these config files are not used at all.

@xet7 I guess you added this symlink to solve the issue, which didn't work out. Right?

...** SANDSTORM SUPERVISOR: Starting up grain. Sandbox type: userns
[12-Mar-2018 06:48:59] NOTICE: [pool www] 'user' directive is ignored when FPM is not running as root
[12-Mar-2018 06:48:59] NOTICE: [pool www] 'group' directive is ignored when FPM is not running as root
[12-Mar-2018 06:48:59] ERROR: unable to bind listening socket for address '/run/php/php7.0-fpm.sock': No such file or directory (2)
[12-Mar-2018 06:48:59] ERROR: FPM initialization failed
started php-fpm. status code: 70
started nginx. status code: 0

[xet7]

@JamborJan

Yes, it did not work, it caused error about symlink to symlink, or something like that.

[xet7]

@JamborJan

I removed that symlink from repo.

[JamborJan]

We are able to build the app again. For details see https://github.com/wpsandstorm/wordpress-sandstorm/commit/fec80241dd53a8e70074b2706835e1bdbaeb62d9

Is there anything left we need to clean up or can we close this issue @xet7 ?

[ocdtrekkie]

I am super excited that progress has been made here!