ctubio / Krypto-trading-bot

Self-hosted crypto trading bot (automated high frequency market making) written in C++
https://127.0.0.1:3000
Other
3.39k stars 827 forks source link

How to compile in a chroot enviroment #112

Closed Jorman closed 7 years ago

Jorman commented 7 years ago

Hi, I've an old Qnap nas, on it I've a chroot enviroment with debian

From the root user (I had some problem with the user so I use the root instead) I cloned the source and the make process seems fine

root@Qnap:~/K# make reinstall
rm -rf app
git checkout .
git fetch
git merge FETCH_HEAD
Already up-to-date.
make[1]: Entering directory '/root/K'
make[2]: Entering directory '/root/K'
test -n "`command -v apt-get`" && sudo apt-get -y install g++ build-essential automake autoconf libtool libxml2 libxml2-dev zlib1g-dev libsqlite3-dev libcurl4-openssl-dev libssl-dev openssl stunnel python curl gzip imagemagick\
|| (test -n "`command -v yum`" && sudo yum -y install gcc-c++ automake autoconf libtool libxml2 libxml2-devel zlib-devel sqlite-devel libcurl-devel openssl openssl-devel zlib-devel stunnel python curl gzip imagemagick) \
|| (test -n "`command -v brew`" && (xcode-select --install || :) && (brew install automake autoconf libxml2 sqlite openssl zlib libuv stunnel python curl gzip imagemagick || brew upgrade || :))
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'stunnel4' instead of 'stunnel'
autoconf is already the newest version (2.69-10).
automake is already the newest version (1:1.15-6).
build-essential is already the newest version (12.3).
curl is already the newest version (7.52.1-5).
libcurl4-openssl-dev is already the newest version (7.52.1-5).
g++ is already the newest version (4:6.3.0-4).
gzip is already the newest version (1.6-5+b1).
imagemagick is already the newest version (8:6.9.7.4+dfsg-11).
libtool is already the newest version (2.4.6-2).
libxml2 is already the newest version (2.9.4+dfsg1-2.2).
libxml2-dev is already the newest version (2.9.4+dfsg1-2.2).
libssl-dev is already the newest version (1.1.0f-3).
openssl is already the newest version (1.1.0f-3).
python is already the newest version (2.7.13-2).
libsqlite3-dev is already the newest version (3.16.2-5).
stunnel4 is already the newest version (3:5.39-2).
zlib1g-dev is already the newest version (1:1.2.8.dfsg-5).
The following packages were automatically installed and are no longer required:
  libaudcore2 libavcodec56 libavformat56 libavresample2 libavutil54 libbinio1ldbl libdns-export100 libegl1-mesa-drivers libgif4 libgnutls-deb0-28
  libgphoto2-port10 libhogweed2 libhunspell-1.3-0 libicu52 libirs-export91 libisc-export95 libisccfg-export90 libjson-c2 liblircclient0 libllvm3.5
  liblognorm1 libmicrohttpd10 libnettle4 libopenjpeg5 libpcsclite1 libpng12-0 libprocps3 libprotobuf9 libschroedinger-1.0-0 libsdl1.2debian libvpx1
  libx264-142 libxtables10 xbase-clients
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
sudo mkdir -p /data/db/
sudo chown `id -u` /data/db
make dist
make[3]: Entering directory '/root/K'
mkdir -p build app/server/lib
make quickfix
make[4]: Entering directory '/root/K'
(test -f /usr/local/lib/libquickfix.so || test -f /usr/local/lib/libquickfix.dylib) || ( \
curl -L https://github.com/quickfix/quickfix/archive/v.1.14.4.tar.gz | tar xz -C build  \
&& cd build/quickfix-v.1.14.4 && ./bootstrap && ./configure && make                     \
&& sudo make install && sudo cp config.h /usr/local/include/quickfix/                  \
&& (test -f /sbin/ldconfig && sudo ldconfig || :)                                      )
make[4]: Leaving directory '/root/K'
make json
make[4]: Entering directory '/root/K'
test -f build/json-v2.1.1/json.h || (mkdir -p build/json-v2.1.1 && curl -L https://github.com/nlohmann/json/releases/download/v2.1.1/json.hpp -o build/json-v2.1.1/json.h)
make[4]: Leaving directory '/root/K'
make png16
make[4]: Entering directory '/root/K'
test -d build/libpng-1.6.31 || (curl -L https://github.com/glennrp/libpng/archive/v1.6.31.tar.gz | tar xz -C build && cd build/libpng-1.6.31 && ./autogen.sh && ./configure --prefix=/root/K/build/libpng-1.6.31 && make && sudo make install)
cp build/libpng-1.6.31/lib/libpng16* app/server/lib
make[4]: Leaving directory '/root/K'
make uws
make[4]: Entering directory '/root/K'
test -d build/uWebSockets-0.14.3 || curl -L https://github.com/uNetworking/uWebSockets/archive/v0.14.3.tar.gz | tar xz -C build
make[4]: Leaving directory '/root/K'
for K in dist/lib/*K*; do chmod +x $K && cp $K app/server/lib; done
make[3]: Leaving directory '/root/K'
make gdax -s
make[3]: Entering directory '/root/K'
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA
verify return:1
depth=0 C = US, ST = California, L = San Francisco, O = "Coinbase, Inc.", CN = *.gdax.com
verify return:1
DONE
make[3]: Leaving directory '/root/K'
make[3]: Entering directory '/root/K'
make[3]: Leaving directory '/root/K'
make[2]: Leaving directory '/root/K'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.1.2 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.1.2: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"ia32"})

added 114 packages in 46.206s
make[2]: Entering directory '/root/K'
Building server files..
./node_modules/.bin/tsc --alwaysStrict -t ES6 -m commonjs --outDir app src/server/*.ts src/server/*/*.ts src/share/*.ts
DONE
Building client dynamic files..
./node_modules/.bin/tsc --alwaysStrict --experimentalDecorators -t ES6 -m commonjs --outDir app/pub/js src/client/*.ts src/share/*.ts
DONE
Building client static files..
cp -R src/pub/* app/pub/
DONE
Building client bundle file..
./node_modules/.bin/browserify -t [ babelify --presets [ babili es2016 ] ] app/pub/js/client/main.js app/pub/js/lib/*.js | ./node_modules/.bin/uglifyjs | gzip > app/pub/js/client/bundle.min.js
DONE
make[2]: Leaving directory '/root/K'
make[1]: Leaving directory '/root/K'

  Benchmark C shared objects
    ✓ JS round is faster
    ✓ C stdev is faster

  EWMATargetPositionCalculator
    ✓ Should work

  3 passing (95ms)

./node_modules/.bin/forever restartall
info:    No forever processes running

..done! Please refresh the GUI if is currently opened in your browser.

but on start I've some problem

root@Qnap:~/K# valgrind node K
==17182== Memcheck, a memory error detector
==17182== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==17182== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==17182== Command: node K
==17182==
K.linux.57
/root/K/app/server/main.js:14
            throw new Error(e);
            ^

Error: Error: /root/K/app/server/lib/K.linux.57.node: wrong ELF class: ELFCLASS64
    at bindings (/root/K/app/server/main.js:14:19)
    at Object.<anonymous> (/root/K/app/server/main.js:16:3)
    at Module._compile (module.js:569:30)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:503:32)
    at tryModuleLoad (module.js:466:12)
    at Function.Module._load (module.js:458:3)
    at Module.require (module.js:513:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/root/K/K.js:2:1)
==17182==
==17182== HEAP SUMMARY:
==17182==     in use at exit: 345,966 bytes in 1,807 blocks
==17182==   total heap usage: 268,527 allocs, 266,720 frees, 205,313,153 bytes allocated
==17182==
==17182== LEAK SUMMARY:
==17182==    definitely lost: 0 bytes in 0 blocks
==17182==    indirectly lost: 0 bytes in 0 blocks
==17182==      possibly lost: 820 bytes in 10 blocks
==17182==    still reachable: 345,146 bytes in 1,797 blocks
==17182==                       of which reachable via heuristic:
==17182==                         newarray           : 3,356 bytes in 4 blocks
==17182==         suppressed: 0 bytes in 0 blocks
==17182== Rerun with --leak-check=full to see details of leaked memory
==17182==
==17182== For counts of detected and suppressed errors, rerun with: -v
==17182== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

I searched a bit and seems that is some about the version (32 or 64bit) I made some test without success! Here some info about the chroot environment

root@Qnap:~/K# cat /etc/*-release
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
root@Qnap:~/K# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 9.1 (stretch)
Release:        9.1
Codename:       stretch
root@Qnap:~/K# uname -a
Linux Qnap 3.4.6 #1 SMP Mon Jul 3 01:22:05 CST 2017 x86_64 GNU/Linux
root@Qnap:~/K# node -v
v8.2.1

Anyone have some skill to do some guess to try to compile it again?

ctubio commented 7 years ago

root user is not supported (see https://github.com/ctubio/Krypto-trading-bot/issues/110)

Jorman commented 7 years ago

So I've to use a user inside the chroot! Damm, lol, node have a little problem with that https://github.com/npm/npm/issues/17965

So I'll try on my nas in future thank you

Jorman commented 7 years ago

Ok, I successfully reinstlled K under a non-root use and with node v7.10.1 I don't have any error for node but I can't start K.

==30371== Memcheck, a memory error detector
==30371== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==30371== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==30371== Command: node K
==30371==
K.linux.51
/home/web/K/app/server/main.js:14
            throw new Error(e);
            ^

Error: Error: /home/web/K/app/server/lib/K.linux.51.node: wrong ELF class: ELFCLASS64
    at bindings (/home/web/K/app/server/main.js:14:19)
    at Object.<anonymous> (/home/web/K/app/server/main.js:16:3)
    at Module._compile (module.js:571:32)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:488:32)
    at tryModuleLoad (module.js:447:12)
    at Function.Module._load (module.js:439:3)
    at Module.require (module.js:498:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (/home/web/K/K.js:2:1)
==30371==
==30371== HEAP SUMMARY:
==30371==     in use at exit: 367,019 bytes in 2,347 blocks
==30371==   total heap usage: 47,260 allocs, 44,913 frees, 81,528,373 bytes allocated
==30371==
==30371== LEAK SUMMARY:
==30371==    definitely lost: 0 bytes in 0 blocks
==30371==    indirectly lost: 0 bytes in 0 blocks
==30371==      possibly lost: 820 bytes in 10 blocks
==30371==    still reachable: 366,199 bytes in 2,337 blocks
==30371==                       of which reachable via heuristic:
==30371==                         newarray           : 3,276 bytes in 4 blocks
==30371==         suppressed: 0 bytes in 0 blocks
==30371== Rerun with --leak-check=full to see details of leaked memory
==30371==
==30371== For counts of detected and suppressed errors, rerun with: -v
==30371== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

What do you think @ctubio

ctubio commented 7 years ago

i think you are running a 32bit operating system, can you show your versions? something liiike:

$ uname -a
Linux debian 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2 (2017-04-30) x86_64 GNU/Linux
$ cat /etc/issue
Debian GNU/Linux 8 \n \l
$ node -v
v8.2.1
$ g++ --version
g++ (Debian 4.9.2-10) 4.9.2

if so, i will try to fix 32bit systems (meanwhile a solution is to use a 64bit operating system)

can you confirm what is your OS? thanks'

Jorman commented 7 years ago

Yep

web@Qnap:~$ uname -a
Linux Qnap 3.4.6 #1 SMP Thu Jul 27 01:10:21 CST 2017 x86_64 GNU/Linux
web@Qnap:~$ cat /etc/issue
Debian GNU/Linux 9 \n \l

web@Qnap:~$ node -v
v7.10.1
web@Qnap:~$ g++ --version
g++ (Debian 6.3.0-18) 6.3.0 20170516
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

web@Qnap:~$

For what I know, on qnap like the one I've the kernel is 64bit but the system is 32. Let me know if you need more info about it thanks

ctubio commented 7 years ago

ok thanks, will try to fix soOn