search

maiha / rcm.cr

Redis Cluster Manager in Crystal
MIT License
45 stars 5 forks source link

i make rcm.cr-v.0.6.6 fail，the following is the error log #9

Closed soulaxin closed 7 years ago

soulaxin commented 7 years ago 
crystal build --release src/bin/rcm.cr -o bin/rcm --link-flags "-static" -D without_openssl
Error in src/bin/rcm.cr:365: instantiating 'Rcm::Main:Class#run()'

Rcm::Main.run
          ^~~

in src/bin/rcm.cr:6: expanding macro

  include Opts
  ^

in macro 'included' /home/yaxin/redis/rcm.cr-0.6.6/lib/opts/src/opts.cr:138, line 1:

>  1.     def self.run(argv = ARGV)
   2.       main = new
   3.       main.argv = argv
   4.       main.args                        # kick parse!
   5.       main.exit(main.show_usage) if main.responds_to?(:help) && main.help
   6.       main.exit(main.show_version) if main.responds_to?(:version) && main.version
   7.       main.run
   8.     rescue err
   9.       STDERR.puts err.to_s.colorize(:red)
  10.       exit 1
  11.     end
  12. 
  13.     def show_usage
  14.       USAGE.gsub(/{{program}}/, PROGRAM).gsub(/{{version}}/, VERSION).gsub(/{{args}}/, ARGS).gsub(/{{options}}/, new_option_parser.to_s)
  15.     end
  16. 
  17.     def show_version
  18.       "#{PROGRAM} #{VERSION}"
  19.     end
  20.   

instantiating 'run(Array(String))'
in src/bin/rcm.cr:6: expanding macro

  include Opts
  ^

in macro 'included' /home/yaxin/redis/rcm.cr-0.6.6/lib/opts/src/opts.cr:138, line 7:

   1.     def self.run(argv = ARGV)
   2.       main = new
   3.       main.argv = argv
   4.       main.args                        # kick parse!
   5.       main.exit(main.show_usage) if main.responds_to?(:help) && main.help
   6.       main.exit(main.show_version) if main.responds_to?(:version) && main.version
>  7.       main.run
   8.     rescue err
   9.       STDERR.puts err.to_s.colorize(:red)
  10.       exit 1
  11.     end
  12. 
  13.     def show_usage
  14.       USAGE.gsub(/{{program}}/, PROGRAM).gsub(/{{version}}/, VERSION).gsub(/{{args}}/, ARGS).gsub(/{{options}}/, new_option_parser.to_s)
  15.     end
  16. 
  17.     def show_version
  18.       "#{PROGRAM} #{VERSION}"
  19.     end
  20.   

instantiating 'Rcm::Main#run()'
in src/bin/rcm.cr:185: instantiating 'Rcm::Httpd::Server:Class#new(Redis::Client, Redis::Cluster::Bootstrap)'

      server = Httpd::Server.new(client, Bootstrap.parse(listen))
                             ^~~

in src/rcm/httpd/server.cr:11: undefined constant Kemal::Middleware::HTTPBasicAuth

        auth_handler = Kemal::Middleware::HTTPBasicAuth.new("redis", pass)
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
maiha commented 7 years ago

Oh, It's my fault. v0.6.6 depends on master branch of kemal which removes HTTPBasicAuth recently. I'll release v0.6.7 soon.

maiha commented 7 years ago

@soulaxin Released v0.7.0. Could you try it? Thank you for bug reporting!

soulaxin commented 7 years ago

i make the version v0.7.1, and found anther error

rcm.cr-0.7.1]$ make
crystal build --release src/bin/rcm.cr -o bin/rcm --link-flags "-static" -D without_openssl
fork: Cannot allocate memory
0x53f75e: ??? at ??
0x6ac195: ??? at ??
0x1133eda: ??? at ??
0x58ecaa: ??? at ??
0x56023b: main at ??
0x7f237a6f0af5: __libc_start_main at ??
0x53b7e1: ??? at ??

Error: you've found a bug in the Crystal compiler. Please open an issue, including source code that will allow us to reproduce the bug: https://github.com/crystal-lang/crystal/issues
make: *** [rcm] Error 1

here is my memory

 rcm.cr-0.7.1]$ free -h
             total       used       free     shared    buffers     cached
Mem:          992M       626M       366M        50M        39M        66M
-/+ buffers/cache:       519M       473M
Swap:           0B         0B         0B
maiha commented 7 years ago

Hmm, crystal compiler would really need much more memory than that you have. It would pass when removing '--release' flag although the created binary will kill performance at some level.

crystal build src/bin/rcm.cr -o bin/rcm --link-flags "-static" -D without_openssl

I'll try compile this on 1GB machine. Thanks.

soulaxin commented 7 years ago

i add 2GB swap memory, and make fail also, here is another error.

rcm.cr-0.7.1]$ make
crystal build --release src/bin/rcm.cr -o bin/rcm --link-flags "-static" -D without_openssl
/usr/bin/ld: cannot find -lz
/usr/bin/ld: cannot find -lm
/usr/bin/ld: cannot find -lpthread
/usr/bin/ld: cannot find -lrt
/usr/bin/ld: cannot find -lncursesw
/usr/bin/ld: cannot find -lgpm
/usr/bin/ld: cannot find -ldl
/usr/bin/ld: cannot find -lc
collect2: error: ld returned 1 exit status
Error: execution of command failed with code: 1: `cc -o "/home/yaxin/redis/rcm.cr/rcm.cr-0.7.1/bin/rcm" "${@}" -static -rdynamic  -lyaml -lz -lpcre -lm -lgc -lpthread /opt/crystal/src/ext/libcrystal.a -levent -lrt -lncursesw -lgpm -ldl -L/usr/lib -L/usr/local/lib`
make: *** [rcm] Error 1
maiha commented 7 years ago

Yup, we need those libraries(*.a) to compile rcm statically. What OS and Distribution are you using? I'll check how to install them.

If you don't need static binary, please try dynamic compilation that depends less libraries I think. crystal build --release src/bin/rcm.cr -o bin/rcm -D without_openssl

soulaxin commented 7 years ago

OS version

Linux version 3.10.0-123.9.3.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.2 20140120 (Red Hat 4.8.2-16) (GCC) )

dynamic compilation also make fail

rcm.cr-0.7.1]$ crystal build --release src/bin/rcm.cr -o bin/rcm -D without_openssl
/usr/bin/ld: cannot find -lz
collect2: error: ld returned 1 exit status
Error: execution of command failed with code: 1: `cc -o "/home/yaxin/redis/rcm.cr/rcm.cr-0.7.1/bin/rcm" "${@}"  -rdynamic  -lyaml -lz -lpcre -lm -lgc -lpthread /opt/crystal/src/ext/libcrystal.a -levent -lrt -lncursesw -lgpm -ldl -L/usr/lib -L/usr/local/lib`
maiha commented 7 years ago

Ok, it seems CentOS7. Please try following packages.

yum install zlib-devel ncurses-devel gpm-devel libyaml-devel

This should provide dynamic compilation. If you want static one, please install required libraries in official doc. https://github.com/crystal-lang/crystal/wiki/All-required-libraries

Thanks.

soulaxin commented 7 years ago

i executed this command. yum install zlib-devel ncurses-devel gpm-devel libyaml-devel and then dynamic compilation success.

and then i try to static compilation, so i fllow the link, install bdwgc, then make rcm.cr, here is the error:

rcm.cr-0.7.1]$ make | head -5
crystal build --release src/bin/rcm.cr -o bin/rcm --link-flags "-static" -D without_openssl
/home/yaxin/.cache/crystal/home-yaxin-redis-rcm.cr-rcm.cr-0.7.1-src-bin-rcm.cr/macro-run-_opt_crystal_src_ecr_process.cr: symbol lookup error: /home/yaxin/.cache/crystal/home-yaxin-redis-rcm.cr-rcm.cr-0.7.1-src-bin-rcm.cr/macro-run-_opt_crystal_src_ecr_process.cr: undefined symbol: GC_get_push_other_roots
Error in src/bin/rcm.cr:1: while requiring "../rcm"

require "../rcm"
^
make: *** [rcm] Error 1

and now dynamic compilation also compile fail with same error

rcm.cr-0.7.1]$ crystal build --release src/bin/rcm.cr -o bin/rcm -D without_openssl | head -5
/home/yaxin/.cache/crystal/home-yaxin-redis-rcm.cr-rcm.cr-0.7.1-src-bin-rcm.cr/macro-run-_opt_crystal_src_ecr_process.cr: symbol lookup error: /home/yaxin/.cache/crystal/home-yaxin-redis-rcm.cr-rcm.cr-0.7.1-src-bin-rcm.cr/macro-run-_opt_crystal_src_ecr_process.cr: undefined symbol: GC_get_push_other_roots
Error in src/bin/rcm.cr:1: while requiring "../rcm"

require "../rcm"
maiha commented 7 years ago

Oh, how do you install BOEHM gc? From git source? It seems crystal needs boehm-7.6. And centos7 has 7.2 in its repos.

# yum list gc-devel
gc-devel.x86_64                         7.2d-7.el7                          base
soulaxin commented 7 years ago

from git source, here is make check and make install 's log.

make check
make[1]: Entering directory `/home/yaxin/redis/rcm.cr/dependens/bdwgc'
make  libstaticrootslib_test.la libstaticrootslib2_test.la cordtest gctest leaktest middletest smashtest hugetest realloc_test staticrootstest  threadleaktest threadkey_test subthreadcreate_test initsecondarythread_test  disclaim_test disclaim_bench
make[2]: Entering directory `/home/yaxin/redis/rcm.cr/dependens/bdwgc'
make[2]: `libstaticrootslib2_test.la' is up to date.
make[2]: `cordtest' is up to date.
make[2]: `gctest' is up to date.
make[2]: `leaktest' is up to date.
make[2]: `middletest' is up to date.
make[2]: `smashtest' is up to date.
make[2]: `hugetest' is up to date.
make[2]: `realloc_test' is up to date.
make[2]: `staticrootstest' is up to date.
make[2]: `threadleaktest' is up to date.
make[2]: `threadkey_test' is up to date.
make[2]: `subthreadcreate_test' is up to date.
make[2]: `initsecondarythread_test' is up to date.
make[2]: `disclaim_test' is up to date.
make[2]: `disclaim_bench' is up to date.
make[2]: Leaving directory `/home/yaxin/redis/rcm.cr/dependens/bdwgc'
make  check-TESTS
make[2]: Entering directory `/home/yaxin/redis/rcm.cr/dependens/bdwgc'
make[3]: Entering directory `/home/yaxin/redis/rcm.cr/dependens/bdwgc'
PASS: cordtest
PASS: gctest
PASS: leaktest
PASS: middletest
PASS: smashtest
PASS: hugetest
PASS: realloc_test
PASS: staticrootstest
PASS: threadleaktest
PASS: threadkey_test
PASS: subthreadcreate_test
PASS: initsecondarythread_test
PASS: disclaim_test
PASS: disclaim_bench
make[4]: Entering directory `/home/yaxin/redis/rcm.cr/dependens/bdwgc'
make[5]: Entering directory `/home/yaxin/redis/rcm.cr/dependens/bdwgc'
make[5]: Leaving directory `/home/yaxin/redis/rcm.cr/dependens/bdwgc'
make[4]: Leaving directory `/home/yaxin/redis/rcm.cr/dependens/bdwgc'
============================================================================
Testsuite summary for gc 7.7.0
============================================================================
# TOTAL: 14
# PASS:  14
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
make[3]: Leaving directory `/home/yaxin/redis/rcm.cr/dependens/bdwgc'
make[2]: Leaving directory `/home/yaxin/redis/rcm.cr/dependens/bdwgc'
make[1]: Leaving directory `/home/yaxin/redis/rcm.cr/dependens/bdwgc'

make install #with user root

make[1]: Entering directory `/home/yaxin/redis/rcm.cr/dependens/bdwgc'
make[2]: Entering directory `/home/yaxin/redis/rcm.cr/dependens/bdwgc'
 /usr/bin/mkdir -p '/usr/local/lib'
 /bin/sh ./libtool   --mode=install /usr/bin/install -c   libgc.la libcord.la '/usr/local/lib'
libtool: install: /usr/bin/install -c .libs/libgc.so.1.0.3 /usr/local/lib/libgc.so.1.0.3
libtool: install: (cd /usr/local/lib && { ln -s -f libgc.so.1.0.3 libgc.so.1 || { rm -f libgc.so.1 && ln -s libgc.so.1.0.3 libgc.so.1; }; })
libtool: install: (cd /usr/local/lib && { ln -s -f libgc.so.1.0.3 libgc.so || { rm -f libgc.so && ln -s libgc.so.1.0.3 libgc.so; }; })
libtool: install: /usr/bin/install -c .libs/libgc.lai /usr/local/lib/libgc.la
libtool: install: (cd /home/yaxin/redis/rcm.cr/dependens/bdwgc; /bin/sh /home/yaxin/redis/rcm.cr/dependens/bdwgc/libtool  --tag CC --mode=relink gcc -fexceptions -Wall -Wextra -Wpedantic -Wno-long-long -g -O2 -fno-strict-aliasing -version-info 1:3:0 -no-undefined -o libcord.la -rpath /usr/local/lib cord/libcord_la-cordbscs.lo cord/libcord_la-cordprnt.lo cord/libcord_la-cordxtra.lo ./libgc.la )
libtool: relink: gcc -shared  -fPIC -DPIC  cord/.libs/libcord_la-cordbscs.o cord/.libs/libcord_la-cordprnt.o cord/.libs/libcord_la-cordxtra.o   -Wl,-rpath -Wl,/usr/local/lib -L/usr/local/lib -lgc -lpthread -ldl  -O2   -Wl,-soname -Wl,libcord.so.1 -o .libs/libcord.so.1.0.3
libtool: install: /usr/bin/install -c .libs/libcord.so.1.0.3T /usr/local/lib/libcord.so.1.0.3
libtool: install: (cd /usr/local/lib && { ln -s -f libcord.so.1.0.3 libcord.so.1 || { rm -f libcord.so.1 && ln -s libcord.so.1.0.3 libcord.so.1; }; })
libtool: install: (cd /usr/local/lib && { ln -s -f libcord.so.1.0.3 libcord.so || { rm -f libcord.so && ln -s libcord.so.1.0.3 libcord.so; }; })
libtool: install: /usr/bin/install -c .libs/libcord.lai /usr/local/lib/libcord.la
libtool: install: /usr/bin/install -c .libs/libgc.a /usr/local/lib/libgc.a
libtool: install: chmod 644 /usr/local/lib/libgc.a
libtool: install: ranlib /usr/local/lib/libgc.a
libtool: install: /usr/bin/install -c .libs/libcord.a /usr/local/lib/libcord.a
libtool: install: chmod 644 /usr/local/lib/libcord.a
libtool: install: ranlib /usr/local/lib/libcord.a
libtool: finish: PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin:/sbin" ldconfig -n /usr/local/lib
----------------------------------------------------------------------
Libraries have been installed in:
   /usr/local/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the `LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the `LD_RUN_PATH' environment variable
     during linking
   - use the `-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to `/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
 /usr/bin/mkdir -p '/usr/local/share/gc'
 /usr/bin/install -c -m 644 AUTHORS README.md doc/README.DGUX386 doc/README.Mac doc/README.OS2 doc/README.amiga doc/README.arm.cross doc/README.autoconf doc/README.cmake doc/README.cords doc/README.darwin doc/README.environment doc/README.ews4800 doc/README.hp doc/README.linux doc/README.macros doc/README.rs6000 doc/README.sgi doc/README.solaris2 doc/README.symbian doc/README.uts doc/README.win32 doc/README.win64 doc/debugging.html doc/finalization.html doc/gc.man doc/gcdescr.html doc/gcinterface.html doc/leak.html doc/overview.html doc/porting.html doc/scale.html doc/simple_example.html doc/tree.html '/usr/local/share/gc'
 /usr/bin/mkdir -p '/usr/local/include'
 /usr/bin/install -c -m 644 include/extra/gc.h '/usr/local/include'
 /usr/bin/mkdir -p '/usr/local/lib/pkgconfig'
 /usr/bin/install -c -m 644 bdw-gc.pc '/usr/local/lib/pkgconfig'
 /usr/bin/mkdir -p '/usr/local/include/gc'
 /usr/bin/install -c -m 644 include/gc.h include/gc_allocator.h include/gc_backptr.h include/gc_config_macros.h include/gc_disclaim.h include/gc_gcj.h include/gc_inline.h include/gc_mark.h include/gc_pthread_redirects.h include/gc_tiny_fl.h include/gc_typed.h include/gc_version.h include/javaxfc.h include/leak_detector.h include/weakpointer.h include/cord.h include/cord_pos.h include/ec.h '/usr/local/include/gc'
make[2]: Leaving directory `/home/yaxin/redis/rcm.cr/dependens/bdwgc'
make[1]: Leaving directory `/home/yaxin/redis/rcm.cr/dependens/bdwgc'
maiha commented 7 years ago

@soulaxin Thank you for install-log. It seems BOEHM-7.7 has been successfully installed to /usr/local/lib/.

But compilation failed with this. It's weird.

undefined symbol: GC_get_push_other_roots

There are two possibilities.

  1. /usr/local/lib/libgc.a is not installed one by some reason. You can validate the libgc is what we expect by following command. 
    % nm -o /usr/local/lib/libgc.a |grep GC_get_push_other_roots
/usr/local/lib/libgc.a:os_dep.o:0000000000000b50 T GC_get_push_other_roots

Here, T means defined function.

  1. other libgc would be used by compiler Please check that /etc/ld.so.conf or /etc/ld.so.conf.d/* contain /usr/local/lib. This is a cc issue rather than crystal one.

Finally, you can fix undefined symbol problem. But, It would be other link error like -lz not found. Because libz.a is needed for static link. And this is also cc issue.

Thanks.