search

gigasquid / clojure-mxnet

Clojure Package for MXNET
69 stars 3 forks source link

Test failing under Ubuntu 18.04 64bit #5

Open whilo opened 6 years ago

whilo commented 6 years ago

I have just tried the MNIST imclassification example, when I figured out that my model was not learning (accuracy stays at random). Running the tests yielded:

christian@gramsci ~/D/clojure-mxnet> lein test
INFO  MXNetJVM: Try loading mxnet-scala from native path.
INFO  MXNetJVM: Try loading mxnet-scala-linux-x86_64-gpu from native path.
INFO  MXNetJVM: Try loading mxnet-scala-linux-x86_64-cpu from native path.
WARN  MXNetJVM: MXNet Scala native library not found in path. Copying native library from the archive. Consider installing the library somewhere in the path (for Windows: PATH, for Linux: LD_LIBRARY_PATH), or specifying by Java cmd option -Djava.library.path=[lib path].
INFO  org.apache.mxnet.util.NativeLibraryLoader: Loading libmxnet-scala.so from /lib/native/ copying to mxnet-scala
[00:38:47] src/io/iter_mnist.cc:113: MNISTIter: load 60000 images, shuffle=1, shape=[100,1,28,28]
[00:38:47] src/io/iter_mnist.cc:113: MNISTIter: load 10000 images, shuffle=1, shape=[100,1,28,28]
WARN  org.apache.mxnet.WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.NDArray was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing

lein test org.apache.clojure-mxnet.conv-test
WARN  org.apache.mxnet.WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.Symbol was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing
Epoch  0  Train- [accuracy 0.09871667]
Epoch  0  Time cost- 6805
Epoch  0  Validation-  [accuracy 0.098]
Score [accuracy 0.098]

lein test :only org.apache.clojure-mxnet.conv-test/test-conv

FAIL in (test-conv) (conv_test.clj:71)
expected: (< 0.92 (last score))
  actual: (not (< 0.92 0.098))

lein test org.apache.clojure-mxnet.executor-test

lein test org.apache.clojure-mxnet.io-test
[00:38:57] src/io/iter_mnist.cc:110: MNISTIter: load 60000 images, shuffle=1, shape=(100,784)
[00:38:59] src/io/iter_mnist.cc:110: MNISTIter: load 60000 images, shuffle=1, shape=(100,784)
WARN  org.apache.mxnet.WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.Executor was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing
[00:38:59] src/io/iter_image_recordio_2.cc:170: ImageRecordIOParser2: data/cifar/train.rec, use 1 threads for decoding..
[00:39:02] src/io/iter_mnist.cc:110: MNISTIter: load 60000 images, shuffle=1, shape=(100,784)
[00:39:03] src/io/iter_mnist.cc:110: MNISTIter: load 60000 images, shuffle=1, shape=(100,784)
[00:39:05] src/io/iter_mnist.cc:110: MNISTIter: load 60000 images, shuffle=1, shape=(100,784)

lein test org.apache.clojure-mxnet.kvstore-test

lein test org.apache.clojure-mxnet.module-test
INFO  org.apache.mxnet.module.Module: Saved checkpoint to test-0000.params
INFO  org.apache.mxnet.module.Module: Saved optimizer state to test-0000.states
INFO  org.apache.mxnet.Model: Auto - select kvstore type = local_update_cpu

lein test :only org.apache.clojure-mxnet.module-test/test-module-reshape

FAIL in (test-module-reshape) (module_test.clj:129)
expected: (every? (fn* [p1__4712#] (= -1.0 p1__4712#)) (-> (m/params mod) (first) (get "fc_bias") (ndarray/->vec)))
  actual: (not (every? #object[org.apache.clojure_mxnet.module_test$fn__4714$fn__4719$fn__4720 0x51f009ef "org.apache.clojure_mxnet.module_test$fn__4714$fn__4719$fn__4720@51f009ef"] [0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0]))
INFO  org.apache.mxnet.Model: Auto - select kvstore type = local_update_cpu
INFO  org.apache.mxnet.module.Module: Saved checkpoint to test-0000.params
INFO  org.apache.mxnet.module.Module: Saved optimizer state to test-0000.states

lein test org.apache.clojure-mxnet.ndarray-test

lein test :only org.apache.clojure-mxnet.ndarray-test/test-arrange

ERROR in (test-arrange) (Base.scala:131)
expected: (= [0.0 0.0 0.5 0.5 1.0 1.0 1.5 1.5 2.0 2.0 2.5 2.5 3.0 3.0 3.5 3.5 4.0 4.0 4.5 4.5] (->vec (ndarray/arange start stop {:step step, :repeat repeat})))
[00:39:05] src/ndarray/./ndarray_function-inl.h:97: The operator onehot_encode is deprecated; use one_hot instead.
  actual: org.apache.mxnet.MXNetError: [00:39:05] src/operator/tensor/./init_op.h:468: Check failed: param.step != 0 (0 vs. 0) Range does not support step=0, received 0

Stack trace returned 9 entries:
[bt] (0) /tmp/mxnet3894755353450822729/mxnet-scala(dmlc::StackTrace[abi:cxx11]()+0x1bc) [0x7f8ddcc8299c]
[bt] (1) /tmp/mxnet3894755353450822729/mxnet-scala(dmlc::LogMessageFatal::~LogMessageFatal()+0x28) [0x7f8ddcc83bb8]
[bt] (2) /tmp/mxnet3894755353450822729/mxnet-scala(mxnet::op::RangeShape(nnvm::NodeAttrs const&, std::vector<nnvm::TShape, std::allocator<nnvm::TShape> >*, std::vector<nnvm::TShape, std::allocator<nnvm::TShape> >*)+0x3fa) [0x7f8ddd82afda]
[bt] (3) /tmp/mxnet3894755353450822729/mxnet-scala(mxnet::imperative::SetShapeType(mxnet::Context const&, nnvm::NodeAttrs const&, std::vector<mxnet::NDArray*, std::allocator<mxnet::NDArray*> > const&, std::vector<mxnet::NDArray*, std::allocator<mxnet::NDArray*> > const&, mxnet::DispatchMode*)+0xe11) [0x7f8ddf260671]
[bt] (4) /tmp/mxnet3894755353450822729/mxnet-scala(mxnet::Imperative::Invoke(mxnet::Context const&, nnvm::NodeAttrs const&, std::vector<mxnet::NDArray*, std::allocator<mxnet::NDArray*> > const&, std::vector<mxnet::NDArray*, std::allocator<mxnet::NDArray*> > const&)+0x35f) [0x7f8ddf2483ef]
[bt] (5) /tmp/mxnet3894755353450822729/mxnet-scala(MXImperativeInvokeImpl(void*, int, void**, int*, void***, int, char const**, char const**)+0xe70) [0x7f8ddf6e1350]
[bt] (6) /tmp/mxnet3894755353450822729/mxnet-scala(MXImperativeInvoke+0x4c) [0x7f8ddf6e27cc]
[bt] (7) /tmp/mxnet3894755353450822729/mxnet-scala(Java_org_apache_mxnet_LibInfo_mxImperativeInvoke+0x40f) [0x7f8ddcc72e1f]
[bt] (8) [0x7f8df05b1061]

 at org.apache.mxnet.Base$.checkCall (Base.scala:131)
    org.apache.mxnet.NDArray$.genericNDArrayFunctionInvoke (NDArray.scala:97)
    org.apache.mxnet.NDArray$.arange (NDArray.scala:408)
    org.apache.mxnet.NDArray.arange (NDArray.scala:-1)
    org.apache.clojure_mxnet.ndarray$arange.invokeStatic (ndarray.clj:91)
    org.apache.clojure_mxnet.ndarray$arange.invoke (ndarray.clj:84)
    org.apache.clojure_mxnet.ndarray_test$fn__4968.invokeStatic (ndarray_test.clj:131)
    org.apache.clojure_mxnet.ndarray_test/fn (ndarray_test.clj:125)
    clojure.test$test_var$fn__9209.invoke (test.clj:716)
    clojure.test$test_var.invokeStatic (test.clj:716)
    clojure.test$test_var.invoke (test.clj:707)
    clojure.test$test_vars$fn__9235$fn__9240.invoke (test.clj:734)
    clojure.test$default_fixture.invokeStatic (test.clj:686)
    clojure.test$default_fixture.invoke (test.clj:682)
    clojure.test$test_vars$fn__9235.invoke (test.clj:734)
    clojure.test$default_fixture.invokeStatic (test.clj:686)
    clojure.test$default_fixture.invoke (test.clj:682)
    clojure.test$test_vars.invokeStatic (test.clj:730)
    clojure.test$test_all_vars.invokeStatic (test.clj:736)
    clojure.test$test_ns.invokeStatic (test.clj:757)
    clojure.test$test_ns.invoke (test.clj:742)
    user$eval233$fn__294.invoke (form-init8353745245477399979.clj:1)
    clojure.lang.AFn.applyToHelper (AFn.java:156)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invokeStatic (core.clj:659)
    clojure.core$apply.invoke (core.clj:652)
    leiningen.core.injected$compose_hooks$fn__163.doInvoke (form-init8353745245477399979.clj:1)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invokeStatic (core.clj:657)
    clojure.core$apply.invoke (core.clj:652)
    leiningen.core.injected$run_hooks.invokeStatic (form-init8353745245477399979.clj:1)
    leiningen.core.injected$run_hooks.invoke (form-init8353745245477399979.clj:1)
    leiningen.core.injected$prepare_for_hooks$fn__168$fn__169.doInvoke (form-init8353745245477399979.clj:1)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.lang.AFunction$1.doInvoke (AFunction.java:29)
    clojure.lang.RestFn.invoke (RestFn.java:408)
    clojure.core$map$fn__5587.invoke (core.clj:2747)
    clojure.lang.LazySeq.sval (LazySeq.java:40)
    clojure.lang.LazySeq.seq (LazySeq.java:49)
    clojure.lang.Cons.next (Cons.java:39)
    clojure.lang.RT.next (RT.java:706)
    clojure.core$next__5108.invokeStatic (core.clj:64)
    clojure.core$reduce1.invokeStatic (core.clj:936)
    clojure.core$reduce1.invokeStatic (core.clj:926)
    clojure.core$merge_with.invokeStatic (core.clj:3051)
    clojure.core$merge_with.doInvoke (core.clj:3043)
    clojure.lang.RestFn.applyTo (RestFn.java:139)
    clojure.core$apply.invokeStatic (core.clj:659)
    clojure.test$run_tests.invokeStatic (test.clj:767)
    clojure.test$run_tests.doInvoke (test.clj:767)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invokeStatic (core.clj:657)
    clojure.core$apply.invoke (core.clj:652)
    user$eval233$fn__306$fn__339.invoke (form-init8353745245477399979.clj:1)
    user$eval233$fn__306$fn__307.invoke (form-init8353745245477399979.clj:1)
    user$eval233$fn__306.invoke (form-init8353745245477399979.clj:1)
    user$eval233.invokeStatic (form-init8353745245477399979.clj:1)
    user$eval233.invoke (form-init8353745245477399979.clj:1)
    clojure.lang.Compiler.eval (Compiler.java:7062)
    clojure.lang.Compiler.eval (Compiler.java:7052)
    clojure.lang.Compiler.load (Compiler.java:7514)
    clojure.lang.Compiler.loadFile (Compiler.java:7452)
    clojure.main$load_script.invokeStatic (main.clj:278)
    clojure.main$init_opt.invokeStatic (main.clj:280)
    clojure.main$init_opt.invoke (main.clj:280)
    clojure.main$initialize.invokeStatic (main.clj:311)
    clojure.main$null_opt.invokeStatic (main.clj:345)
    clojure.main$null_opt.invoke (main.clj:342)
    clojure.main$main.invokeStatic (main.clj:424)
    clojure.main$main.doInvoke (main.clj:387)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.lang.Var.applyTo (Var.java:702)
    clojure.main.main (main.java:37)

lein test org.apache.clojure-mxnet.operator-test
WARN  org.apache.mxnet.WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.KVStore was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing
WARN  org.apache.mxnet.WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.io.MXDataIter was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing

lein test :only org.apache.clojure-mxnet.operator-test/test-symbol-pow

FAIL in (test-symbol-pow) (operator_test.clj:162)
expected: (every? true? (map (fn [x y] (approx= tolerance x y)) expected grad-arrays))
  actual: (not (every? #object[clojure.core$true_QMARK_ 0x7d4135c9 "clojure.core$true_QMARK_@7d4135c9"] (true false)))
Embedded symbol: {
  "nodes": [
    {
      "op": "null", 
      "name": "data", 
      "inputs": []
    }, 
    {
      "op": "null", 
      "name": "embed_weight", 
      "attrs": {
        "input_dim": "10", 
        "output_dim": "4"
      }, 
      "inputs": []
    }, 
    {
      "op": "Embedding", 
      "name": "embed", 
      "attrs": {
        "input_dim": "10", 
        "output_dim": "4"
      }, 
      "inputs": [[0, 0, 0], [1, 0, 0]]
    }
  ], 
  "arg_nodes": [0, 1], 
  "node_row_ptr": [0, 1, 2, 3], 
  "heads": [[2, 0, 0]], 
  "attrs": {"mxnet_version": ["int", 10200]}
}

lein test org.apache.clojure-mxnet.optimizer-test
Testing optimizer -  sgd
Testing optimizer -  dcasgd
Testing optimizer -  nag
Testing optimizer -  ada-delta
Testing optimizer -  rms-prop
Testing optimizer -  ada-grad
Testing optimizer -  adam
Testing optimizer -  sgld

lein test org.apache.clojure-mxnet.random-test

lein test :only org.apache.clojure-mxnet.random-test/test-normal-on-cpu

ERROR in (test-normal-on-cpu) (Base.scala:131)
expected: (= ret1 ret2)
  actual: org.apache.mxnet.MXNetError: [00:39:07] src/operator/random/./sample_op.h:297: Check failed: param.scale > 0 (0 vs. 0) scale parameter in gaussian has to be positive

Stack trace returned 10 entries:
[bt] (0) /tmp/mxnet3894755353450822729/mxnet-scala(dmlc::StackTrace[abi:cxx11]()+0x1bc) [0x7f8ddcc8299c]
[bt] (1) /tmp/mxnet3894755353450822729/mxnet-scala(dmlc::LogMessageFatal::~LogMessageFatal()+0x28) [0x7f8ddcc83bb8]
[bt] (2) /tmp/mxnet3894755353450822729/mxnet-scala(mxnet::op::SampleMaster<mshadow::cpu, mxnet::op::NormalSampler<mshadow::cpu> >::op(nnvm::NodeAttrs const&, mxnet::OpContext const&, mxnet::OpReqType const&, mxnet::TBlob*)+0xe8) [0x7f8ddd1cc2e8]
[bt] (3) /tmp/mxnet3894755353450822729/mxnet-scala(void mxnet::op::Sample_<mshadow::cpu, mxnet::op::NormalSampler<mshadow::cpu> >(nnvm::NodeAttrs const&, mxnet::OpContext const&, std::vector<mxnet::TBlob, std::allocator<mxnet::TBlob> > const&, std::vector<mxnet::OpReqType, std::allocator<mxnet::OpReqType> > const&, std::vector<mxnet::TBlob, std::allocator<mxnet::TBlob> > const&)+0x8b) [0x7f8ddd1cd0fb]
[bt] (4) /tmp/mxnet3894755353450822729/mxnet-scala(mxnet::imperative::PushFCompute(std::function<void (nnvm::NodeAttrs const&, mxnet::OpContext const&, std::vector<mxnet::TBlob, std::allocator<mxnet::TBlob> > const&, std::vector<mxnet::OpReqType, std::allocator<mxnet::OpReqType> > const&, std::vector<mxnet::TBlob, std::allocator<mxnet::TBlob> > const&)> const&, nnvm::Op const*, nnvm::NodeAttrs const&, mxnet::Context const&, std::vector<mxnet::engine::Var*, std::allocator<mxnet::engine::Var*> > const&, std::vector<mxnet::engine::Var*, std::allocator<mxnet::engine::Var*> > const&, std::vector<mxnet::Resource, std::allocator<mxnet::Resource> > const&, std::vector<mxnet::NDArray*, std::allocator<mxnet::NDArray*> > const&, std::vector<mxnet::NDArray*, std::allocator<mxnet::NDArray*> > const&, std::vector<unsigned int, std::allocator<unsigned int> > const&, std::vector<mxnet::OpReqType, std::allocator<mxnet::OpReqType> > const&)::{lambda(mxnet::RunContext)#1}::operator()(mxnet::RunContext) const+0x273) [0x7f8ddf265043]
[bt] (5) /tmp/mxnet3894755353450822729/mxnet-scala(+0x2ecbbcb) [0x7f8ddf68ebcb]
[bt] (6) /tmp/mxnet3894755353450822729/mxnet-scala(mxnet::engine::ThreadedEngine::ExecuteOprBlock(mxnet::RunContext, mxnet::engine::OprBlock*)+0xcb5) [0x7f8ddf688aa5]
[bt] (7) /tmp/mxnet3894755353450822729/mxnet-scala(std::_Function_handler<void (std::shared_ptr<dmlc::ManualEvent>), mxnet::engine::ThreadedEnginePerDevice::PushToExecute(mxnet::engine::OprBlock*, bool)::{lambda()#1}::operator()() const::{lambda(std::shared_ptr<dmlc::ManualEvent>)#1}>::_M_invoke(std::_Any_data const&, std::shared_ptr<dmlc::ManualEvent>&&)+0xd9) [0x7f8ddf69b769]
[bt] (8) /tmp/mxnet3894755353450822729/mxnet-scala(std::thread::_Impl<std::_Bind_simple<std::function<void (std::shared_ptr<dmlc::ManualEvent>)> (std::shared_ptr<dmlc::ManualEvent>)> >::_M_run()+0x4a) [0x7f8ddf69a7ca]
[bt] (9) /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xbeae4) [0x7f8e046e5ae4]

 at org.apache.mxnet.Base$.checkCall (Base.scala:131)
    org.apache.mxnet.NDArray.internal (NDArray.scala:935)
    org.apache.mxnet.NDArray.toArray (NDArray.scala:928)
    org.apache.mxnet.NDArray.equals (NDArray.scala:1009)
    clojure.lang.Util.equiv (Util.java:33)
    clojure.core$_EQ_.invokeStatic (core.clj:785)
    clojure.core$_EQ_.invoke (core.clj:775)
    clojure.lang.AFn.applyToHelper (AFn.java:156)
    clojure.lang.RestFn.applyTo (RestFn.java:132)
    clojure.core$apply.invokeStatic (core.clj:657)
    clojure.core$apply.invoke (core.clj:652)
    org.apache.clojure_mxnet.random_test$fn__5947$fn__5951.invoke (random_test.clj:29)
    org.apache.clojure_mxnet.random_test$fn__5947.invokeStatic (random_test.clj:29)
    org.apache.clojure_mxnet.random_test/fn (random_test.clj:22)
    clojure.test$test_var$fn__9209.invoke (test.clj:716)
    clojure.test$test_var.invokeStatic (test.clj:716)
    clojure.test$test_var.invoke (test.clj:707)
    clojure.test$test_vars$fn__9235$fn__9240.invoke (test.clj:734)
    clojure.test$default_fixture.invokeStatic (test.clj:686)
    clojure.test$default_fixture.invoke (test.clj:682)
    clojure.test$test_vars$fn__9235.invoke (test.clj:734)
    clojure.test$default_fixture.invokeStatic (test.clj:686)
    clojure.test$default_fixture.invoke (test.clj:682)
    clojure.test$test_vars.invokeStatic (test.clj:730)
    clojure.test$test_all_vars.invokeStatic (test.clj:736)
    clojure.test$test_ns.invokeStatic (test.clj:757)
    clojure.test$test_ns.invoke (test.clj:742)
    user$eval233$fn__294.invoke (form-init8353745245477399979.clj:1)
    clojure.lang.AFn.applyToHelper (AFn.java:156)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invokeStatic (core.clj:659)
    clojure.core$apply.invoke (core.clj:652)
    leiningen.core.injected$compose_hooks$fn__163.doInvoke (form-init8353745245477399979.clj:1)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invokeStatic (core.clj:657)
    clojure.core$apply.invoke (core.clj:652)
    leiningen.core.injected$run_hooks.invokeStatic (form-init8353745245477399979.clj:1)
    leiningen.core.injected$run_hooks.invoke (form-init8353745245477399979.clj:1)
    leiningen.core.injected$prepare_for_hooks$fn__168$fn__169.doInvoke (form-init8353745245477399979.clj:1)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.lang.AFunction$1.doInvoke (AFunction.java:29)
    clojure.lang.RestFn.invoke (RestFn.java:408)
    clojure.core$map$fn__5587.invoke (core.clj:2747)
    clojure.lang.LazySeq.sval (LazySeq.java:40)
    clojure.lang.LazySeq.seq (LazySeq.java:49)
    clojure.lang.Cons.next (Cons.java:39)
    clojure.lang.RT.next (RT.java:706)
    clojure.core$next__5108.invokeStatic (core.clj:64)
    clojure.core$reduce1.invokeStatic (core.clj:936)
    clojure.core$reduce1.invokeStatic (core.clj:926)
    clojure.core$merge_with.invokeStatic (core.clj:3051)
    clojure.core$merge_with.doInvoke (core.clj:3043)
    clojure.lang.RestFn.applyTo (RestFn.java:139)
    clojure.core$apply.invokeStatic (core.clj:659)
    clojure.test$run_tests.invokeStatic (test.clj:767)
    clojure.test$run_tests.doInvoke (test.clj:767)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invokeStatic (core.clj:657)
    clojure.core$apply.invoke (core.clj:652)
    user$eval233$fn__306$fn__339.invoke (form-init8353745245477399979.clj:1)
    user$eval233$fn__306$fn__307.invoke (form-init8353745245477399979.clj:1)
    user$eval233$fn__306.invoke (form-init8353745245477399979.clj:1)
    user$eval233.invokeStatic (form-init8353745245477399979.clj:1)
    user$eval233.invoke (form-init8353745245477399979.clj:1)
    clojure.lang.Compiler.eval (Compiler.java:7062)
    clojure.lang.Compiler.eval (Compiler.java:7052)
    clojure.lang.Compiler.load (Compiler.java:7514)
    clojure.lang.Compiler.loadFile (Compiler.java:7452)
    clojure.main$load_script.invokeStatic (main.clj:278)
    clojure.main$init_opt.invokeStatic (main.clj:280)
    clojure.main$init_opt.invoke (main.clj:280)
    clojure.main$initialize.invokeStatic (main.clj:311)
    clojure.main$null_opt.invokeStatic (main.clj:345)
    clojure.main$null_opt.invoke (main.clj:342)
    clojure.main$main.invokeStatic (main.clj:424)
    clojure.main$main.doInvoke (main.clj:387)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.lang.Var.applyTo (Var.java:702)
    clojure.main.main (main.java:37)

lein test :only org.apache.clojure-mxnet.random-test/test-normal-on-cpu

FAIL in (test-normal-on-cpu) (random_test.clj:35)
expected: (< (Math/abs (- mean mu)) 0.1)
  actual: (not (< 9.878657910405863E31 0.1))

lein test :only org.apache.clojure-mxnet.random-test/test-normal-on-cpu

FAIL in (test-normal-on-cpu) (random_test.clj:36)
expected: (< (Math/abs (- stddev sigma)) 0.1)
  actual: (not (< 9.670942343425893E33 0.1))

lein test org.apache.clojure-mxnet.shape-test

lein test org.apache.clojure-mxnet.symbol-test
Symbol Outputs:
    output[0]=composed(0)
Variable:data
Variable:fc1_weight
Variable:fc1_bias
--------------------
Op:FullyConnected, Name=fc1
Inputs:
    arg[0]=data(0) version=0
    arg[1]=fc1_weight(0) version=0
    arg[2]=fc1_bias(0) version=0
Attrs:
    num_hidden=10
Variable:fc2_weight
Variable:fc2_bias
--------------------
Op:FullyConnected, Name=fc2
Inputs:
    arg[0]=fc1(0)
    arg[1]=fc2_weight(0) version=0
    arg[2]=fc2_bias(0) version=0
Attrs:
    num_hidden=100
Variable:fc3_weight
Variable:fc3_bias
--------------------
Op:FullyConnected, Name=fc3
Inputs:
    arg[0]=fc2(0)
    arg[1]=fc3_weight(0) version=0
    arg[2]=fc3_bias(0) version=0
Attrs:
    num_hidden=10
--------------------
Op:Activation, Name=activation0
Inputs:
    arg[0]=fc3(0)
Attrs:
    act_type=relu
Variable:fc4_weight
Variable:fc4_bias
--------------------
Op:FullyConnected, Name=composed
Inputs:
    arg[0]=activation0(0)
    arg[1]=fc4_weight(0) version=0
    arg[2]=fc4_bias(0) version=0
Attrs:
    num_hidden=20

lein test org.apache.clojure-mxnet.test-util

Ran 131 tests containing 479 assertions.
5 failures, 2 errors.
INFO  org.apache.mxnet.util.NativeLibraryLoader: Deleting /tmp/mxnet3894755353450822729/mxnet-scala
INFO  org.apache.mxnet.util.NativeLibraryLoader: Deleting /tmp/mxnet3894755353450822729
Tests failed.

Again the fitting test for the convolutions does not work, besides the other errors. I have hand-compiled mxnet before, but have reverted to just load the library from the classpath, which works at least when constructing ndarrays and reading them. Any ideas what I am doing wrong?

gigasquid commented 6 years ago

Thanks for reporting the issue. Can you provide the details on the system that you are running? Maybe that will help diagnose the cause.

gigasquid commented 6 years ago

I see that it is ubuntu - but what jvm version? Also are you using the linux-cpu or gpu version?

I did find an issue that might be related with gcc version https://github.com/apache/incubator-mxnet/issues/9765

gigasquid commented 6 years ago

I also found this that seems to point to an Ubuntu/ nvidia/ gcc version compatibility for 18.04 https://medium.com/@at15/install-mxnet-on-ubuntu-18-04-8039f63b32e1

whilo commented 6 years ago

Hmm, I have opted for the linux-cpu version, using the dependency for [org.apache.mxnet/mxnet-full_2.11-linux-x86_64-cpu "1.2.0"] for minimal friction (I also do not have a GPU on my dev laptop). This at least excludes cuda as a source of error, I think. It could be that the linked system libraries are somehow problematic. I will try to use my own compiled version next.

Btw. I can do arithmetic on ndarray objects just fine.

whilo commented 6 years ago

Ok, building locally and loading [org.apache.mxnet/mxnet-full_2.11-linux-x86_64-cpu "1.3.0-SNAPSHOT"] causes the same problem.

whilo commented 6 years ago

Only linking to system libs. 

christian@gramsci /t/mxnet7434723396455381190> ldd mxnet-scala 
    linux-vdso.so.1 (0x00007ffc9dfed000)
    libcblas.so.3 => /usr/lib/x86_64-linux-gnu/libcblas.so.3 (0x00007f75c40dd000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f75c3ed5000)
    libopencv_imgcodecs.so.3.2 => /usr/lib/x86_64-linux-gnu/libopencv_imgcodecs.so.3.2 (0x00007f75c3ca0000)
    libopencv_imgproc.so.3.2 => /usr/lib/x86_64-linux-gnu/libopencv_imgproc.so.3.2 (0x00007f75c374f000)
    libopencv_core.so.3.2 => /usr/lib/x86_64-linux-gnu/libopencv_core.so.3.2 (0x00007f75c3314000)
    libjemalloc.so.1 => /usr/lib/x86_64-linux-gnu/libjemalloc.so.1 (0x00007f75c30de000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f75c2d50000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f75c29b2000)
    libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f75c2782000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f75c256a000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f75c234b000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f75c1f5a000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f75c79c8000)
    libatlas.so.3 => /usr/lib/x86_64-linux-gnu/libatlas.so.3 (0x00007f75c19d1000)
    libjpeg.so.8 => /usr/lib/x86_64-linux-gnu/libjpeg.so.8 (0x00007f75c1769000)
    libwebp.so.6 => /usr/lib/x86_64-linux-gnu/libwebp.so.6 (0x00007f75c1500000)
    libpng16.so.16 => /usr/lib/x86_64-linux-gnu/libpng16.so.16 (0x00007f75c12ce000)
    libgdcmMSFF.so.2.8 => /usr/lib/x86_64-linux-gnu/libgdcmMSFF.so.2.8 (0x00007f75c0e4b000)
    libtiff.so.5 => /usr/lib/x86_64-linux-gnu/libtiff.so.5 (0x00007f75c0bd4000)
    libIlmImf-2_2.so.22 => /usr/lib/x86_64-linux-gnu/libIlmImf-2_2.so.22 (0x00007f75c0711000)
    libHalf.so.12 => /usr/lib/x86_64-linux-gnu/libHalf.so.12 (0x00007f75c04ce000)
    libgdal.so.20 => /usr/lib/libgdal.so.20 (0x00007f75bf2c0000)
    libgdcmDSED.so.2.8 => /usr/lib/x86_64-linux-gnu/libgdcmDSED.so.2.8 (0x00007f75befad000)
    libtbb.so.2 => /usr/lib/x86_64-linux-gnu/libtbb.so.2 (0x00007f75bed71000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f75beb54000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f75be950000)
    libgdcmIOD.so.2.8 => /usr/lib/x86_64-linux-gnu/libgdcmIOD.so.2.8 (0x00007f75be73a000)
    libgdcmDICT.so.2.8 => /usr/lib/x86_64-linux-gnu/libgdcmDICT.so.2.8 (0x00007f75be2da000)
    libgdcmjpeg8.so.2.8 => /usr/lib/x86_64-linux-gnu/libgdcmjpeg8.so.2.8 (0x00007f75be0b2000)
    libgdcmjpeg12.so.2.8 => /usr/lib/x86_64-linux-gnu/libgdcmjpeg12.so.2.8 (0x00007f75bde8a000)
    libgdcmjpeg16.so.2.8 => /usr/lib/x86_64-linux-gnu/libgdcmjpeg16.so.2.8 (0x00007f75bdc62000)
    libopenjp2.so.7 => /usr/lib/x86_64-linux-gnu/libopenjp2.so.7 (0x00007f75bda0c000)
    libCharLS.so.1 => /usr/lib/x86_64-linux-gnu/libCharLS.so.1 (0x00007f75bd7c5000)
    libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f75bd5be000)
    libjson-c.so.3 => /lib/x86_64-linux-gnu/libjson-c.so.3 (0x00007f75bd3b3000)
    libgdcmCommon.so.2.8 => /usr/lib/x86_64-linux-gnu/libgdcmCommon.so.2.8 (0x00007f75bd18a000)
    liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f75bcf64000)
    libjbig.so.0 => /usr/lib/x86_64-linux-gnu/libjbig.so.0 (0x00007f75bcd56000)
    libIex-2_2.so.12 => /usr/lib/x86_64-linux-gnu/libIex-2_2.so.12 (0x00007f75bcb38000)
    libIlmThread-2_2.so.12 => /usr/lib/x86_64-linux-gnu/libIlmThread-2_2.so.12 (0x00007f75bc931000)
    libarmadillo.so.8 => /usr/lib/libarmadillo.so.8 (0x00007f75bc728000)
    libproj.so.12 => /usr/lib/x86_64-linux-gnu/libproj.so.12 (0x00007f75bc4bf000)
    libpoppler.so.73 => /usr/lib/x86_64-linux-gnu/libpoppler.so.73 (0x00007f75bc02b000)
    libfreexl.so.1 => /usr/lib/x86_64-linux-gnu/libfreexl.so.1 (0x00007f75bbe22000)
    libqhull.so.7 => /usr/lib/x86_64-linux-gnu/libqhull.so.7 (0x00007f75bbbc8000)
    libgeos_c.so.1 => /usr/lib/x86_64-linux-gnu/libgeos_c.so.1 (0x00007f75bb999000)
    libepsilon.so.1 => /usr/lib/x86_64-linux-gnu/libepsilon.so.1 (0x00007f75bb781000)
    libodbc.so.2 => /usr/lib/x86_64-linux-gnu/libodbc.so.2 (0x00007f75bb514000)
    libodbcinst.so.2 => /usr/lib/x86_64-linux-gnu/libodbcinst.so.2 (0x00007f75bb2ff000)
    libkmlbase.so.1 => /usr/lib/x86_64-linux-gnu/libkmlbase.so.1 (0x00007f75bb0e4000)
    libkmldom.so.1 => /usr/lib/x86_64-linux-gnu/libkmldom.so.1 (0x00007f75bae2d000)
    libkmlengine.so.1 => /usr/lib/x86_64-linux-gnu/libkmlengine.so.1 (0x00007f75babf5000)
    libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f75ba9c3000)
    libxerces-c-3.2.so => /usr/lib/x86_64-linux-gnu/libxerces-c-3.2.so (0x00007f75ba44f000)
    libnetcdf.so.13 => /usr/lib/x86_64-linux-gnu/libnetcdf.so.13 (0x00007f75ba12d000)
    libhdf5_serial.so.100 => /usr/lib/x86_64-linux-gnu/libhdf5_serial.so.100 (0x00007f75b9bd5000)
    libmfhdfalt.so.0 => /usr/lib/libmfhdfalt.so.0 (0x00007f75b99af000)
    libdfalt.so.0 => /usr/lib/libdfalt.so.0 (0x00007f75b970c000)
    libogdi.so.3.2 => /usr/lib/libogdi.so.3.2 (0x00007f75b94ec000)
    libgif.so.7 => /usr/lib/x86_64-linux-gnu/libgif.so.7 (0x00007f75b92e3000)
    libgeotiff.so.2 => /usr/lib/x86_64-linux-gnu/libgeotiff.so.2 (0x00007f75b90ad000)
    libpq.so.5 => /usr/lib/x86_64-linux-gnu/libpq.so.5 (0x00007f75b8e65000)
    libdapclient.so.6 => /usr/lib/x86_64-linux-gnu/libdapclient.so.6 (0x00007f75b8c24000)
    libdap.so.25 => /usr/lib/x86_64-linux-gnu/libdap.so.25 (0x00007f75b8894000)
    libspatialite.so.7 => /usr/lib/x86_64-linux-gnu/libspatialite.so.7 (0x00007f75b8112000)
    libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f75b7ea0000)
    libcurl-gnutls.so.4 => /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4 (0x00007f75b7c23000)
    libfyba.so.0 => /usr/lib/x86_64-linux-gnu/libfyba.so.0 (0x00007f75b79ce000)
    libxml2.so.2 => /usr/lib/x86_64-linux-gnu/libxml2.so.2 (0x00007f75b760d000)
    libmysqlclient.so.20 => /usr/lib/x86_64-linux-gnu/libmysqlclient.so.20 (0x00007f75b6ffd000)
    libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f75b6b85000)
    libblas.so.3 => /usr/lib/x86_64-linux-gnu/libblas.so.3 (0x00007f75b692a000)
    liblapack.so.3 => /usr/lib/x86_64-linux-gnu/liblapack.so.3 (0x00007f75b60a4000)
    libarpack.so.2 => /usr/lib/x86_64-linux-gnu/libarpack.so.2 (0x00007f75b5e5a000)
    libsuperlu.so.5 => /usr/lib/x86_64-linux-gnu/libsuperlu.so.5 (0x00007f75b5bea000)
    libfreetype.so.6 => /usr/lib/x86_64-linux-gnu/libfreetype.so.6 (0x00007f75b5936000)
    libfontconfig.so.1 => /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 (0x00007f75b56f1000)
    libnss3.so => /usr/lib/x86_64-linux-gnu/libnss3.so (0x00007f75b53ad000)
    libsmime3.so => /usr/lib/x86_64-linux-gnu/libsmime3.so (0x00007f75b5181000)
    libnspr4.so => /usr/lib/x86_64-linux-gnu/libnspr4.so (0x00007f75b4f44000)
    liblcms2.so.2 => /usr/lib/x86_64-linux-gnu/liblcms2.so.2 (0x00007f75b4cec000)
    libgeos-3.6.2.so => /usr/lib/x86_64-linux-gnu/libgeos-3.6.2.so (0x00007f75b4953000)
    libltdl.so.7 => /usr/lib/x86_64-linux-gnu/libltdl.so.7 (0x00007f75b4749000)
    libminizip.so.1 => /usr/lib/x86_64-linux-gnu/libminizip.so.1 (0x00007f75b453e000)
    liburiparser.so.1 => /usr/lib/x86_64-linux-gnu/liburiparser.so.1 (0x00007f75b4323000)
    libicuuc.so.60 => /usr/lib/x86_64-linux-gnu/libicuuc.so.60 (0x00007f75b3f6c000)
    libhdf5_serial_hl.so.100 => /usr/lib/x86_64-linux-gnu/libhdf5_serial_hl.so.100 (0x00007f75b3d49000)
    libsz.so.2 => /usr/lib/x86_64-linux-gnu/libsz.so.2 (0x00007f75b3b46000)
    libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f75b38dc000)
    libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f75b3691000)
    libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007f75b343f000)
    libsqlite3.so.0 => /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007f75b3136000)
    libnghttp2.so.14 => /usr/lib/x86_64-linux-gnu/libnghttp2.so.14 (0x00007f75b2f11000)
    libidn2.so.0 => /usr/lib/x86_64-linux-gnu/libidn2.so.0 (0x00007f75b2cf4000)
    librtmp.so.1 => /usr/lib/x86_64-linux-gnu/librtmp.so.1 (0x00007f75b2ad8000)
    libpsl.so.5 => /usr/lib/x86_64-linux-gnu/libpsl.so.5 (0x00007f75b28ca000)
    libnettle.so.6 => /usr/lib/x86_64-linux-gnu/libnettle.so.6 (0x00007f75b2694000)
    libgnutls.so.30 => /usr/lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007f75b232f000)
    liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007f75b2121000)
    libfyut.so.0 => /usr/lib/x86_64-linux-gnu/libfyut.so.0 (0x00007f75b1f18000)
    libfygm.so.0 => /usr/lib/x86_64-linux-gnu/libfygm.so.0 (0x00007f75b1d11000)
    libopenblas.so.0 => /usr/lib/x86_64-linux-gnu/libopenblas.so.0 (0x00007f75afa6b000)
    libgfortran.so.4 => /usr/lib/x86_64-linux-gnu/libgfortran.so.4 (0x00007f75af68c000)
    libnssutil3.so => /usr/lib/x86_64-linux-gnu/libnssutil3.so (0x00007f75af45d000)
    libplc4.so => /usr/lib/x86_64-linux-gnu/libplc4.so (0x00007f75af258000)
    libplds4.so => /usr/lib/x86_64-linux-gnu/libplds4.so (0x00007f75af054000)
    libicudata.so.60 => /usr/lib/x86_64-linux-gnu/libicudata.so.60 (0x00007f75ad4ab000)
    libaec.so.0 => /usr/lib/x86_64-linux-gnu/libaec.so.0 (0x00007f75ad2a3000)
    libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f75acfcd000)
    libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f75acd9b000)
    libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f75acb97000)
    libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f75ac98c000)
    libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f75ac771000)
    libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007f75ac556000)
    libgssapi.so.3 => /usr/lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007f75ac315000)
    libunistring.so.2 => /usr/lib/x86_64-linux-gnu/libunistring.so.2 (0x00007f75abf97000)
    libhogweed.so.4 => /usr/lib/x86_64-linux-gnu/libhogweed.so.4 (0x00007f75abd63000)
    libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f75abae2000)
    libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f75ab7b3000)
    libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007f75ab5a0000)
    libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f75ab360000)
    libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f75ab15c000)
    libheimntlm.so.0 => /usr/lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007f75aaf53000)
    libkrb5.so.26 => /usr/lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007f75aacc6000)
    libasn1.so.8 => /usr/lib/x86_64-linux-gnu/libasn1.so.8 (0x00007f75aaa24000)
    libhcrypto.so.4 => /usr/lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007f75aa7ee000)
    libroken.so.18 => /usr/lib/x86_64-linux-gnu/libroken.so.18 (0x00007f75aa5d8000)
    libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007f75aa3d0000)
    libwind.so.0 => /usr/lib/x86_64-linux-gnu/libwind.so.0 (0x00007f75aa1a7000)
    libheimbase.so.1 => /usr/lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007f75a9f98000)
    libhx509.so.5 => /usr/lib/x86_64-linux-gnu/libhx509.so.5 (0x00007f75a9d4e000)
    libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f75a9b16000)

Trying to shrink the error case to a simple forward case and see whether this works.

whilo commented 6 years ago

My JVM config is: 

{"sun.os.patch.level" "unknown",
 "sun.boot.class.path"
 "/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/resources.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jsse.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jce.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/charsets.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jfr.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/classes",
 "clojure-mxnet.version" "0.1.1-SNAPSHOT",
 "sun.management.compiler" "HotSpot 64-Bit Tiered Compilers",
 "java.vm.info" "mixed mode",
 "clojure.debug" "false",
 "sun.boot.library.path"
 "/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64",
 "java.awt.graphicsenv" "sun.awt.X11GraphicsEnvironment",
 "path.separator" ":",
 "user.dir" "/home/christian/Development/clojure-mxnet",
 "java.vm.specification.vendor" "Oracle Corporation",
 "user.country" "US",
 "java.vendor.url" "http://java.oracle.com/",
 "java.specification.name" "Java Platform API Specif
ication",
 "sun.desktop" "gnome",
 "line.separator" "\n",
 "file.encoding.pkg" "sun.io",
 "sun.cpu.endian" "little",
 "java.vm.version" "25.171-b11",
 "sun.java.launcher" "SUN_STANDARD",
 "apple.awt.UIElement" "true",
 "java.vendor" "Oracle Corporation",
 "java.specification.vendor" "Oracle Corporation",
 "awt.toolkit" "sun.awt.X11.XToolkit",
 "clojure.compile.path"
 "/home/christian/Development/clojure-mxnet/target/classes",
 "java.vm.specification.version" "1.8",
 "java.io.tmpdir" "/tmp",
 "java.home" "/usr/lib/jvm/java-8-openjdk-amd64/jre",
 "java.class.path"
 "/home/christian/Development/clojure-mxnet/test:/home/christian/Development/clojure-mxnet/src:/home/christian/Development/clojure-mxnet/dev-resources:/home/christian/Development/clojure-mxnet/resources:/home/christian/Development/clojure-mxnet/target/classes:/home/christian/.m2/repository/org/clojure/tools.logging/0.4.0/tools.logging-0.4.0.jar:/home/christian/.m2/repository/org/apache/logging/log4j/log4j-api/2.8.1/log4j-api-2.8.1.jar:/home/christian/
.m2/repository/t6/from-scala/0.3.0/from-scala-0.3.0.jar:/home/christian/.m2/repository/cider/cider-nrepl/0.18.0-SNAPSHOT/cider-nrepl-0.18.0-SNAPSHOT.jar:/home/christian/.m2/repository/org/scala-lang/scala-compiler/2.11.8/scala-compiler-2.11.8.jar:/home/christian/.m2/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar:/home/christian/.m2/repository/org/slf4j/slf4j-api/1.7.5/slf4j-api-1.7.5.jar:/home/christian/.m2/repository/org/apache/mxnet/mxnet-core_2.11/1.3.0-SNAPSHOT/mxnet-core_2.11-1.3.0-SNAPSHOT.jar:/home/christian/.m2/repository/org/apache/mxnet/mxnet-full_2.11-linux-x86_64-cpu/1.3.0-SNAPSHOT/mxnet-full_2.11-linux-x86_64-cpu-1.3.0-SNAPSHOT.jar:/home/christian/.m2/repository/clojure-complete/clojure-complete/0.2.4/clojure-complete-0.2.4.jar:/home/christian/.m2/repository/funcool/cats/1.2.1/cats-1.2.1.jar:/home/christian/.m2/repository/refactor-nrepl/refactor-nrepl/2.4.0-SNAPSHOT/refactor-nrepl-2.4.0-SNAPSHOT.jar:/home/christian/.m2/repository/org/clojure/tools.nrepl/0.2.13/tools.nrepl-0.2.
13.jar:/home/christian/.m2/repository/org/apache/logging/log4j/log4j-core/2.8.1/log4j-core-2.8.1.jar:/home/christian/.m2/repository/org/scala-lang/scala-reflect/2.11.8/scala-reflect-2.11.8.jar:/home/christian/.m2/repository/args4j/args4j/2.0.29/args4j-2.0.29.jar:/home/christian/.m2/repository/potemkin/potemkin/0.4.3/potemkin-0.4.3.jar:/home/christian/.m2/repository/org/clojure/clojure/1.9.0/clojure-1.9.0.jar:/home/christian/.m2/repository/org/scala-lang/modules/scala-parser-combinators_2.11/1.0.4/scala-parser-combinators_2.11-1.0.4.jar:/home/christian/.m2/repository/clj-tuple/clj-tuple/0.2.2/clj-tuple-0.2.2.jar:/home/christian/.m2/repository/org/clojure/core.specs.alpha/0.1.24/core.specs.alpha-0.1.24.jar:/home/christian/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/home/christian/.m2/repository/riddley/riddley/0.1.12/riddley-0.1.12.jar:/home/christian/.m2/repository/org/scala-lang/scala-library/2.11.8/scala-library-2.11.8.jar:/home/christian/.m2/repository/org/scala-lang/modules/scala-xml_2.11/1.0.4/sca
la-xml_2.11-1.0.4.jar:/home/christian/.m2/repository/commons-codec/commons-codec/1.10/commons-codec-1.10.jar:/home/christian/.m2/repository/org/clojure/spec.alpha/0.1.143/spec.alpha-0.1.143.jar",
 "java.runtime.version" "1.8.0_171-8u171-b11-0ubuntu0.18.04.1-b11",
 "java.vm.vendor" "Oracle Corporation",
 "user.language" "en",
 "java.vm.name" "OpenJDK 64-Bit Server VM",
 "java.version" "1.8.0_171",
 "os.arch" "amd64",
 "user.home" "/home/christian",
 "java.specification.version" "1.8",
 "java.ext.dirs"
 "/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext:/usr/java/packages/lib/ext",
 "sun.jnu.encoding" "UTF-8",
 "java.runtime.name" "OpenJDK Runtime Environment",
 "sun.io.unicode.encoding" "UnicodeLittle",
 "sun.arch.data.model" "64",
 "java.class.version" "52.0",
 "sun.cpu.isalist" "",
 "file.encoding" "UTF-8",
 "user.timezone" "Europe/Berlin",
 "os.name" "Linux",
 "os.version" "4.15.0-23-generic",
 "file.separator" "/",
 "java.vm.specification.name" "Java Virtual Machine Specification",
 "java.awt.printerjob" "sun.
print.PSPrinterJob",
 "java.endorsed.dirs"
 "/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/endorsed",
 "java.library.path"
 "/home/christian/torch/install/lib:/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib",
 "java.vendor.url.bug" "http://bugreport.sun.com/bugreport/",
 "sun.java.command"
 "clojure.main -i /tmp/form-init1809604993349017573.clj",
 "user.name" "christian"}
whilo commented 6 years ago

I think this should not be zero, right?

(let [num-class 10
        data1 (sym/variable "data1")
        conv1 (sym/convolution {:data data1 :kernel [2 2] :num-filter 2 :stride [2 2]})
        pooling1 (sym/pooling {:data conv1 :kernel [2 2] :pool-type "avg" :stride [1 1]})
        flatten1 (sym/flatten {:data pooling1})
        sum (sym/sum {:data flatten1 :axis 1})
        fc (sym/fully-connected {:data sum :num-hidden num-class})
        my-sym (sym/softmax-output "softmax" {:data fc})

        d-shape1 [10 3 64 64]
        l-shape [10]
        mod (m/module my-sym {:data-names ["data1" "data2"]})
        data-batch {:data [(ndarray/random-uniform 0 9 (str (mx-shape/->shape d-shape1)))]
                    :label [(ndarray/ones l-shape)]
                    :index nil
                    :pad 0}]

   ;; train with the original shapes
    (-> mod
        (m/bind {:data-shapes [{:name "data1" :shape d-shape1}]
                 :label-shapes [{:name "softmax_label" :shape l-shape :layout "N"}]})
        (m/init-params)
        (m/init-optimizer {:optimizer (optimizer/sgd {:learning-rate 0.1})})
        (m/forward data-batch))
    (let [[k v] (first (first (m/params mod)))]
      [k (ndarray/->double-vec v)])) ;; => ["convolution97_weight" [0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0]]
whilo commented 6 years ago

make scalatests shows the same behaviour so this is unrelated to clojure-mxnet. The python2 mxnet library installation through pip works on my machine though. It links:

christian@gramsci /usr> ldd ./local/lib/python2.7/dist-packages/mxnet/libmxnet.so
    linux-vdso.so.1 (0x00007ffc28bf7000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f77dcc43000)
    libgfortran.so.3 => /usr/./local/lib/python2.7/dist-packages/mxnet/libgfortran.so.3 (0x00007f77dc91c000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f77dc718000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f77dc38a000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f77dbfec000)
    libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f77dbdbc000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f77dbba4000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f77db985000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f77db594000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f77e1ea7000)
    libquadmath.so.0 => /usr/./local/lib/python2.7/dist-packages/mxnet/libquadmath.so.0 (0x00007f77db354000)

The main difference being the local Python specific libgfortran.so.3 vs. libgfortran.so.4 for scala-mxnet.

whilo commented 6 years ago

On my 18.04 server all tests pass:

christian@roberts:~/clojure-mxnet$ lein test
INFO  MXNetJVM: Try loading mxnet-scala from native path.
INFO  MXNetJVM: Try loading mxnet-scala-linux-x86_64-gpu from native path.
INFO  MXNetJVM: Try loading mxnet-scala-linux-x86_64-cpu from native path.
WARN  MXNetJVM: MXNet Scala native library not found in path. Copying native library from the archive. Consider installing the library somewhere in the path (for Windows: PATH, for Linux: LD_LIBRARY_PATH), or specifying by Java cmd option -Djava.library.path=[lib path].
INFO  org.apache.mxnet.util.NativeLibraryLoader: Loading libmxnet-scala.so from /lib/native/ copying to mxnet-scala
[20:12:09] src/io/iter_mnist.cc:113: MNISTIter: load 60000 images, shuffle=1, shape=[100,1,28,28]
[20:12:09] src/io/iter_mnist.cc:113: MNISTIter: load 10000 images, shuffle=1, shape=[100,1,28,28]
WARN  org.apache.mxnet.WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.NDArray was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing

lein test org.apache.clojure-mxnet.conv-test
WARN  org.apache.mxnet.WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.Symbol was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing
Epoch  0  Train- [accuracy 0.76921666]
Epoch  0  Time cost- 5467
Epoch  0  Validation-  [accuracy 0.9295]
Score [accuracy 0.9295]

lein test org.apache.clojure-mxnet.executor-test

lein test org.apache.clojure-mxnet.io-test
[20:12:18] src/io/iter_mnist.cc:110: MNISTIter: load 60000 images, shuffle=1, shape=(100,784)
[20:12:19] src/io/iter_mnist.cc:110: MNISTIter: load 60000 images, shuffle=1, shape=(100,784)
[20:12:34] src/io/iter_image_recordio_2.cc:170: ImageRecordIOParser2: data/cifar/train.rec, use 3 threads for decoding..
[20:12:36] src/io/iter_mnist.cc:110: MNISTIter: load 60000 images, shuffle=1, shape=(100,784)
[20:12:37] src/io/iter_mnist.cc:110: MNISTIter: load 60000 images, shuffle=1, shape=(100,784)
[20:12:38] src/io/iter_mnist.cc:110: MNISTIter: load 60000 images, shuffle=1, shape=(100,784)
WARN  org.apache.mxnet.WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.Executor was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing
WARN  org.apache.mxnet.WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.io.MXDataIter was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing

lein test org.apache.clojure-mxnet.kvstore-test

lein test org.apache.clojure-mxnet.module-test
INFO  org.apache.mxnet.module.Module: Saved checkpoint to test-0000.params
INFO  org.apache.mxnet.module.Module: Saved optimizer state to test-0000.states
INFO  org.apache.mxnet.Model: Auto - select kvstore type = local_update_cpu
INFO  org.apache.mxnet.Model: Auto - select kvstore type = local_update_cpu
INFO  org.apache.mxnet.module.Module: Saved checkpoint to test-0000.params
INFO  org.apache.mxnet.module.Module: Saved optimizer state to test-0000.states

lein test org.apache.clojure-mxnet.ndarray-test
[20:12:39] src/ndarray/./ndarray_function-inl.h:97: The operator onehot_encode is deprecated; use one_hot instead.

lein test org.apache.clojure-mxnet.operator-test
WARN  org.apache.mxnet.WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.KVStore was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing
Embedded symbol: {
  "nodes": [
    {
      "op": "null", 
      "name": "data", 
      "inputs": []
    }, 
    {
      "op": "null", 
      "name": "embed_weight", 
      "attrs": {
        "input_dim": "10", 
        "output_dim": "4"
      }, 
      "inputs": []
    }, 
    {
      "op": "Embedding", 
      "name": "embed", 
      "attrs": {
        "input_dim": "10", 
        "output_dim": "4"
      }, 
      "inputs": [[0, 0, 0], [1, 0, 0]]
    }
  ], 
  "arg_nodes": [0, 1], 
  "node_row_ptr": [0, 1, 2, 3], 
  "heads": [[2, 0, 0]], 
  "attrs": {"mxnet_version": ["int", 10200]}
}

lein test org.apache.clojure-mxnet.optimizer-test
Testing optimizer -  sgd
Testing optimizer -  dcasgd
Testing optimizer -  nag
Testing optimizer -  ada-delta
Testing optimizer -  rms-prop
Testing optimizer -  ada-grad
Testing optimizer -  adam
Testing optimizer -  sgld

lein test org.apache.clojure-mxnet.random-test

lein test org.apache.clojure-mxnet.shape-test

lein test org.apache.clojure-mxnet.symbol-test
Symbol Outputs:
    output[0]=composed(0)
Variable:data
Variable:fc1_weight
Variable:fc1_bias
--------------------
Op:FullyConnected, Name=fc1
Inputs:
    arg[0]=data(0) version=0
    arg[1]=fc1_weight(0) version=0
    arg[2]=fc1_bias(0) version=0
Attrs:
    num_hidden=10
Variable:fc2_weight
Variable:fc2_bias
--------------------
Op:FullyConnected, Name=fc2
Inputs:
    arg[0]=fc1(0)
    arg[1]=fc2_weight(0) version=0
    arg[2]=fc2_bias(0) version=0
Attrs:
    num_hidden=100
Variable:fc3_weight
Variable:fc3_bias
--------------------
Op:FullyConnected, Name=fc3
Inputs:
    arg[0]=fc2(0)
    arg[1]=fc3_weight(0) version=0
    arg[2]=fc3_bias(0) version=0
Attrs:
    num_hidden=10
--------------------
Op:Activation, Name=activation0
Inputs:
    arg[0]=fc3(0)
Attrs:
    act_type=relu
Variable:fc4_weight
Variable:fc4_bias
--------------------
Op:FullyConnected, Name=composed
Inputs:
    arg[0]=activation0(0)
    arg[1]=fc4_weight(0) version=0
    arg[2]=fc4_bias(0) version=0
Attrs:
    num_hidden=20

lein test org.apache.clojure-mxnet.test-util

Ran 131 tests containing 479 assertions.
0 failures, 0 errors.
INFO  org.apache.mxnet.util.NativeLibraryLoader: Deleting /tmp/mxnet3658704930836012199/mxnet-scala
INFO  org.apache.mxnet.util.NativeLibraryLoader: Deleting /tmp/mxnet3658704930836012199

So it is definitely something regarding the setup on my laptop.

gigasquid commented 6 years ago

Weird. Thanks so much for investigating and looking into it. Please keep posting what you find. I'm sure it will be helpful to others

aaelony commented 6 years ago

Hi, can you specify which CUDA and cuDNN versions you are running where your 18.04 server tests all pass?

Perhaps a data file or table can be added to the repo that chronicles by OS name & version, CUDA version, cuDNN version, python version, and any other relevant variables whether lein test passed and when it was tested?

whilo commented 6 years ago

This was without CUDA.

aaelony commented 6 years ago

I'm on an18.04 laptop as well. I'm open to trying to replicate your steps if you provide them.

aaelony commented 6 years ago

btw, thanks. I hadn't noticed you stated earlier you are testing the CPU build.

For me, the issue boils down to a missing libmxnet-scala.so, i.e. https://mxnet.incubator.apache.org/install/index.html?platform=Linux&language=Scala&processor=CPU punts saying Will be available soon for Scala. The R and Python versions are available with instructions describing how to build.

Starting from a clean git clone https://github.com/gigasquid/clojure-mxnet.git followed by cd clojure-mxnet then lein test on 18.04 I get:

$ lein test
...
INFO  MXNetJVM: Try loading mxnet-scala from native path.
INFO  MXNetJVM: Try loading mxnet-scala-linux-x86_64-gpu from native path.
INFO  MXNetJVM: Try loading mxnet-scala-linux-x86_64-cpu from native path.
WARN  MXNetJVM: MXNet Scala native library not found in path. Copying native library from the archive. Consider installing the library somewhere in the path (for Windows: PATH, for Linux: LD_LIBRARY_PATH), or specifying by Java cmd option -Djava.library.path=[lib path].
ERROR MXNetJVM: Couldn't find native library mxnet-scala
Exception in thread "main" java.lang.UnsatisfiedLinkError: Couldn't find the resource libmxnet-scala.so, compiling:(base.clj:21:19)
...
Caused by: java.lang.UnsatisfiedLinkError: Couldn't find the resource libmxnet-scala.so

I'll update if I can build either the cpu or gpu version.

gigasquid commented 6 years ago

@aaelony can you please post what your project.clj file looks like?

aaelony commented 6 years ago

Hi @gigasquid, I simply do

git clone https://github.com/gigasquid/clojure-mxnet.git
cd clojure-mxnet
lein test

I'm reading through https://mxnet.incubator.apache.org/install/build_from_source.html?highlight=source and seeing if I can build the missing shared lib.

update: I do see that the project.clj has a default dependency for osx and a linux dependency commented out. I reversed those comments now. Still errors but that is still due to the missing shared lib.

ERROR MXNetJVM: Couldn't find native library mxnet-scala
gigasquid commented 6 years ago

@aaelony I think that error might be because you don't have the right scala dependency for your system. In the project.clj file, you need to comment out the one for osx and uncomment the one for linux-cpu

So it should look like this:

:dependencies [[org.clojure/clojure "1.9.0"]
                 [t6/from-scala "0.3.0"]
                 ;; Choose the right dependency for your system
                 ;[org.apache.mxnet/mxnet-full_2.11-osx-x86_64-cpu "1.2.1"]
                 [org.apache.mxnet/mxnet-full_2.11-linux-x86_64-cpu "1.2.1"]
                 ;[org.apache.mxnet/mxnet-full_2.11-linux-x86_64-gpu "1.2.1"]
                 [org.clojure/tools.logging "0.4.0"]
                 [org.apache.logging.log4j/log4j-core "2.8.1"]
                 [org.apache.logging.log4j/log4j-api "2.8.1"]
                 [org.slf4j/slf4j-log4j12 "1.7.25" :exclusions [org.slf4j/slf4j-api]]]

Please note that I just pushed a commit to upgrade to 1.2.1 which is currently in the staging repo in apache. If you have any troubles, you can try to reset it to 1.2.0.

Let me know if that helps.

aaelony commented 6 years ago

Unfortunately, that didn't change anything. I already had the change you mention for the project.clj in place. I tried both 1.2.1 and 1.2.0 and neither pass lein test.

Here is the output: 

lein test

INFO  MXNetJVM: Try loading mxnet-scala from native path.
INFO  MXNetJVM: Try loading mxnet-scala-linux-x86_64-gpu from native path.
INFO  MXNetJVM: Try loading mxnet-scala-linux-x86_64-cpu from native path.
WARN  MXNetJVM: MXNet Scala native library not found in path. Copying native library from the archive. Consider installing the library somewhere in the path (for Windows: PATH, for Linux: LD_LIBRARY_PATH), or specifying by Java cmd option -Djava.library.path=[lib path].
INFO  org.apache.mxnet.util.NativeLibraryLoader: Loading libmxnet-scala.so from /lib/native/ copying to mxnet-scala
ERROR org.apache.mxnet.util.NativeLibraryLoader: Couldn't load copied link file: java.lang.UnsatisfiedLinkError: /tmp/mxnet3717881072106464854/mxnet-scala: libopencv_imgcodecs.so.3.4: cannot open shared object file: No such file or directory
ERROR MXNetJVM: Couldn't find native library mxnet-scala
Exception in thread "main" java.lang.UnsatisfiedLinkError: /tmp/mxnet3717881072106464854/mxnet-scala: libopencv_imgcodecs.so.3.4: cannot open shared object file: No such file or directory, compiling:(base.clj:21:19)
...
Caused by: java.lang.UnsatisfiedLinkError: /tmp/mxnet3717881072106464854/mxnet-scala: libopencv_imgcodecs.so.3.4: cannot open shared object file: No such file or directory

Also, incidentally:

$ sudo apt install libopencv-imgcodecs-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
libopencv-imgcodecs-dev is already the newest version (3.2.0+dfsg-4build2).
libopencv-imgcodecs-dev set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 47 not upgraded.

Perhaps you can add the dependency for libopencv_imgcodecs.so.3.4 as well?

gigasquid commented 6 years ago

@aaelony - thanks so much for working through these install issues. It will enable us to document the steps and make it smoother for people in the future.

The good news is that I recognize this error msg :)

Try this:

sudo add-apt-repository ppa:timsc/opencv-3.4
sudo apt-get update
sudo apt install libopencv-imgcodecs3.4

and see if that moves it forward

aaelony commented 6 years ago

Hi @gigasquid, I am happy to report that adding the above ppa for opencv-3.4 led to successfully installing the cpu version on ubuntu 18.04. I did get another error after the above commands, i.e. 

Caused by: java.lang.UnsatisfiedLinkError: /tmp/mxnet5694919404105839977/mxnet-scala: /usr/lib/x86_64-linux-gnu/libcurl.so.4: version `CURL_OPENSSL_3' not found (required by /tmp/mxnet5694919404105839977/mxnet-scala)

Revealing that apparently, Ubuntu 18.04 prefers libcurl4. That said installing libcurl3 solved the issue. i.e.

sudo apt install libcurl3

So finally here is what lein test produces for me now on 18.04:

$ clear; lein test

INFO  MXNetJVM: Try loading mxnet-scala from native path.
INFO  MXNetJVM: Try loading mxnet-scala-linux-x86_64-gpu from native path.
INFO  MXNetJVM: Try loading mxnet-scala-linux-x86_64-cpu from native path.
WARN  MXNetJVM: MXNet Scala native library not found in path. Copying native library from the archive. Consider installing the library somewhere in the path (for Windows: PATH, for Linux: LD_LIBRARY_PATH), or specifying by Java cmd option -Djava.library.path=[lib path].
INFO  org.apache.mxnet.util.NativeLibraryLoader: Loading libmxnet-scala.so from /lib/native/ copying to mxnet-scala
[09:20:43] src/io/iter_mnist.cc:113: MNISTIter: load 60000 images, shuffle=1, shape=[100,1,28,28]
[09:20:44] src/io/iter_mnist.cc:113: MNISTIter: load 10000 images, shuffle=1, shape=[100,1,28,28]
WARN  org.apache.mxnet.WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.NDArray was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing

lein test dev.generator-test

lein test org.apache.clojure-mxnet.conv-test
WARN  org.apache.mxnet.WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.Symbol was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing
Epoch  0  Train- [accuracy 0.76921666]
Epoch  0  Time cost- 5552
Epoch  0  Validation-  [accuracy 0.9295]
Score [accuracy 0.9295]

lein test org.apache.clojure-mxnet.executor-test

lein test org.apache.clojure-mxnet.io-test
[09:20:54] src/io/iter_mnist.cc:110: MNISTIter: load 60000 images, shuffle=1, shape=(100,784)
[09:20:55] src/io/iter_mnist.cc:110: MNISTIter: load 60000 images, shuffle=1, shape=(100,784)
[09:20:55] src/io/iter_image_recordio_2.cc:170: ImageRecordIOParser2: data/cifar/train.rec, use 3 threads for decoding..
[09:20:56] src/io/iter_mnist.cc:110: MNISTIter: load 60000 images, shuffle=1, shape=(100,784)
[09:20:58] src/io/iter_mnist.cc:110: MNISTIter: load 60000 images, shuffle=1, shape=(100,784)
WARN  org.apache.mxnet.WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.io.MXDataIter was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing
WARN  org.apache.mxnet.WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.Executor was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing
[09:20:59] src/io/iter_mnist.cc:110: MNISTIter: load 60000 images, shuffle=1, shape=(100,784)

lein test org.apache.clojure-mxnet.kvstore-test

lein test org.apache.clojure-mxnet.module-test
INFO  org.apache.mxnet.module.Module: Saved checkpoint to test-0000.params
INFO  org.apache.mxnet.module.Module: Saved optimizer state to test-0000.states
INFO  org.apache.mxnet.Model: Auto - select kvstore type = local_update_cpu
INFO  org.apache.mxnet.Model: Auto - select kvstore type = local_update_cpu
INFO  org.apache.mxnet.module.Module: Saved checkpoint to test-0000.params
INFO  org.apache.mxnet.module.Module: Saved optimizer state to test-0000.states

lein test org.apache.clojure-mxnet.ndarray

lein test org.apache.clojure-mxnet.ndarray-test
[09:20:59] src/ndarray/./ndarray_function-inl.h:97: The operator onehot_encode is deprecated; use one_hot instead.

lein test org.apache.clojure-mxnet.operator-test
WARN  org.apache.mxnet.WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.KVStore was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing
Embedded symbol: {
  "nodes": [
    {
      "op": "null", 
      "name": "data", 
      "inputs": []
    }, 
    {
      "op": "null", 
      "name": "embed_weight", 
      "attrs": {
        "input_dim": "10", 
        "output_dim": "4"
      }, 
      "inputs": []
    }, 
    {
      "op": "Embedding", 
      "name": "embed", 
      "attrs": {
        "input_dim": "10", 
        "output_dim": "4"
      }, 
      "inputs": [[0, 0, 0], [1, 0, 0]]
    }
  ], 
  "arg_nodes": [0, 1], 
  "node_row_ptr": [0, 1, 2, 3], 
  "heads": [[2, 0, 0]], 
  "attrs": {"mxnet_version": ["int", 10200]}
}

lein test org.apache.clojure-mxnet.optimizer-test
Testing optimizer -  sgd
Testing optimizer -  dcasgd
Testing optimizer -  nag
Testing optimizer -  ada-delta
Testing optimizer -  rms-prop
Testing optimizer -  ada-grad
Testing optimizer -  adam
Testing optimizer -  sgld

lein test org.apache.clojure-mxnet.random-test

lein test org.apache.clojure-mxnet.shape-test

lein test org.apache.clojure-mxnet.symbol

lein test org.apache.clojure-mxnet.symbol-test
Symbol Outputs:
    output[0]=composed(0)
Variable:data
Variable:fc1_weight
Variable:fc1_bias
--------------------
Op:FullyConnected, Name=fc1
Inputs:
    arg[0]=data(0) version=0
    arg[1]=fc1_weight(0) version=0
    arg[2]=fc1_bias(0) version=0
Attrs:
    num_hidden=10
Variable:fc2_weight
Variable:fc2_bias
--------------------
Op:FullyConnected, Name=fc2
Inputs:
    arg[0]=fc1(0)
    arg[1]=fc2_weight(0) version=0
    arg[2]=fc2_bias(0) version=0
Attrs:
    num_hidden=100
Variable:fc3_weight
Variable:fc3_bias
--------------------
Op:FullyConnected, Name=fc3
Inputs:
    arg[0]=fc2(0)
    arg[1]=fc3_weight(0) version=0
    arg[2]=fc3_bias(0) version=0
Attrs:
    num_hidden=10
--------------------
Op:Activation, Name=activation0
Inputs:
    arg[0]=fc3(0)
Attrs:
    act_type=relu
Variable:fc4_weight
Variable:fc4_bias
--------------------
Op:FullyConnected, Name=composed
Inputs:
    arg[0]=activation0(0)
    arg[1]=fc4_weight(0) version=0
    arg[2]=fc4_bias(0) version=0
Attrs:
    num_hidden=20

lein test org.apache.clojure-mxnet.test-util

lein test org.apache.clojure-mxnet.util-test
val: 1 fails spec: :org.apache.clojure-mxnet.util-test/x predicate: string?

Ran 174 tests containing 577 assertions.
0 failures, 0 errors.
INFO  org.apache.mxnet.util.NativeLibraryLoader: Deleting /tmp/mxnet2036263063689829261/mxnet-scala
INFO  org.apache.mxnet.util.NativeLibraryLoader: Deleting /tmp/mxnet2036263063689829261

So it works! There are a few "leak" warnings and I'm not sure what to make of the mxnet-scala deletions from /tmp/mxnet* at the end of that , but there are 0 failures, 0 errors using [org.apache.mxnet/mxnet-full_2.11-linux-x86_64-cpu "1.2.0"]

I should also add that this is using java 8.

$ java -version
java version "1.8.0_171"
Java(TM) SE Runtime Environment (build 1.8.0_171-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.171-b11, mixed mode)

I guess testing the GPU version would be next up when I find time... ;)

gigasquid commented 6 years ago

@aaelony Yay! 🎉 - Thanks again for working through all the rough edges. I'll update the README to make it easier for people on Ubuntu to get started.

mariussoutier commented 5 years ago

@whilo have you ever found a solution to this on your local laptop? I seem to have the same issue on Ubuntu 18 with Scala, both 1.2.1 and 1.3.0, as well as CPU and GPU have this problem.

whilo commented 5 years ago

@mariussoutier Unfortunately I have not had time yet to get back to this. I don't know whether it works right now.