search

jcrist / skein

A tool and library for easily deploying applications on Apache YARN
https://jcristharif.com/skein/
BSD 3-Clause "New" or "Revised" License
142 stars 38 forks source link

run echo_server failed #198

Open better629 opened 4 years ago

better629 commented 4 years ago

I run the skein client in Mac os, The error info

19/12/03 09:50:46 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
19/12/03 09:50:47 INFO client.RMProxy: Connecting to ResourceManager at /192.168.3.21:8032
Dec 03, 2019 9:50:47 AM com.anaconda.skein.shaded.io.grpc.netty.GrpcSslContexts defaultSslProvider
信息: netty-tcnative unavailable (this may be normal)
java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty_tcnative_osx_x86_64, netty_tcnative_x86_64, netty_tcnative]
    at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:104)
    at com.anaconda.skein.shaded.io.netty.handler.ssl.OpenSsl.loadTcNative(OpenSsl.java:440)
    at com.anaconda.skein.shaded.io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:97)
    at com.anaconda.skein.shaded.io.grpc.netty.GrpcSslContexts.defaultSslProvider(GrpcSslContexts.java:244)
    at com.anaconda.skein.shaded.io.grpc.netty.GrpcSslContexts.configure(GrpcSslContexts.java:171)
    at com.anaconda.skein.shaded.io.grpc.netty.GrpcSslContexts.forServer(GrpcSslContexts.java:151)
    at com.anaconda.skein.Driver.startServer(Driver.java:124)
    at com.anaconda.skein.Driver.run(Driver.java:287)
    at com.anaconda.skein.Driver.main(Driver.java:175)
    Suppressed: java.lang.UnsatisfiedLinkError: could not load a native library: com_anaconda_skein_shaded_netty_tcnative_osx_x86_64
        at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:233)
        at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:96)
        ... 8 more
    Caused by: java.io.FileNotFoundException: META-INF/native/libcom_anaconda_skein_shaded_netty_tcnative_osx_x86_64.jnilib
        at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:168)
        ... 9 more
        Suppressed: java.lang.UnsatisfiedLinkError: no com_anaconda_skein_shaded_netty_tcnative_osx_x86_64 in java.library.path
            at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
            at java.lang.Runtime.loadLibrary0(Runtime.java:870)
            at java.lang.System.loadLibrary(System.java:1122)
            at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
            at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:316)
            at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136)
            ... 9 more
            Suppressed: java.lang.UnsatisfiedLinkError: no com_anaconda_skein_shaded_netty_tcnative_osx_x86_64 in java.library.path
                at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
                at java.lang.Runtime.loadLibrary0(Runtime.java:870)
                at java.lang.System.loadLibrary(System.java:1122)
                at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                at java.lang.reflect.Method.invoke(Method.java:498)
                at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:336)
                at java.security.AccessController.doPrivileged(Native Method)
                at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:328)
                at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:306)
                ... 10 more
    Suppressed: java.lang.UnsatisfiedLinkError: could not load a native library: com_anaconda_skein_shaded_netty_tcnative_x86_64
        at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:233)
        at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:96)
        ... 8 more
    Caused by: java.io.FileNotFoundException: META-INF/native/libcom_anaconda_skein_shaded_netty_tcnative_x86_64.jnilib
        at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:168)
        ... 9 more
        Suppressed: java.lang.UnsatisfiedLinkError: no com_anaconda_skein_shaded_netty_tcnative_x86_64 in java.library.path
            at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
            at java.lang.Runtime.loadLibrary0(Runtime.java:870)
            at java.lang.System.loadLibrary(System.java:1122)
            at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
            at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:316)
            at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136)
            ... 9 more
            Suppressed: java.lang.UnsatisfiedLinkError: no com_anaconda_skein_shaded_netty_tcnative_x86_64 in java.library.path
                at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
                at java.lang.Runtime.loadLibrary0(Runtime.java:870)
                at java.lang.System.loadLibrary(System.java:1122)
                at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                at java.lang.reflect.Method.invoke(Method.java:498)
                at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:336)
                at java.security.AccessController.doPrivileged(Native Method)
                at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:328)
                at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:306)
                ... 10 more
    Suppressed: java.lang.UnsatisfiedLinkError: could not load a native library: com_anaconda_skein_shaded_netty_tcnative
        at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:233)
        at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:96)
        ... 8 more
    Caused by: java.io.FileNotFoundException: META-INF/native/libcom_anaconda_skein_shaded_netty_tcnative.jnilib
        at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:168)
        ... 9 more
        Suppressed: java.lang.UnsatisfiedLinkError: no com_anaconda_skein_shaded_netty_tcnative in java.library.path
            at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
            at java.lang.Runtime.loadLibrary0(Runtime.java:870)
            at java.lang.System.loadLibrary(System.java:1122)
            at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
            at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:316)
            at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136)
            ... 9 more
            Suppressed: java.lang.UnsatisfiedLinkError: no com_anaconda_skein_shaded_netty_tcnative in java.library.path
                at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
                at java.lang.Runtime.loadLibrary0(Runtime.java:870)
                at java.lang.System.loadLibrary(System.java:1122)
                at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                at java.lang.reflect.Method.invoke(Method.java:498)
                at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:336)
                at java.security.AccessController.doPrivileged(Native Method)
                at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:328)
                at com.anaconda.skein.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:306)
                ... 10 more

Dec 03, 2019 9:50:47 AM com.anaconda.skein.shaded.io.grpc.netty.GrpcSslContexts defaultSslProvider
信息: Conscrypt not found (this may be normal)
Dec 03, 2019 9:50:47 AM com.anaconda.skein.shaded.io.grpc.netty.GrpcSslContexts defaultSslProvider
信息: Jetty ALPN unavailable (this may be normal)
java.lang.ClassNotFoundException: com/anaconda/skein/shaded/org/eclipse/jetty/alpn/ALPN
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at com.anaconda.skein.shaded.io.grpc.netty.JettyTlsUtil.isJettyAlpnConfigured(JettyTlsUtil.java:64)
    at com.anaconda.skein.shaded.io.grpc.netty.GrpcSslContexts.findJdkProvider(GrpcSslContexts.java:266)
    at com.anaconda.skein.shaded.io.grpc.netty.GrpcSslContexts.defaultSslProvider(GrpcSslContexts.java:248)
    at com.anaconda.skein.shaded.io.grpc.netty.GrpcSslContexts.configure(GrpcSslContexts.java:171)
    at com.anaconda.skein.shaded.io.grpc.netty.GrpcSslContexts.forServer(GrpcSslContexts.java:151)
    at com.anaconda.skein.Driver.startServer(Driver.java:124)
    at com.anaconda.skein.Driver.run(Driver.java:287)
    at com.anaconda.skein.Driver.main(Driver.java:175)

19/12/03 09:50:47 ERROR skein.Driver: Error running Driver
java.lang.IllegalStateException: Could not find TLS ALPN provider; no working netty-tcnative, Conscrypt, or Jetty NPN/ALPN available
    at com.anaconda.skein.shaded.io.grpc.netty.GrpcSslContexts.defaultSslProvider(GrpcSslContexts.java:258)
    at com.anaconda.skein.shaded.io.grpc.netty.GrpcSslContexts.configure(GrpcSslContexts.java:171)
    at com.anaconda.skein.shaded.io.grpc.netty.GrpcSslContexts.forServer(GrpcSslContexts.java:151)
    at com.anaconda.skein.Driver.startServer(Driver.java:124)
    at com.anaconda.skein.Driver.run(Driver.java:287)
    at com.anaconda.skein.Driver.main(Driver.java:175)
Error: Failed to start java process
Exception ignored in: <object repr() failed>
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/skein/core.py", line 491, in __del__
    self.close()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/skein/core.py", line 478, in close
    if self._proc is not None:
AttributeError: _proc
better629 commented 4 years ago

The author reply: 

Skein is only designed to be run from Linux. Historically this has been fine - most Hadoop clusters have Linux machines for both compute and edge nodes.
better629 commented 4 years ago

The commands in echo_server spec.yaml is not working,
It happens when I run the echo_server on master branch. The _check_keys will find the commands is not a defined key. So I use the script instead. I have refered the echo_server example.

Data-drone commented 4 years ago

I had issues with commands as well what do you mean buy you used script instead?

better629 commented 4 years ago

I had issues with commands as well what do you mean buy you used script instead?

yes, use the script like "script": "source environment/bin/activate & python server.py" instead of command in *spec.yaml.