Closed reijin90 closed 6 years ago
Does it have sth. to do with the plugin.bundles within the config.properties? These are not set for my presto server.
root@presto:/presto-server-0.200/etc# cat config.properties coordinator=true node-scheduler.include-coordinator=true query.max-memory=4GB query.max-memory-per-node=1GB http-server.http.port=8080 discovery-server.enabled=true discovery.uri=http://presto:8080
in case this helps, here is my dir tree:
. |-bin |---procname |-----Linux-ppc64le |-----Linux-x86_64 |-data |---var |-----log |-----run |-etc |---catalog |-lib |-plugin |---accumulo |---atop |---blackhole |---cassandra |---ethereum |---example-http |---geospatial |---hive-hadoop2 |---jmx |---kafka |---localfile |---memory |---ml |---mongodb |---mysql |---password-authenticators |---postgresql |---presto-thrift |---raptor |---redis |---redshift |---resource-group-managers |---sqlserver |---teradata-functions |---tpcds |---tpch
And the plugin seems to get installed properly:
2018-05-08T10:30:37.295Z INFO main com.facebook.presto.server.PluginManager -- Loading plugin /presto-server-0.200/data/plugin/ethereum -- 2018-05-08T10:30:38.627Z INFO main com.facebook.presto.server.PluginManager Installing im.xiaoyao.presto.ethereum.EthereumPlugin 2018-05-08T10:30:38.678Z INFO main com.facebook.presto.server.PluginManager Registering connector ethereum 2018-05-08T10:30:38.689Z INFO main com.facebook.presto.server.PluginManager Registering functions from im.xiaoyao.presto.ethereum.udfs.EthereumUDFs 2018-05-08T10:30:38.775Z INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /presto-server-0.200/data/plugin/ethereum --
aha, I have not yet upgraded dependencies to support version 0.200. The latest version is 0.191. Would you mind switching to ver 0.191 and try?
Ok, I tried it with 191, but same issue. What I'm actually trying to achieve is creating a Docker container out of this. I'm running OpenJDK 8.
2018-05-08T19:30:55.089Z INFO main com.facebook.presto.metadata.StaticCatalogStore -- Loading catalog etc/catalog/ethereum.properties -- 2018-05-08T19:30:55.105Z ERROR main com.facebook.presto.server.PrestoServer No factory for connector ethereum java.lang.IllegalArgumentException: No factory for connector ethereum at com.google.common.base.Preconditions.checkArgument(Preconditions.java:191)
If you are interested, here is the Dockerfile (largely based on docker-presto):
# Pull Base Image
FROM openjdk:8
# Set Maintainer Details
MAINTAINER Zhichun Wu <zhicwu@gmail.com>
# Set Environment Variables
ENV PRESTO_VERSION=0.191 PRESTO_HOME=/presto BASE_URL=https://repo1.maven.org/maven2/com/facebook/presto
# Download Presto
RUN apt-get update \
&& apt-get install -y maven wget git python \
&& wget ${BASE_URL}/presto-server/${PRESTO_VERSION}/presto-server-${PRESTO_VERSION}.tar.gz \
${BASE_URL}/presto-cli/${PRESTO_VERSION}/presto-cli-${PRESTO_VERSION}-executable.jar \
&& rm -rf /var/lib/apt/lists/*
# Install Presto
RUN chmod +x presto-*executable.jar \
&& tar zxvf presto-server-${PRESTO_VERSION}.tar.gz \
&& ln -s presto-server-${PRESTO_VERSION} presto \
&& mv *.jar presto/. \
&& cd presto \
&& ln -s presto-cli-${PRESTO_VERSION}-executable.jar presto-cli \
&& cd -
# install ethereum blockchain adapter from https://github.com/xiaoyao1991/presto-ethereum
RUN git clone https://github.com/xiaoyao1991/presto-ethereum \
&& cd presto-ethereum \
&& mvn clean package \
&& mkdir -p ${PRESTO_HOME}/etc/catalog \
&& test -f target/presto-ethereum-*-plugin.tar.gz \
&& mkdir -p ${PRESTO_HOME}/plugin/ethereum \
&& mv target/presto-ethereum-*-plugin.tar.gz . \
&& tar xfz presto-ethereum-*-plugin.tar.gz -C $PRESTO_HOME/plugin/ethereum --strip-components=1
WORKDIR $PRESTO_HOME
VOLUME ["$PRESTO_HOME/etc", "$PRESTO_HOME/data"]
EXPOSE 8080
ENTRYPOINT ["./bin/launcher", "run"]
Could you post the etc/catalog/ethereum.properties
? You can mask out endpoints if you don't want to.
root@presto:/presto-server-0.191# cat etc/catalog/ethereum.properties
connector.name=ethereum
# You can connect through Ethereum HTTP JSON RPC endpoint
# IMPORTANT - for local testing start geth with rpcport # geth --rpc --rpcaddr "127.0.0.1" --rpcport "8545"
# ethereum.jsonrpc=http://localhost:8545/
# Or you can connect through IPC socket
# ethereum.ipc=/path/to/ipc_socketfile
# Or you can connect to Infura
ethereum.infura=https://mainnet.infura.io/YYYYYYYYYY
edit: Strangely enough it seems I'm able to start the presto-cli (but queries lead to errors):
root@presto:/presto-server-0.191# ./presto-cli --server 127.0.0.1:8080 --catalog ethereum --schema default
presto:default> show tables;
Error running command: Error starting query at http://127.0.0.1:8080/v1/statement returned an invalid response: JsonResponse{statusCode=404, statusMessage=Not Found, headers={cache-control=[must-revalidate,no-cache,no-store], content-length=[244], content-type=[text/html;charset=iso-8859-1], date=[Tue, 08 May 2018 20:59:50 GMT]}, hasValue=false} [Error: <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Not Found</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /v1/statement. Reason:
<pre> Not Found</pre></p>
</body>
</html>
]
I played around with your Dockerfile and it works for me.
I had issues running it directly cuz I don't have the data volume images. So I remove the ENTRYPOINT
, and I manually ssh into the container and created the etc/catalog/ethereum/ethereum.properties
, etc/config.properties
, and etc/jvm.config
. And then it works.
I guess the problem lies in your data volume? maybe it's not mounted correctly?
@reijin90 if you want, we can talk offline. I can do telegram, wechat, whatsapp, gitter.
Thanks! I noticed, that you wrote "etc/catalog/ethereum/ethereum.properties" in your answer, but in the readme in the repo it says "etc/catalog/ethereum.properties" so that was the issue with the factory, but I still cant query as I'm getting 404 errors
I'm currently in gitter btw :)
@reijin90 sorry for being late. I didn't have gitter run in background. I'm online now.
It seems my configs are availble normally and in the correct folders. I start the docker container based form the Dockerfile I posted earlier.
Is etc/catalog/ethereum.properties or etc/catalog/ethereum/ethereum.properties correct?
root@presto:/presto-server-0.191# cat etc/catalog/ethereum.properties
connector.name=ethereum
# You can connect through Ethereum HTTP JSON RPC endpoint
# IMPORTANT - for local testing start geth with rpcport # geth --rpc --rpcaddr "127.0.0.1" --rpcport "8545"
# ethereum.jsonrpc=http://localhost:8545/
# Or you can connect through IPC socket
# ethereum.ipc=/path/to/ipc_socketfile
# Or you can connect to Infura
ethereum.infura=https://mainnet.infura.io/XXXXXXX
root@presto:/presto-server-0.191# cat etc/config.properties
coordinator=false
query.max-memory=4GB
query.max-memory-per-node=1GB
http-server.http.port=8080
discovery.uri=http://presto:8080
root@presto:/presto-server-0.191# cat etc/jvm.config
-server
-Xmx2G
-XX:+UseG1GC
-XX:G1HeapRegionSize=32M
-XX:+UseGCOverheadLimit
-XX:+ExplicitGCInvokesConcurrent
-XX:+HeapDumpOnOutOfMemoryError
-XX:OnOutOfMemoryError=kill -9 0
root@presto:/presto-server-0.191# cat etc/log.properties
com.facebook.presto=INFO
root@presto:/presto-server-0.191# cat etc/node.properties
node.environment=production
node.id=e0e50bdc-52ab-11e8-afa9-42010a800002
node.data-dir=/presto/data
root@presto:/presto-server-0.191#
Can you spot any mistake in my configs? I'm still getting 404 errors when trying to access the infura endpoint from the presto-cli within the same container.
I'm online in gitter btw :)
@reijin90 Hi, I am encounter similar problem when deploying. Could you please share your settings?
I'm getting an error that the factory for the ethereum connector is missing, what am I doing wrong? I'm no presto expert, so I'm a bit clueless here...