search

dremio-hub / dremio-flight-connector

Dremio Flight connector. Access Dremio using Arrow flight
40 stars 8 forks source link

dremio.conf settings #9

Closed martindut closed 4 years ago

martindut commented 5 years ago

Dremio client version: Dremio version: Build: 4.0.0-201909121834570395-c7a5071 Edition: Community Edition Python/Arrow version: Operating System: Windows

Description

I'm trying to setup dremio-flight-connector on our Azure deployment.

What I Did

I followed all the steps as described, but I'm not sure how to put the settings in the dremio.conf file. Do I just add those settings at the bottom of the file, as is: -Ddremio.flight.use-ssl=true -Ddremio.flight.port=47470 -Ddremio.flight.host=localhost

martindut commented 5 years ago

Hi @rymurr I would really like to get the flight connector working, so any advise and help would be much appreciated. Here is what if have done so far.

1) Fire up docker container dremio/dremio-oss:latest with the dremio-flight-connector-0.11.0-SNAPSHOT-shaded.jar file copied to /opt/dremio/jars/ 2) Using dremio_client, I can connect to Dremio

from dremio_client import init
import pandas as pd
client = init(simple_client=False)
client.data.tdr.tdr_snapshot_omi_holdings_obelix_v1_0
vds = client.data.tdr.tdr_snapshot_omi_holdings_obelix_v1_0.get()

3) But this command fails df = vds.query() FlightUnavailableError: gRPC returned unavailable error, with message: Transport closed

Can you please point me in some directions where I can start to find the problem.

Thanks

4) When I run this

from dremio_client.flight import query
sql = "SELECT * FROM snapshot_omi_holdings_obelix_v1.0"
df = query(sql, "localhost", 47470, "martindut", "Dr3m10test")

This error pops up: AttributeError: 'str' object has no attribute 'get_flight_info'

rymurr commented 5 years ago

Hey Martin,

Apologies for the delay in responding. I have introduced a bug #10 which doesn't expose the flight port to anything but localhost. This will be corrected in the next patch fix. In the meantime you can set the environment variable JAVA_EXTRA_OPTS="$JAVA_EXTRA_OPTS -Ddremio.flight.host=0.0.0.0" to force it to listen on all interfaces