BlazingDB / blazingsql

BlazingSQL is a lightweight, GPU accelerated, SQL engine for Python. Built on RAPIDS cuDF.
https://blazingsql.com
Apache License 2.0
1.92k stars 181 forks source link

It'd be great if `ok` was returned when creating a file system connection #1529

Closed lucharo closed 3 years ago

lucharo commented 3 years ago

Hello, I am using blazing with two main HDFS NameNodes, I do not know how the NameNodes work but I always seem to get only one of them working at a single time. I would like to be able to tell if one NameNode was registered succesfully programatically, but unfortunately, because this ok variable is not returned I cannot do that only the success/error message is printed. Could it be possible to have that ok variable returned?? Otherwise how can I tell if my HDFS connection was registered succesfully programmatically??

my code:

import getpass
from blazingsql import BlazingContext
bc = BlazingContext()
bc.hdfs(
    'dbnn',
    host ='namenode.company.com',
    port = 8020,
    user=getpass.getuser(),
    kerb_ticket="/tmp/krb5cc_132855"
)
"""output
HDFS Storage Plugin Error: HDFS list directory failed, errno: 255 (Unknown error 255) Please check that you are connecting to the correct HDFS RPC port. Filesystem HDFS=>hdfs.driver.type:LIBHDFS|hdfs.host:namenode.company.com|hdfs.kerberos.ticket:/tmp/krb5cc_132855|hdfs.port:8020|hdfs.user:chavesrl
"""

https://github.com/BlazingDB/blazingsql/blob/febceadd9f1fb24016f355f34b2be87459661d94/pyblazing/pyblazing/apiv2/filesystem.py#L87

felipeblazing commented 3 years ago

Good point. Alot of us use Jupyter Notebooks when testing and we sometimes forget that these apis need to be invoked not just by a human who typed it in to see the output there.

lucharo commented 3 years ago

Thanks 💯