Python Client for Pinot Requires a Username

[MinuraPunchihewa]

I have been trying to connect to Apache Pinot using the Python client pinotdb for quite a while now, to no avail.

I am simply running Pinot as a Docker container using the following command, docker run -p 9000:9000 apachepinot/pinot:0.9.3 QuickStart -type hybrid

Given below is what my Python code looks like,

from pinotdb import connect

conn = connect(host='localhost', port=9000, path='/query/sql', scheme='http')
curs = conn.cursor()
curs.execute("""
    SELECT * FROM airlineStats
""")

Every time I run this, I get the following error, AttributeError: 'NoneType' object has no attribute '_username'

Is there a default username or password that comes with Pinot or is there some way that I can create a user? It looks like the problem is that a username is somehow required.

I realize that I this may not be the right place to raise this issue, but I have already created an issue on the pinotdb repository, but I am yet to receive a response.

Any help on this matter would be greatly appreciated.

Here is the complete stack trace of the error I am facing,

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [12], in <cell line: 1>()
----> 1 curs.execute("""
      2     SELECT * FROM airlineStats
      3 """)
      4 for row in curs:
      5     print(row)

File ~\Anaconda3\envs\pinotdb\lib\site-packages\pinotdb\db.py:56, in check_closed.<locals>.g(self, *args, **kwargs)
     54 if self.closed:
     55     raise exceptions.Error(f"{self.__class__.__name__} already closed")
---> 56 return f(self, *args, **kwargs)

File ~\Anaconda3\envs\pinotdb\lib\site-packages\pinotdb\db.py:448, in Cursor.execute(self, operation, parameters)
    441 @check_closed
    442 def execute(self, operation, parameters=None):
    443     query = self.finalize_query_payload(operation, parameters)
    445     r = self.session.post(
    446         self.url,
    447         json=query,
--> 448         auth=(self.auth._username, self.auth._password))
    449     return self.normalize_query_response(query, r)

AttributeError: 'NoneType' object has no attribute '_username'

[xiangfu0]

Have you tried the latest pinotdb version? This should be fixed already.

[MinuraPunchihewa]

@xiangfu0 I just upgraded to pinotdb-0.4.3 and that username error does not seem to be coming up anymore. However, now I get this,

DatabaseError: Query

{'sql': 'SELECT * FROM airlineStats'} timed out: Out of -1, only -1 responded, while needed was -1

Do you know what it could be?

This is my code,

from pinotdb import connect

conn = connect(host='localhost', port=9000, path='/query/sql', scheme='http')
curs = conn.cursor()

curs.execute("SELECT * FROM airlineStats")
for row in curs:
    print(row)

[xiangfu0]

cc: @walterddr We will check

[xiangfu0]

I've tested the current master code, seems it's working, let me cut a new release 0.4.4 then

[MinuraPunchihewa]

@xiangfu0 That would be greatly appreciated. Please let me know when you make the release.

[xiangfu0]

0.4.4 is published. And this is the example I'm using to test: https://github.com/python-pinot-dbapi/pinot-dbapi/blob/master/examples/pinot_quickstart_batch.py

[walterddr]

yeah. it should've been fixed by https://github.com/python-pinot-dbapi/pinot-dbapi/commit/8d0ca2be6a54921224028c686589a74c310977da#diff-f3f44b3ab7bb620101c696ae06a9355d31e982b890a382f64157046fc7095643

[MinuraPunchihewa]

@xiangfu0 @walterddr It seems to be working perfectly now! Thank you very much. I will close this issue. I was hoping that I could ask you for one more favor though. With Pinot, what is the command for getting all the available tables (equivalent of SHOW TABLES) and for getting the columns of a particular table (equivalent of SHOW TABLES IN )?

[walterddr]

i dont think we support such DML DDL syntax yet. please feel free to file another github issues for these syntax (such as: SHOW TABLES, DESCRIBE TABLE <tbl_name>, etc)

[MinuraPunchihewa]

@walterddr Sounds good. I will do that now. Is there a way to get it through the Python client? A list of tables in the database and the columns along with their types of a specific table?

[walterddr]

you can do so by querying the REST API of pinot but that's not part of the DDL/DML, see: https://docs.pinot.apache.org/users/api/pinot-rest-admin-interface

[MinuraPunchihewa]

@walterddr Ah, this is great! But, could you please explain something to me? To get my Python client running, I had to start up my Docker instance the same way given in the example by @xiangfu0, docker run --name pinot-quickstart -p 2123:2123 -p 9000:9000 -p 8000:8000 -d apachepinot/pinot:latest QuickStart -type batch

I have a basic idea of the components of Pinot, but could you please help me understand what these ports that we are exposing are?

[walterddr]

just 9000 is the one you should use

[MinuraPunchihewa]

@walterddr Connecting the Python client to port 9000 does not work for me. Similar to the example given, I had to use port 8000. When I issue queries to port 9000, I get this error, {'sql': 'SELECT * FROM baseballStats LIMIT 5'} timed out: Out of -1, only -1 responded, while needed was -1

However, to get the list of tables using the REST interface like you mentioned, I need to send the requests to port 9000.

[MinuraPunchihewa]

@xiangfu0 @walterddr Any comments on this? From what I can understand, 9000 here is the port for the Controller and 8000 is the port for the Broker?

[walterddr]

@MinuraPunchihewa correct.

• all the SQL statement should be hitting broker port (8000 or 8099 depending on your default configuration))
• all the controller endpoints (REST API calls) goes to 9000 controller port.

[MinuraPunchihewa]

@walterddr Thank you so much for all your help. I really appreciate it.