tokern / lakecli

A CLI to manage and monitor permissions in AWS Lake Formation
Apache License 2.0
25 stars 7 forks source link

no such table: #17

Open superdba111 opened 3 years ago

superdba111 commented 3 years ago

\r:iamdb> SELECT FROM database_privileges; no such table: database_privileges \r:iamdb> SELECT FROM table_privileges; no such table: table_privileges \r:iamdb>

vrajat commented 3 years ago

Hi Thanks for trying out lakecli. Lakecli writes logs to ~/.lakecli/app.log . Can you check if there are any errors in the log?

superdba111 commented 3 years ago

2020-10-22 23:33:25,840 (38148/MainThread) lakecli.sqlexecute INFO - Connect to /Users/yli/.lakecli/iamdb 2020-10-22 23:33:25,841 (38148/completion_refresh) lakecli.sqlexecute INFO

On Sun, Oct 25, 2020 at 7:42 AM Rajat Venkatesh notifications@github.com wrote:

Hi Thanks for trying out lakecli. Lakecli writes logs to ~/.lakecli/app.log . Can you check if there are any errors in the log?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/tokern/lakecli/issues/17#issuecomment-716142009, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHTW7AJ4CBU2TPX2RM7K4GLSMQMLJANCNFSM4S57TN3A .

superdba111 commented 3 years ago

any update?

On Sun, Oct 25, 2020 at 1:57 PM Yingyou yingyou@gmail.com wrote:

2020-10-22 23:33:25,840 (38148/MainThread) lakecli.sqlexecute INFO - Connect to /Users/yli/.lakecli/iamdb 2020-10-22 23:33:25,841 (38148/completion_refresh) lakecli.sqlexecute INFO

  • Connect to /Users/yli/.lakecli/iamdb 2020-10-22 23:37:16,679 (38148/MainThread) lakecli.main ERROR - sql: "SELECT FROM table_priviledges WHERE principal like 'yli%';", error: OperationalError('no such table: table_priviledges') 2020-10-22 23:37:16,679 (38148/MainThread) lakecli.main ERROR - sql: "SELECT FROM table_priviledges WHERE principal like 'yli%';", error: OperationalError('no such table: table_priviledges') 2020-10-22 23:37:16,683 (38148/MainThread) lakecli.main ERROR - traceback: 'Traceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 74, in run\n for result in special.execute(cur, sql):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/packages/special/main.py", line 66, in execute\n raise CommandNotFound\nlakecli.packages.special.main.CommandNotFound\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/main.py", line 331, in one_iteration\n for title, rows, headers, status in res:\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 77, in run\n cur.execute(sql)\nsqlite3.OperationalError: no such table: table_priviledges\n' 2020-10-22 23:37:16,683 (38148/MainThread) lakecli.main ERROR - traceback: 'Traceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 74, in run\n for result in special.execute(cur, sql):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/packages/special/main.py", line 66, in execute\n raise CommandNotFound\nlakecli.packages.special.main.CommandNotFound\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/main.py", line 331, in one_iteration\n for title, rows, headers, status in res:\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 77, in run\n cur.execute(sql)\nsqlite3.OperationalError: no such table: table_priviledges\n' 2020-10-22 23:37:46,526 (38148/MainThread) lakecli.main ERROR - sql: "SELECT FROM table_privileges WHERE principal like 'yli%';", error: OperationalError('no such table: table_privileges') 2020-10-22 23:37:46,526 (38148/MainThread) lakecli.main ERROR - sql: "SELECT FROM table_privileges WHERE principal like 'yli%';", error: OperationalError('no such table: table_privileges') 2020-10-22 23:37:46,526 (38148/MainThread) lakecli.main ERROR - traceback: 'Traceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 74, in run\n for result in special.execute(cur, sql):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/packages/special/main.py", line 66, in execute\n raise CommandNotFound\nlakecli.packages.special.main.CommandNotFound\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/main.py", line 331, in one_iteration\n for title, rows, headers, status in res:\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 77, in run\n cur.execute(sql)\nsqlite3.OperationalError: no such table: table_privileges\n' @ "app.log" 84L, 34906C 2020-10-22 23:33:25,840 (38148/MainThread) lakecli.sqlexecute INFO - Connect to /Users/yli/.lakecli/iamdb 2020-10-22 23:33:25,841 (38148/completion_refresh) lakecli.sqlexecute INFO
  • Connect to /Users/yli/.lakecli/iamdb 2020-10-22 23:37:16,679 (38148/MainThread) lakecli.main ERROR - sql: "SELECT FROM table_priviledges WHERE principal like 'yli%';", error: OperationalError('no such table: table_priviledges') 2020-10-22 23:37:16,679 (38148/MainThread) lakecli.main ERROR - sql: "SELECT FROM table_priviledges WHERE principal like 'yli%';", error: OperationalError('no such table: table_priviledges') 2020-10-22 23:37:16,683 (38148/MainThread) lakecli.main ERROR - traceback: 'Traceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 74, in run\n for result in special.execute(cur, sql):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/packages/special/main.py", line 66, in execute\n raise CommandNotFound\nlakecli.packages.special.main.CommandNotFound\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/main.py", line 331, in one_iteration\n for title, rows, headers, status in res:\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 77, in run\n cur.execute(sql)\nsqlite3.OperationalError: no such table: table_priviledges\n' 2020-10-22 23:37:16,683 (38148/MainThread) lakecli.main ERROR - traceback: 'Traceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 74, in run\n for result in special.execute(cur, sql):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/packages/special/main.py", line 66, in execute\n raise CommandNotFound\nlakecli.packages.special.main.CommandNotFound\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/main.py", line 331, in one_iteration\n for title, rows, headers, status in res:\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 77, in run\n cur.execute(sql)\nsqlite3.OperationalError: no such table: table_priviledges\n' 2020-10-22 23:37:46,526 (38148/MainThread) lakecli.main ERROR - sql: "SELECT FROM table_privileges WHERE principal like 'yli%';", error: OperationalError('no such table: table_privileges') 2020-10-22 23:37:46,526 (38148/MainThread) lakecli.main ERROR - sql: "SELECT FROM table_privileges WHERE principal like 'yli%';", error: OperationalError('no such table: table_privileges') 2020-10-22 23:37:46,526 (38148/MainThread) lakecli.main ERROR - traceback: 'Traceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 74, in run\n for result in special.execute(cur, sql):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/packages/special/main.py", line 66, in execute\n raise CommandNotFound\nlakecli.packages.special.main.CommandNotFound\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/main.py", line 331, in one_iteration\n for title, rows, headers, status in res:\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 77, in run\n cur.execute(sql)\nsqlite3.OperationalError: no such table: table_privileges\n' 2020-10-22 23:37:46,526 (38148/MainThread) lakecli.main ERROR - traceback: 'Traceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 74, in run\n for result in special.execute(cur, sql):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/packages/special/main.py", line 66, in execute\n raise CommandNotFound\nlakecli.packages.special.main.CommandNotFound\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/main.py", line 331, in one_iteration\n for title, rows, headers, status in res:\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 77, in run\n cur.execute(sql)\nsqlite3.OperationalError: no such table: table_privileges\n' 2020-10-22 23:40:17,880 (38148/MainThread) lakecli.main ERROR - sql: 'SELECT * FROM maxdb;', error: OperationalError('no such table: maxdb') 2020-10-22 23:33:25,840 (38148/MainThread) lakecli.sqlexecute INFO - Connect to /Users/yli/.lakecli/iamdb 2020-10-22 23:33:25,841 (38148/completion_refresh) lakecli.sqlexecute INFO
  • Connect to /Users/yli/.lakecli/iamdb 2020-10-22 23:37:16,679 (38148/MainThread) lakecli.main ERROR - sql: "SELECT FROM table_priviledges WHERE principal like 'yli%';", error: OperationalError('no such table: table_priviledges') 2020-10-22 23:37:16,679 (38148/MainThread) lakecli.main ERROR - sql: "SELECT FROM table_priviledges WHERE principal like 'yli%';", error: OperationalError('no such table: table_priviledges') 2020-10-22 23:37:16,683 (38148/MainThread) lakecli.main ERROR - traceback: 'Traceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 74, in run\n for result in special.execute(cur, sql):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/packages/special/main.py", line 66, in execute\n raise CommandNotFound\nlakecli.packages.special.main.CommandNotFound\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/main.py", line 331, in one_iteration\n for title, rows, headers, status in res:\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 77, in run\n cur.execute(sql)\nsqlite3.OperationalError: no such table: table_priviledges\n' 2020-10-22 23:37:16,683 (38148/MainThread) lakecli.main ERROR - traceback: 'Traceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 74, in run\n for result in special.execute(cur, sql):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/packages/special/main.py", line 66, in execute\n raise CommandNotFound\nlakecli.packages.special.main.CommandNotFound\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/main.py", line 331, in one_iteration\n for title, rows, headers, status in res:\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 77, in run\n cur.execute(sql)\nsqlite3.OperationalError: no such table: table_priviledges\n' 2020-10-22 23:37:46,526 (38148/MainThread) lakecli.main ERROR - sql: "SELECT FROM table_privileges WHERE principal like 'yli%';", error: OperationalError('no such table: table_privileges') 2020-10-22 23:37:46,526 (38148/MainThread) lakecli.main ERROR - sql: "SELECT FROM table_privileges WHERE principal like 'yli%';", error: OperationalError('no such table: table_privileges') 2020-10-22 23:37:46,526 (38148/MainThread) lakecli.main ERROR - traceback: 'Traceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 74, in run\n for result in special.execute(cur, sql):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/packages/special/main.py", line 66, in execute\n raise CommandNotFound\nlakecli.packages.special.main.CommandNotFound\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/main.py", line 331, in one_iteration\n for title, rows, headers, status in res:\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 77, in run\n cur.execute(sql)\nsqlite3.OperationalError: no such table: table_privileges\n' 2020-10-22 23:37:46,526 (38148/MainThread) lakecli.main ERROR - traceback: 'Traceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 74, in run\n for result in special.execute(cur, sql):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/packages/special/main.py", line 66, in execute\n raise CommandNotFound\nlakecli.packages.special.main.CommandNotFound\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/main.py", line 331, in one_iteration\n for title, rows, headers, status in res:\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 77, in run\n cur.execute(sql)\nsqlite3.OperationalError: no such table: table_privileges\n' 2020-10-22 23:40:17,880 (38148/MainThread) lakecli.main ERROR - sql: 'SELECT FROM maxdb;', error: OperationalError('no such table: maxdb') 2020-10-22 23:40:17,880 (38148/MainThread) lakecli.main ERROR - sql: 'SELECT FROM maxdb;', error: OperationalError('no such table: maxdb')

On Sun, Oct 25, 2020 at 7:42 AM Rajat Venkatesh notifications@github.com wrote:

Hi Thanks for trying out lakecli. Lakecli writes logs to ~/.lakecli/app.log . Can you check if there are any errors in the log?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/tokern/lakecli/issues/17#issuecomment-716142009, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHTW7AJ4CBU2TPX2RM7K4GLSMQMLJANCNFSM4S57TN3A .

vrajat commented 3 years ago

Can you run with --scan ? This is required the first time to create and load the tables.

superdba111 commented 3 years ago

~ ❯ lakecli --scan \r:iamdb> select from database_privileges; 0 rows in set Time: 0.000s \r:iamdb> select from table_privileges; 0 rows in set Time: 0.000s \r:iamdb>

but I have databases and tables there, why they cannot be seen?

On Mon, Oct 26, 2020 at 2:13 AM Rajat Venkatesh notifications@github.com wrote:

Can you run with --scan ? This is required the first time to create and load the tables.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

vrajat commented 3 years ago

Can you paste the logs for the run with --scan ?

superdba111 commented 3 years ago

~/.lakecli ❯ vim app.log 2020-10-22 23:33:25,840 (38148/MainThread) lakecli.sqlexecute INFO - Connect to /Users/yli/.lakecli/iamdb 2020-10-22 23:33:25,841 (38148/completion_refresh) lakecli.sqlexecute INFO

On Mon, Oct 26, 2020 at 9:31 AM Rajat Venkatesh notifications@github.com wrote:

Can you paste the logs for the run with --scan ?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/tokern/lakecli/issues/17#issuecomment-716585282, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHTW7AOB4VWWAEY7MGSW64LSMWB4VANCNFSM4S57TN3A .

superdba111 commented 3 years ago

now,

~/.lakecli ❯ lakecli --scan \r:iamdb> select from table_privileges; 0 rows in set Time: 0.001s \r:iamdb> \r:iamdb> select from database_privileges; +----+-------------+-----------+--------------+-------+ | id | schema_name | principal | permission | grant | +----+-------------+-----------+--------------+-------+ | 1 | default | user/yli2 | ALTER | 1 | | 2 | default | user/yli2 | CREATE_TABLE | 1 | | 3 | default | user/yli2 | DESCRIBE | 1 | | 4 | default | user/yli2 | DROP | 1 | +----+-------------+-----------+--------------+-------+ 4 rows in set Time: 0.015s \r:iamdb> quit

On Mon, Oct 26, 2020 at 1:00 PM Yingyou yingyou@gmail.com wrote:

~/.lakecli ❯ vim app.log 2020-10-22 23:33:25,840 (38148/MainThread) lakecli.sqlexecute INFO - Connect to /Users/yli/.lakecli/iamdb 2020-10-22 23:33:25,841 (38148/completion_refresh) lakecli.sqlexecute INFO

  • Connect to /Users/yli/.lakecli/iamdb 2020-10-22 23:37:16,679 (38148/MainThread) lakecli.main ERROR - sql: "SELECT FROM table_priviledges WHERE principal like 'yli%';", error: OperationalError('no such table: table_priviledges') 2020-10-22 23:37:16,679 (38148/MainThread) lakecli.main ERROR - sql: "SELECT FROM table_priviledges WHERE principal like 'yli%';", error: OperationalError('no such table: table_priviledges') 2020-10-22 23:37:16,683 (38148/MainThread) lakecli.main ERROR - traceback: 'Traceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 74, in run\n for result in special.execute(cur, sql):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/packages/special/main.py", line 66, in execute\n raise CommandNotFound\nlakecli.packages.special.main.CommandNotFound\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/main.py", line 331, in one_iteration\n for title, rows, headers, status in res:\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 77, in run\n cur.execute(sql)\nsqlite3.OperationalError: no such table: table_priviledges\n' 2020-10-22 23:37:16,683 (38148/MainThread) lakecli.main ERROR - traceback: 'Traceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 74, in run\n for result in special.execute(cur, sql):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/packages/special/main.py", line 66, in execute\n raise CommandNotFound\nlakecli.packages.special.main.CommandNotFound\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/main.py", line 331, in one_iteration\n for title, rows, headers, status in res:\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 77, in run\n cur.execute(sql)\nsqlite3.OperationalError: no such table: table_priviledges\n' 2020-10-22 23:37:46,526 (38148/MainThread) lakecli.main ERROR - sql: "SELECT FROM table_privileges WHERE principal like 'yli%';", error: OperationalError('no such table: table_privileges') 2020-10-22 23:37:46,526 (38148/MainThread) lakecli.main ERROR - sql: "SELECT FROM table_privileges WHERE principal like 'yli%';", error: OperationalError('no such table: table_privileges') 2020-10-22 23:37:46,526 (38148/MainThread) lakecli.main ERROR - traceback: 'Traceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 74, in run\n for result in special.execute(cur, sql):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/packages/special/main.py", line 66, in execute\n raise CommandNotFound\nlakecli.packages.special.main.CommandNotFound\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/main.py", line 331, in one_iteration\n for title, rows, headers, status in res:\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 77, in run\n cur.execute(sql)\nsqlite3.OperationalError: no such table: table_privileges\n' 2020-10-22 23:37:46,526 (38148/MainThread) lakecli.main ERROR - traceback: 'Traceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 74, in run\n for result in special.execute(cur, sql):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/packages/special/main.py", line 66, in execute\n raise CommandNotFound\nlakecli.packages.special.main.CommandNotFound\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/main.py", line 331, in one_iteration\n for title, rows, headers, status in res:\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 77, in run\n cur.execute(sql)\nsqlite3.OperationalError: no such table: table_privileges\n' 2020-10-22 23:40:17,880 (38148/MainThread) lakecli.main ERROR - sql: 'SELECT FROM maxdb;', error: OperationalError('no such table: maxdb') 2020-10-22 23:40:17,880 (38148/MainThread) lakecli.main ERROR - sql: 'SELECT FROM maxdb;', error: OperationalError('no such table: maxdb') 2020-10-22 23:40:17,881 (38148/MainThread) lakecli.main ERROR - traceback: 'Traceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 74, in run\n for result in special.execute(cur, sql):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/packages/special/main.py", line 66, in execute\n raise CommandNotFound\nlakecli.packages.special.main.CommandNotFound\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/main.py", line 331, in one_iteration\n for title, rows, headers, status in res:\n File "/Users/yli/opt/anaconda3/lib/python3.8/site-packages/lakecli/sqlexecute.py", line 77, in run\n cur.execute(sql)\nsqlite3.OperationalError: no such table: maxdb\n' @

                                                         @
    
                                         @
    
                         @

    :$ 2020-10-26 12:03:45,599 (60179/MainThread) lakecli.iam.scanner INFO - 6 columns found in tickit.tickit_public_event 2020-10-26 12:03:45,599 (60179/MainThread) lakecli.iam.scanner INFO - 8 columns found in tickit.tickit_public_listing 2020-10-26 12:03:45,599 (60179/MainThread) lakecli.iam.scanner INFO - 10 columns found in tickit.tickit_public_sales 2020-10-26 12:03:45,599 (60179/MainThread) lakecli.iam.scanner INFO - 18 columns found in tickit.tickit_public_users 2020-10-26 12:03:45,599 (60179/MainThread) lakecli.iam.scanner INFO - 5 columns found in tickit.tickit_public_venue 2020-10-26 12:03:45,619 (60179/MainThread) lakecli.sqlexecute INFO - Connect to /Users/yli/.lakecli/iamdb 2020-10-26 12:03:45,620 (60179/completion_refresh) lakecli.sqlexecute INFO

  • Connect to /Users/yli/.lakecli/iamdb 2020-10-26 12:59:11,042 (63536/MainThread) lakecli.main INFO - Scanning Lake Formation Permissions 2020-10-26 12:59:11,042 (63536/MainThread) lakecli.main INFO - Scanning Lake Formation Permissions 2020-10-26 12:59:11,042 (63536/MainThread) lakecli.iam.scanner INFO - Remove old sqlite database at /Users/yli/.lakecli/iamdb 2020-10-26 12:59:11,716 (63536/MainThread) lakecli.iam.scanner INFO - 0 table permissions found. 2020-10-26 12:59:11,716 (63536/MainThread) lakecli.iam.scanner INFO - 1 database permissions found. 2020-10-26 12:59:12,221 (63536/MainThread) lakecli.iam.scanner INFO - 8 databases found 2020-10-26 12:59:12,361 (63536/MainThread) lakecli.iam.scanner INFO - 0 tables found in default 2020-10-26 12:59:12,518 (63536/MainThread) lakecli.iam.scanner INFO - 1 tables found in flightdb 2020-10-26 12:59:12,518 (63536/MainThread) lakecli.iam.scanner INFO - 64 columns found in flightdb.flightscsv 2020-10-26 12:59:12,675 (63536/MainThread) lakecli.iam.scanner INFO - 1 tables found in maxdb 2020-10-26 12:59:12,675 (63536/MainThread) lakecli.iam.scanner INFO - 8 columns found in maxdb.customers 2020-10-26 12:59:12,818 (63536/MainThread) lakecli.iam.scanner INFO - 3 tables found in myfirstgluedb 2020-10-26 12:59:12,818 (63536/MainThread) lakecli.iam.scanner INFO - 5 columns found in myfirstgluedb.active 2020-10-26 12:59:12,819 (63536/MainThread) lakecli.iam.scanner INFO - 5 columns found in myfirstgluedb.citydata 2020-10-26 12:59:12,819 (63536/MainThread) lakecli.iam.scanner INFO - 5 columns found in myfirstgluedb.old 2020-10-26 12:59:12,956 (63536/MainThread) lakecli.iam.scanner INFO - 1 tables found in spectrumdb 2020-10-26 12:59:12,956 (63536/MainThread) lakecli.iam.scanner INFO - 10 columns found in spectrumdb.sales 2020-10-26 12:59:13,132 (63536/MainThread) lakecli.iam.scanner INFO - 1 tables found in testbucket 2020-10-26 12:59:13,133 (63536/MainThread) lakecli.iam.scanner INFO - 5 columns found in testbucket.maxwell_citydata_test 2020-10-26 12:59:13,264 (63536/MainThread) lakecli.iam.scanner INFO - 0 tables found in testdb 2020-10-26 12:59:13,422 (63536/MainThread) lakecli.iam.scanner INFO - 7 tables found in tickit 2020-10-26 12:59:13,422 (63536/MainThread) lakecli.iam.scanner INFO - 4 columns found in tickit.tickit_public_category 2020-10-26 12:59:13,422 (63536/MainThread) lakecli.iam.scanner INFO - 8 columns found in tickit.tickit_public_date 2020-10-26 12:59:13,423 (63536/MainThread) lakecli.iam.scanner INFO - 6 columns found in tickit.tickit_public_event 2020-10-26 12:59:13,423 (63536/MainThread) lakecli.iam.scanner INFO - 8 columns found in tickit.tickit_public_listing 2020-10-26 12:59:13,423 (63536/MainThread) lakecli.iam.scanner INFO - 10 columns found in tickit.tickit_public_sales 2020-10-26 12:59:13,423 (63536/MainThread) lakecli.iam.scanner INFO - 18 columns found in tickit.tickit_public_users 2020-10-26 12:59:13,423 (63536/MainThread) lakecli.iam.scanner INFO - 5 columns found in tickit.tickit_public_venue 2020-10-26 12:59:13,452 (63536/MainThread) lakecli.sqlexecute INFO - Connect to /Users/yli/.lakecli/iamdb 2020-10-26 12:59:13,454 (63536/completion_refresh) lakecli.sqlexecute INFO
  • Connect to /Users/yli/.lakecli/iamdb

On Mon, Oct 26, 2020 at 9:31 AM Rajat Venkatesh notifications@github.com wrote:

Can you paste the logs for the run with --scan ?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/tokern/lakecli/issues/17#issuecomment-716585282, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHTW7AOB4VWWAEY7MGSW64LSMWB4VANCNFSM4S57TN3A .

vrajat commented 3 years ago

Great! Looks like progress. It is partially working ? Did you have to exit and restart the application ?

As per logs, it did not find any table permissions but found database level permissions.

2020-10-26 12:59:11,716 (63536/MainThread) lakecli.iam.scanner INFO - 0
table permissions found.
2020-10-26 12:59:11,716 (63536/MainThread) lakecli.iam.scanner INFO - 1
database permissions found.

Is this unexpected? Can you please check and confirm if any table level permissions are set in LakeFormation UI?

Reference: https://docs.aws.amazon.com/lake-formation/latest/dg/granting-table-permissions.html

superdba111 commented 3 years ago

it doesn't work well, and as you see, it cannot detect other databases except default db. please advise how to get other dbs in? and I don't see any tables right now

On Mon, Oct 26, 2020 at 11:25 PM Rajat Venkatesh notifications@github.com wrote:

Great! Looks like it is working ? Did you have to exit and restart ?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

superdba111 commented 3 years ago

log file showed that dbs and tables were found but you cannot see them when you query the two tables

On Mon, Oct 26, 2020 at 11:47 PM Yingyou yingyou@gmail.com wrote:

it doesn't work well, and as you see, it cannot detect other databases except default db. please advise how to get other dbs in? and I don't see any tables right now

On Mon, Oct 26, 2020 at 11:25 PM Rajat Venkatesh notifications@github.com wrote:

Great! Looks like it is working ? Did you have to exit and restart ?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

vrajat commented 3 years ago

table_privileges and database_privileges only store the permissions. table_privileges is empty because no table permissions are set. If you set a permission using grant table ... or revoke table ..., then you will see entries.

Looks like you want to also list databases and tables in glue catalog using SQL? Like 'select * from tables?' ? This is a valid feature request and I can look into it. Can I send you an email to better understand your use case? When I take up a new feature request I want to make sure, I understand the requirements correctly.