wizard: TypeError: string indices must be integers, not str

[erichanson]

I'm running powa Version 3.2.0, on PostgreSQL 11 using the CentOS-packaged distribution. Most everything seems to work except for the optimizer. I click the "Optimize this database" button, and get into this:

[E 200218 22:47:33 web:1495] Uncaught exception GET /metrics/database/restapi/wizard/?from=2020-02-18+15%3A41%3A52-0600&to=2020-02-18+16%3A41%3A52-0600 (10.0.31.59)
    HTTPServerRequest(protocol='http', host='localhost:8888', method='GET', uri='/metrics/database/restapi/wizard/?from=2020-02-18+15%3A41%3A52-0600&to=2020-02-18+16%3A41%3A52-0600', version='HTTP/1.1', remote_ip='10.0.31.59', headers={'Accept-Language': 'en-US,en;q=0.9', 'Accept-Encoding': 'gzip, deflate, br', 'Sec-Fetch-Site': 'same-origin', 'Host': 'localhost:8888', 'Accept': '*/*', 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36', 'Connection': 'keep-alive', 'X-Requested-With': 'XMLHttpRequest', 'Sec-Fetch-Mode': 'cors', 'Referer': 'http://localhost:8888/database/restapi/overview/', 'Cookie': 'username="2|1:0|10:1582046468|8:username|8:cG93YQ==|4587b87aaab09e1423fc19d378bc82a3c97ea3fc1169aae8a8249d1256e1d48c"; password="2|1:0|10:1582046468|8:password|8:aGltb20=|7ccdf24c1f9394fc9fbdcc28516faab3ed140147ed9d1cb5254734fe672bcfc5"; server="2|1:0|10:1582046468|6:server|8:bWFpbg==|c2403441911ba3534203478f8e6ef1c23ce6c6c8a760c125b4a521c0c4e14dab"; _flashes="2|1:0|10:1582065715|8:_flashes|4:Ti4=|3832eed9b3cb0888c29d336cef5f95688b8c861ce3f7e2fa069ea94c49406d2b"'})
    Traceback (most recent call last):
      File "/usr/lib64/python2.7/site-packages/tornado/web.py", line 1412, in _execute
        result = method(*self.path_args, **self.path_kwargs)
      File "/usr/lib/python2.7/site-packages/powa/dashboards.py", line 79, in get
        data = self.post_process(data, **url_params)
      File "/usr/lib/python2.7/site-packages/powa/wizard.py", line 110, in post_process
        data["data"] = resolve_quals(conn, data["data"])
      File "/usr/lib/python2.7/site-packages/powa/sql/__init__.py", line 216, in resolve_quals
        operator_to_look.add(v['opno'])
    TypeError: string indices must be integers, not str
[E 200218 22:47:33 web:1907] 500 GET /metrics/database/restapi/wizard/?from=2020-02-18+15%3A41%3A52-0600&to=2020-02-18+16%3A41%3A52-0600 (10.0.31.59) 32.13ms

Thanks!

[rjuju]

Hello.

I've been looking at this bug, but I'm not sure how this is possible. What versions of the various dependencies (psycopg2, sqlalchemy, tornado) are you using? Is this error always happening or only for specific queries?

[erichanson]

The issue happens every time I click the Optimize this Database button.

Here are version numbers:

NOCDBA(postgres):ip-10-0-12-38 /data-10/tools/powa-archivist $ pip freeze|grep "tornado\|psycopg2\|sqlal"
psycopg2==2.5.1
tornado==4.2.1

^^ note that sqlalchemy does not show up. Ok let's try another way:

NOCDBA(postgres):ip-10-0-12-38 /data-10/tools/powa-archivist $ python
Python 2.7.16 (default, Nov  9 2019, 00:53:10)
[GCC 7.3.1 20180712 (Red Hat 7.3.1-6)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import psycopg2
>>> print psycopg2.__version__
2.5.1 (dt dec pq3 ext)
>>> import sqlalchemy
>>> print sqlalchemy.__version__
0.9.8
>>> import tornado
>>> print tornado.__version__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute '__version__'
>>>

[rjuju]

I guess you'd need a grep -i, since it's reported as SQLAlchemy.

If you're using a virtualenv and can upgrade/downgrade easily, could you try to upgrade psycopg2, say to 2.7.7 (that's the version currently installed in my own virtualenv)? If the problem is still there, maybe tornado to 5.1.1 and SQLAlchemy to 1.2.17, but those seem quite unrelated.

[erichanson]

I actually upgraded SQLAlchemy first and that fixed this issue:

pip install --upgrade SQLAlchemy

and now I have:

NOCDBA(postgres):ip-10-0-12-38 /data-10/tools/powa-web $ pip freeze|grep -i "sqlal\|psycopg\|tornado";
psycopg2==2.5.1
SQLAlchemy==1.3.13
tornado==4.2.1

[rjuju]

Oh, that's surprising. Thanks a lot! I'll update the documentation that SQLAlchemy > 0.9.8 is required. I'll try to see which is the version actually supported.