Incompatible SQLAlchemy versions

[ketzu]

I tried to give powa-web a try today, unfortunately it would not run with the specified sqlalchemy versions (>0.9.8).

Closest bounds I could find are:

sqlalchemy>=1.2,<2.0

[rjuju]

Hi,

I'm not quite sure of what the problem exactly is. Is that because by default pip3 (or similar) resolves the >0.9.8 constraint to some sqlalchemy version that's not compatible with powa-web? (I'm assuming any version >= 2.0).

Do you know what exactly the problems are? For the record a full UI rewrite is on the way and we got rid of sqlalchemy, but in the mean time I would be happy to fix compatibility with newer version if that's not too much trouble.

[ketzu]

Hey,

yes, pip installed the latest version of sqlalchemy (which is 2.0 since january, 2.0.23 released november 2nd) but 1.4 is still maintained (1.4.50 release this october 23rd). So pip install -r requirements.txt will install 2.0.

I only added the lower bound, because instead of looking up the most recent release, I just tried 1.0 which threw another error :)

Here is the error I get with simply

git clone https://github.com/powa-team/powa-web.git .
pip install -r ./requirements.txt
./powa-web

Error:

Traceback (most recent call last):
  File "/home/david/powa/web2/powa-web/./powa-web", line 4, in <module>
    from powa import make_app
  File "/home/david/powa/web2/powa-web/powa/__init__.py", line 21, in <module>
    from powa.overview import Overview
  File "/home/david/powa/web2/powa-web/powa/overview.py", line 12, in <module>
    from powa.server import ServerOverview
  File "/home/david/powa/web2/powa-web/powa/server.py", line 15, in <module>
    from powa.config import ConfigChangesGlobal
  File "/home/david/powa/web2/powa-web/powa/config.py", line 9, in <module>
    from powa.sql.views import get_config_changes
  File "/home/david/powa/web2/powa-web/powa/sql/views.py", line 5, in <module>
    from powa.sql.utils import diff
  File "/home/david/powa/web2/powa-web/powa/sql/utils.py", line 7, in <module>
    block_size = select([cast(func.current_setting('block_size'), Numeric)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/david/powa/web2/venv/lib/python3.11/site-packages/sqlalchemy/sql/_selectable_constructors.py", line 489, in select
    return Select(*entities)
           ^^^^^^^^^^^^^^^^^
  File "/home/david/powa/web2/venv/lib/python3.11/site-packages/sqlalchemy/sql/selectable.py", line 5136, in __init__
    self._raw_columns = [
                        ^
  File "/home/david/powa/web2/venv/lib/python3.11/site-packages/sqlalchemy/sql/selectable.py", line 5137, in <listcomp>
    coercions.expect(
  File "/home/david/powa/web2/venv/lib/python3.11/site-packages/sqlalchemy/sql/coercions.py", line 413, in expect
    resolved = impl._literal_coercion(
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/david/powa/web2/venv/lib/python3.11/site-packages/sqlalchemy/sql/coercions.py", line 651, in _literal_coercion
    self._raise_for_expected(element, argname)
  File "/home/david/powa/web2/venv/lib/python3.11/site-packages/sqlalchemy/sql/coercions.py", line 1139, in _raise_for_expected
    return super()._raise_for_expected(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/david/powa/web2/venv/lib/python3.11/site-packages/sqlalchemy/sql/coercions.py", line 710, in _raise_for_expected
    super()._raise_for_expected(
  File "/home/david/powa/web2/venv/lib/python3.11/site-packages/sqlalchemy/sql/coercions.py", line 535, in _raise_for_expected
    raise exc.ArgumentError(msg, code=code) from err
sqlalchemy.exc.ArgumentError: Column expression, FROM clause, or other columns clause element expected, got [<sqlalchemy.sql.elements.Label at 0x7f826f186810; block_size>]. Did you mean to say select(<sqlalchemy.sql.elements.Label at 0x7f826f186810; block_size>)?

I used a python3.11 virtual environment on ubuntu 22 I believe. Everything worked with sqlalchemy==1.4.50.

SqlAlchemy reports larger API breaks in their migration guide https://docs.sqlalchemy.org/en/20/changelog/migration_20.html

The specific error seems to be related to this change: https://docs.sqlalchemy.org/en/20/changelog/migration_14.html#change-5284 select and case now accepting positional expressions instead of lists.

Considering there is an sqlalchemy free rewrite ont he way, I would argue just fixing <2.0 or <=1.4 for the timing being will be easier on you :) but I did not look at the code of powa-web at all and I'm just some rando, so do as you please :D

Thanks, good luck and have fun :)

[rjuju]

Thanks for digging into the changelog! Well, if I understand correctly they chose a hard break with no possibility of having code that works with pre-2.0 and post-2.0 version so you have to rewrite everything to the new style and force everyone to switch to 1.4 which is the only version that will support both, for a limited time before switching to 2.0.

I never liked sqlalchemy and I'm even more convinced now that I made the right decision to fully get rid of it for the future v5.

I agree that we should just add some <=1.4 constraint for now and hope they don't deprecate 1.4 in the new few weeks, until we can release powa 5.

[rjuju]

I just pushed that change in the requirements file, thanks a lot for the report!

[kjumybit]

Hi,

I just installed the pip package powa-web in a new Python 3.10 virtual environment and got the same issue. My local package versions are:

...
greenlet==3.0.3
powa-web==4.2.0
psycopg2==2.9.9
SQLAlchemy==2.0.30
tornado==6.4
typing_extensions==4.12.0

  File "/home/hagen/apps/powa_web/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected
    raise exc.ArgumentError(msg, code=code) from err
sqlalchemy.exc.ArgumentError: Column expression, FROM clause, or other columns clause element expected, got [<sqlalchemy.sql.elements.Label at 0x7f9471ef7700; block_size>]. Did you mean to say select(<sqlalchemy.sql.elements.Label at 0x7f9471ef7700; block_size>)?

[rjuju]

Hi,

the requirements file should force sqlachmey <= 1.4 to be installed, wasn't it the case? Is version 1.4.x EOL now?

[kjumybit]

Hi,

I first installed the latest package version on pypi is 4.2.0. I found that content in requirements.txt:

psycopg2
sqlalchemy>0.9.8
tornado>=2.0

After downgrad the sqlalchemy version I got another version conflict with a fresh installed PoWasolution (local setup).

Unable to connect: powa-archivist version 4.1.X does not match powa-web version 4.2.X

So I have to downgrade powa-web to 4.1.4 too. But that is another topic :-)

[rjuju]

I first installed the latest package version on pypi is 4.2.0. I found that content in requirements.txt:

psycopg2
sqlalchemy>0.9.8
tornado>=2.0

ah indeed the requirements file was modified after that version, but since there wasn't any other changes I didn't do any release. I will do one then to make sure no one else hits that problem.

After downgrad the sqlalchemy version I got another version conflict with a fresh installed PoWasolution (local setup).

Unable to connect: powa-archivist version 4.1.X does not match powa-web version 4.2.X

So I have to downgrade powa-web to 4.1.4 too. But that is another topic :-)

that means that you installed powa-archivist 4.1.x. the latest version is 4.2.2. I'm not sure how you installed it but it looks like you didn't install the latest version.