Circular import - Githubissues

From what I understood to use sqlalchemy_firebird we need to import both like this:

import sqlalchemy
import sqlalchemy_firebird

The problem comes when the script is bundled with PyInstaller:

[main ≡ +0 ~3 -0 !]› .\dist\test.exe
Traceback (most recent call last):
  File "test_folder\test.py", line 4, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module
  File "sqlalchemy_firebird\__init__.py", line 23, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module
  File "sqlalchemy_firebird\provision.py", line 1, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module
  File "sqlalchemy\testing\__init__.py", line 12, in <module>
ImportError: cannot import name 'config' from partially initialized module 'sqlalchemy.testing' (most likely due to a circular import) (C:\Users\lucas\AppData\Local\Temp\_MEI77922\sqlalchemy\testing\__init__.pyc)
[23588] Failed to execute script 'test' due to unhandled exception!

The bundling operation in itself works, but the problem arises when the executable is launched.

Am i importing/using it wrong?

The documentation does not include an example code.

The problem is in the sqlalchemy_firebird/__init__.py file:

# firebird/__init__.py
# Copyright (C) 2005-2020 the SQLAlchemy authors and contributors
# <see AUTHORS file>
#
# This module is part of SQLAlchemy and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php

from sqlalchemy.dialects import registry as _registry

from .base import BIGINT
from .base import BLOB
from .base import CHAR
from .base import DATE
from .base import FLOAT
from .base import NUMERIC
from .base import SMALLINT
from .base import TEXT
from .base import TIME
from .base import TIMESTAMP
from .base import VARCHAR
from . import base  # noqa
from . import fdb  # noqa
-from . import provision  # noqa
+# from . import provision  # noqa

# Not supporting kinterbase
# from . import kinterbasdb  # noqa

__version__ = "0.8.0"

base.dialect = dialect = fdb.dialect

_registry.register("firebird", "sqlalchemy_firebird.fdb", "FBDialect_fdb")

__all__ = (
    "SMALLINT",
    "BIGINT",
    "FLOAT",
    "FLOAT",
    "DATE",
    "TIME",
    "TEXT",
    "NUMERIC",
    "FLOAT",
    "TIMESTAMP",
    "VARCHAR",
    "CHAR",
    "BLOB",
    "dialect",
)

If i comment out the line from . import provision it works like a charm.

You do not need to "import sqlalchemy-firebird" when you'd like to use sqlalchemy to connect to a Firebird database, it is automatically used by sqlalchemy when you specify "firebird..." at the beginning of your database connection string. For example:

from sqlalchemy import create_engine

my_database = "firebird://username:password@localhost/c:/projects/databases/my_project.fdb"
engine = create_engine(my_database)

The "engine" above is connected to your Firebird database. All sqlalchemy operations act upon it using _sqlalchemyfirebird as an intermediary - you never actually use sqlalchemy-firebird directly.

That's the advantage of using sqlalchemy. The specifics of the underlying database, whatever it may be, are taken care of and you just manage your application. Your application could switch to PostgreSQL, Oracle, SQLServer, etc., and would continue to work without any code changes on your part - just change the database url to the desired server.

Thank you for pointing out the need for example code. Do have any ideas about an example that would have been useful to you?

Thank you for your quick response @pauldex.

You do not need to "import sqlalchemy-firebird" when you'd like to use sqlalchemy to connect to a Firebird database, it is automatically used by sqlalchemy when you specify "firebird..." at the beginning of your database connection string. For example:

I didn't know that the import statement wasn't necessary, and i don't even understand how can it be working since sqlalchemy-firebird registers against sqlalchemy (but not being imported how can it?)... I would like an explanation to this, just out of curiosity because it isn't very intuitive to me 😄

I tested what you wrote and to my disbelief it's true (I added print ("TEST") to sqlalchemy_firebird/__init__.py and the output is actually TEST)...

from pathlib import Path
from sqlalchemy import create_engine

database_path = Path("~/Desktop/Archivio.eft").expanduser()

engine = create_engine(f"firebird://SYSDBA:masterkey@localhost/{database_path}")

How in the hell is this possible if i never imported it???? NOW I'm confused! 😆

⚠ For PyInstaller users

With this knowledge in mind i tried rewriting the code...

I wrote the following code in test.py:

from pathlib import Path
from sqlalchemy import create_engine, text

database_path = Path("~/Desktop/Archivio.eft").expanduser()
engine = create_engine(f"firebird://SYSDBA:masterkey@localhost/{database_path}")

print(engine.connect().execute(text('SELECT rdb$relation_name FROM "RDB$RELATIONS" WHERE RDB$RELATION_NAME NOT LIKE \'RDB$%\' AND RDB$RELATION_NAME NOT LIKE \'MON$%\'')).fetchall())

As you can see I removed the import sqlalchemy_firebird line as you told me...

To bundle everyihing I used the following command:

pyinstaller --clean --onedir -y --log-level=WARN --hidden-import sqlalchemy_firebird --collect-all sqlalchemy_firebird --recursive-copy-metadata sqlalchemy_firebird test.py

Notice the --hidden-import, --collect-all and --recursive-copy-metadata all set to the name of this package!

💥 The problem

This however still gives me the Circular Import error thing i mentioned in my first post:

[main ↓1 ↑1 +1 ~5 -1 !]› .\dist\test\test.exe
Traceback (most recent call last):
  File "test.py", line 7, in <module>
    engine = create_engine(f"firebird://SYSDBA:masterkey@localhost/{database_path}")
  File "<string>", line 2, in create_engine
  File "sqlalchemy\util\deprecations.py", line 283, in warned
  File "sqlalchemy\engine\create.py", line 552, in create_engine
  File "sqlalchemy\engine\url.py", line 754, in _get_entrypoint
  File "sqlalchemy\util\langhelpers.py", line 366, in load
  File "importlib\metadata\__init__.py", line 171, in load
  File "importlib\__init__.py", line 126, in import_module
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module
  File "sqlalchemy_firebird\__init__.py", line 23, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module
  File "sqlalchemy_firebird\provision.py", line 1, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module
  File "sqlalchemy\testing\__init__.py", line 12, in <module>
ImportError: cannot import name 'config' from partially initialized module 'sqlalchemy.testing' (most likely due to a circular import) (D:\Progetti\test_folder\dist\test\sqlalchemy\testing\__init__.pyc)
[29776] Failed to execute script 'test' due to unhandled exception!

Do you have any idea on how can I prevent this?

If i comment out the line from . import provision it works like a charm.

Keep in mind that again, if i use this workaround, the EXE starts working perfectly...

When you install sqlalchemy_firebird into your environment, it registers itself with sqlalchemy as being the handler for connection uri strings that start with “firebird”.

Your application uses sqlalchemy and a Firebird connection uri string. SQLAlchemy then transparently uses sqlalchemy-firebird to handle all database calls.

I’m not familiar with PyInstaller, what happens if you just build your app with pyinstaller test.py?

Paul

From: Luca Salvarani @.> Sent: Tuesday, May 23, 2023 3:21 PM To: pauldex/sqlalchemy-firebird @.> Cc: Paul Graves-DesLauriers @.>; Mention @.> Subject: Re: [pauldex/sqlalchemy-firebird] Circular import (Issue #49)

Thank you for your quick response @pauldex https://github.com/pauldex .

You do not need to "import sqlalchemy-firebird" when you'd like to use sqlalchemy to connect to a Firebird database, it is automatically used by sqlalchemy when you specify "firebird..." at the beginning of your database connection string. For example:

I didn't know that the import statement wasn't necessary, and i don't even understand how can it be working since sqlalchemy-firebird registers against sqlalchemy (but not being imported how can it?)... I would like an explanation to this, just out of curiosity because it isn't very intuitive to me 😄

I tested what you wrote and to my disbelief it's true (I added print ("TEST") to sqlalchemy_firebird/init.py and the output is actually TEST)...

from pathlib import Path from sqlalchemy import create_engine

database_path = Path("~/Desktop/Archivio.eft").expanduser()

engine = @.***/{database_path}")

How in the hell is this possible if i never imported it???? NOW I'm confused! 😆

⚠ For PyInstaller users

With this knowledge in mind i tried rewriting the code...

I wrote the following code in test.py:

from pathlib import Path from sqlalchemy import create_engine, text

database_path = Path("~/Desktop/Archivio.eft").expanduser() engine = @.***/{database_path}")

print(engine.connect().execute(text('SELECT rdb$relation_name FROM "RDB$RELATIONS" WHERE RDB$RELATION_NAME NOT LIKE \'RDB$%\' AND RDB$RELATION_NAME NOT LIKE \'MON$%\'')).fetchall())

As you can see I removed the import sqlalchemy_firebird line as you told me...

To bundle everyihing I used the following command:

pyinstaller --clean --onedir -y --log-level=INFO --hidden-import sqlalchemy_firebird --collect-all sqlalchemy_firebird --recursive-copy-metadata sqlalchemy_firebird test.py

This however still gives me the Circular Import error thing i mentioned in my first post:

[main ↓1 ↑1 +1 ~5 -1 !]› .\dist\test\test.exe Traceback (most recent call last): File "test.py", line 7, in engine = @.***/{database_path}") File "", line 2, in create_engine File "sqlalchemy\util\deprecations.py", line 283, in warned File "sqlalchemy\engine\create.py", line 552, in create_engine File "sqlalchemy\engine\url.py", line 754, in _get_entrypoint File "sqlalchemy\util\langhelpers.py", line 366, in load File "importlib\metadata__init.py", line 171, in load File "importlib__init.py", line 126, in import_module File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 992, in _find_and_load_unlocked File "", line 241, in _call_with_frames_removed File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module File "sqlalchemy_firebird__init__.py", line 23, in File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module File "sqlalchemy_firebird\provision.py", line 1, in File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module File "sqlalchemy\testing\init__.py", line 12, in ImportError: cannot import name 'config' from partially initialized module 'sqlalchemy.testing' (most likely due to a circular import) (D:\Progetti\test_folder\dist\test\sqlalchemy\testing\init__.pyc) [29776] Failed to execute script 'test' due to unhandled exception!

How can i prevent this?

— Reply to this email directly, view it on GitHub https://github.com/pauldex/sqlalchemy-firebird/issues/49#issuecomment-1560203286 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ABX7Z62YMOTJQ44CU7ABYC3XHUZ4RANCNFSM6AAAAAAYIDGTXU . You are receiving this because you were mentioned. https://github.com/notifications/beacon/ABX7Z6YUDLEQP3HEWRYY32TXHUZ4RA5CNFSM6AAAAAAYIDGTXWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTS473IBM.gif Message ID: @. @.> >

Thank you for your response @pauldex .

If I build the executable without explicitly specifying sqlalchemy_firebird as a hidden dependency, firebird will complain (at runtime) about no handler being defined for firebird connections (since sqlalchemy_firebird has not been included in the bundle).

If instead I include the package in the bundle I get the Circular Import exception when the executable is run.

The problem is solved only if I comment out the from . import provision line as shown in my second comment.

I hope I made the issue more clear, feel free to ask for more info is that is not true 😁

🎉 UPDATE 🎉

Using the code in the master branch (and not the code pushed on PyPi) everything works!!

import sys

from pathlib import Path
from sqlalchemy import create_engine, text

database_path = Path("~/Desktop/Archivio.eft").expanduser()
database_path = Path("C:\\Users\\lucas\\Documents\\Danea Easyfatt\\TestArchivio.eft").resolve()
engine = create_engine(f"firebird+fdb://SYSDBA:masterkey@localhost/{database_path}")

print(engine.connect().execute(text('SELECT rdb$relation_name FROM "RDB$RELATIONS" WHERE RDB$RELATION_NAME NOT LIKE \'RDB$%\' AND RDB$RELATION_NAME NOT LIKE \'MON$%\'')).fetchall())
sys.exit(0)

When will you push the new version to PyPi?

Great – I’m glad it works now. Version 2 was released this weekend.

Paul

From: Luca Salvarani @.> Sent: Tuesday, June 6, 2023 12:28 AM To: pauldex/sqlalchemy-firebird @.> Cc: Paul Graves-DesLauriers @.>; Mention @.> Subject: Re: [pauldex/sqlalchemy-firebird] Circular import (Issue #49)

UPDATE

Using the code in the master branch (and not the code pushed on PyPi) everything works!!

import sys

from pathlib import Path from sqlalchemy import create_engine, text

database_path = Path("~/Desktop/Archivio.eft").expanduser() database_path = Path("C:\Users\lucas\Documents\Danea Easyfatt\TestArchivio.eft").resolve() engine = @.***/{database_path}")

print(engine.connect().execute(text('SELECT rdb$relation_name FROM "RDB$RELATIONS" WHERE RDB$RELATION_NAME NOT LIKE \'RDB$%\' AND RDB$RELATION_NAME NOT LIKE \'MON$%\'')).fetchall()) sys.exit(0)

When will you push the version 2.0?

— Reply to this email directly, view it on GitHub https://github.com/pauldex/sqlalchemy-firebird/issues/49#issuecomment-1578074860 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ABX7Z66E6JUV47Z4LJEDQNTXJ3LXVANCNFSM6AAAAAAYIDGTXU . You are receiving this because you were mentioned. https://github.com/notifications/beacon/ABX7Z637YRSH4TUZ33SPHPTXJ3LXVA5CNFSM6AAAAAAYIDGTXWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTS6B6BOY.gif Message ID: @. @.> >

pauldex / sqlalchemy-firebird

Circular import #49

⚠ For PyInstaller users

💥 The problem

🎉 UPDATE 🎉