Using cache.type='ext:database' doesn't work in V 1.12.0

[claudiofr]

This simple test case works fine in V 1.11.0 bug but fails in V 1.12.0

import os from beaker.ext.database import DatabaseNamespaceManager from beaker.cache import CacheManager from beaker.util import parse_cache_config_options

def setup(): cache_opts = { "cache.type": "ext:database", "cache.lock_dir": "/tmp", "cache.data_dir": "/tmp", "cache.url": 'sqlite:////home/claudiof/gitrepo/gal_train/database/universe.sqlite?isolation_level=IMMEDIATE', "cache.table_name": "bbcache", "cache.schema_name": None } cache_mgr = CacheManager(**parse_cache_config_options(cache_opts)) cache = cache_mgr.get_cache("mulled_resolution")

setup() print("success")

Eror output in V1.12.0 is:

{ "name": "AttributeError", "message": "'NoneType' object has no attribute 'engine_from_config'", "stack": "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31mKeyError\u001b[0m Traceback (most recent call last)\nFile \u001b[0;32m~/gitrepo/gal_dev/.venv/lib/python3.10/site-packages/beaker/cache.py:311\u001b[0m, in \u001b[0;36mCache._get_cache\u001b[0;34m(cls, namespace, kw)\u001b[0m\n\u001b[1;32m 310\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[0;32m--> 311\u001b[0m \u001b[39mreturn\u001b[39;00m cache_managers[key]\n\u001b[1;32m 312\u001b[0m \u001b[39mexcept\u001b[39;00m \u001b[39mKeyError\u001b[39;00m:\n\n\u001b[0;31mKeyError\u001b[0m: \"mulled_resolution{'type': 'ext:database', 'data_dir': '/tmp', 'expire': None, 'log_file': None, 'lock_dir': '/tmp', 'url': 'sqlite:////home/claudiof/gitrepo/gal_train/database/universe.sqlite?isolation_level=IMMEDIATE', 'table_name': 'bbcache', 'schema_name': None, 'enabled': True}\"\n\nDuring handling of the above exception, another exception occurred:\n\n\u001b[0;31mAttributeError\u001b[0m Traceback (most recent call last)\n\u001b[1;32m/home/claudiof/work/test1/beaker_bug.py\u001b[0m in \u001b[0;36mline 18\n\u001b[1;32m 15\u001b[0m cache_mgr \u001b[39m=\u001b[39m CacheManager(\u001b[39m\u001b[39m\u001b[39m\u001b[39mparse_cache_config_options(cache_opts))\n\u001b[1;32m 16\u001b[0m cache \u001b[39m=\u001b[39m cache_mgr\u001b[39m.\u001b[39mget_cache(\u001b[39m\"\u001b[39m\u001b[39mmulled_resolution\u001b[39m\u001b[39m\"\u001b[39m)\n\u001b[0;32m---> 18\u001b[0m setup()\n\u001b[1;32m 19\u001b[0m \u001b[39mprint\u001b[39m(\u001b[39m\"\u001b[39m\u001b[39msuccess\u001b[39m\u001b[39m\"\u001b[39m)\n\n\u001b[1;32m/home/claudiof/work/test1/beaker_bug.py\u001b[0m in \u001b[0;36mline 16\u001b[0m, in \u001b[0;36msetup\u001b[0;34m()\n\u001b[1;32m 7\u001b[0m cache_opts \u001b[39m=\u001b[39m {\n\u001b[1;32m 8\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mcache.type\u001b[39m\u001b[39m\"\u001b[39m: \u001b[39m\"\u001b[39m\u001b[39mext:database\u001b[39m\u001b[39m\"\u001b[39m,\n\u001b[1;32m 9\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mcache.lock_dir\u001b[39m\u001b[39m\"\u001b[39m: \u001b[39m\"\u001b[39m\u001b[39m/tmp\u001b[39m\u001b[39m\"\u001b[39m,\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 13\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mcache.schema_name\u001b[39m\u001b[39m\"\u001b[39m: \u001b[39mNone\u001b[39;00m\n\u001b[1;32m 14\u001b[0m }\n\u001b[1;32m 15\u001b[0m cache_mgr \u001b[39m=\u001b[39m CacheManager(\u001b[39m\u001b[39m\u001b[39m\u001b[39mparse_cache_config_options(cache_opts))\n\u001b[0;32m---> 16\u001b[0m cache \u001b[39m=\u001b[39m cache_mgr\u001b[39m.\u001b[39;49mget_cache(\u001b[39m\"\u001b[39;49m\u001b[39mmulled_resolution\u001b[39;49m\u001b[39m\"\u001b[39;49m)\n\nFile \u001b[0;32m~/gitrepo/gal_dev/.venv/lib/python3.10/site-packages/beaker/cache.py:395\u001b[0m, in \u001b[0;36mCacheManager.get_cache\u001b[0;34m(self, name, kwargs)\u001b[0m\n\u001b[1;32m 393\u001b[0m kw \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mkwargs\u001b[39m.\u001b[39mcopy()\n\u001b[1;32m 394\u001b[0m kw\u001b[39m.\u001b[39mupdate(kwargs)\n\u001b[0;32m--> 395\u001b[0m \u001b[39mreturn\u001b[39;00m Cache\u001b[39m.\u001b[39;49m_get_cache(name, kw)\n\nFile \u001b[0;32m~/gitrepo/gal_dev/.venv/lib/python3.10/site-packages/beaker/cache.py:313\u001b[0m, in \u001b[0;36mCache._get_cache\u001b[0;34m(cls, namespace, kw)\u001b[0m\n\u001b[1;32m 311\u001b[0m \u001b[39mreturn\u001b[39;00m cache_managers[key]\n\u001b[1;32m 312\u001b[0m \u001b[39mexcept\u001b[39;00m \u001b[39mKeyError\u001b[39;00m:\n\u001b[0;32m--> 313\u001b[0m cache_managers[key] \u001b[39m=\u001b[39m cache \u001b[39m=\u001b[39m \u001b[39mcls\u001b[39;49m(namespace, \u001b[39m\u001b[39;49m\u001b[39m\u001b[39;49mkw)\n\u001b[1;32m 314\u001b[0m \u001b[39mreturn\u001b[39;00m cache\n\nFile \u001b[0;32m~/gitrepo/gal_dev/.venv/lib/python3.10/site-packages/beaker/cache.py:302\u001b[0m, in \u001b[0;36mCache.init\u001b[0;34m(self, namespace, type, expiretime, starttime, expire, nsargs)\u001b[0m\n\u001b[1;32m 299\u001b[0m expire \u001b[39m=\u001b[39m \u001b[39mint\u001b[39m(expire)\n\u001b[1;32m 301\u001b[0m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mnamespace_name \u001b[39m=\u001b[39m namespace\n\u001b[0;32m--> 302\u001b[0m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mnamespace \u001b[39m=\u001b[39m \u001b[39mcls\u001b[39;49m(namespace, \u001b[39m\u001b[39;49m\u001b[39m\u001b[39;49mnsargs)\n\u001b[1;32m 303\u001b[0m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mexpiretime \u001b[39m=\u001b[39m expiretime \u001b[39mor\u001b[39;00m expire\n\u001b[1;32m 304\u001b[0m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mstarttime \u001b[39m=\u001b[39m starttime\n\nFile \u001b[0;32m~/gitrepo/gal_dev/.venv/lib/python3.10/site-packages/beaker/ext/database.py:76\u001b[0m, in \u001b[0;36mDatabaseNamespaceManager.init\u001b[0;34m(self, namespace, url, sa_opts, table_name, data_dir, lock_dir, schema_name, params)\u001b[0m\n\u001b[1;32m 73\u001b[0m cache_table\u001b[39m.\u001b[39mcreate(bind\u001b[39m=\u001b[39mengine, checkfirst\u001b[39m=\u001b[39m\u001b[39mTrue\u001b[39;00m)\n\u001b[1;32m 74\u001b[0m \u001b[39mreturn\u001b[39;00m cache_table\n\u001b[0;32m---> 76\u001b[0m engine \u001b[39m=\u001b[39m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m\u001b[39mclass\u001b[39;49m\u001b[39m.\u001b[39;49mbinds\u001b[39m.\u001b[39;49mget(url, \u001b[39mlambda\u001b[39;49;00m: sa\u001b[39m.\u001b[39;49mengine_from_config(sa_opts, \u001b[39m'\u001b[39;49m\u001b[39msa.\u001b[39;49m\u001b[39m'\u001b[39;49m))\n\u001b[1;32m 77\u001b[0m table \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m\u001b[39mclass\u001b[39m\u001b[39m.\u001b[39mtables\u001b[39m.\u001b[39mget(table_key, \u001b[39mlambda\u001b[39;00m: make_table(engine))\n\u001b[1;32m 79\u001b[0m SqlaNamespaceManager\u001b[39m.\u001b[39m\u001b[39minit\u001b[39m(\u001b[39mself\u001b[39m, namespace, engine, table,\n\u001b[1;32m 80\u001b[0m data_dir\u001b[39m=\u001b[39mdata_dir, lock_dir\u001b[39m=\u001b[39mlock_dir)\n\nFile \u001b[0;32m~/gitrepo/gal_dev/.venv/lib/python3.10/site-packages/beaker/util.py:171\u001b[0m, in \u001b[0;36mSyncDict.get\u001b[0;34m(self, key, createfunc, *args, *kwargs)\u001b[0m\n\u001b[1;32m 169\u001b[0m \u001b[39mreturn\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mdict[key]\n\u001b[1;32m 170\u001b[0m \u001b[39melse\u001b[39;00m:\n\u001b[0;32m--> 171\u001b[0m \u001b[39mreturn\u001b[39;00m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49msync_get(key, createfunc, \u001b[39m\u001b[39;49margs, \u001b[39m\u001b[39;49m\u001b[39m\u001b[39;49mkwargs)\n\u001b[1;32m 172\u001b[0m \u001b[39mexcept\u001b[39;00m \u001b[39mKeyError\u001b[39;00m:\n\u001b[1;32m 173\u001b[0m \u001b[39mreturn\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39msync_get(key, createfunc, \u001b[39m\u001b[39margs, \u001b[39m\u001b[39m\u001b[39m\u001b[39mkwargs)\n\nFile \u001b[0;32m~/gitrepo/gal_dev/.venv/lib/python3.10/site-packages/beaker/util.py:182\u001b[0m, in \u001b[0;36mSyncDict.sync_get\u001b[0;34m(self, key, createfunc, args, kwargs)\u001b[0m\n\u001b[1;32m 180\u001b[0m \u001b[39mreturn\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mdict[key]\n\u001b[1;32m 181\u001b[0m \u001b[39melse\u001b[39;00m:\n\u001b[0;32m--> 182\u001b[0m \u001b[39mreturn\u001b[39;00m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m_create(key, createfunc, \u001b[39m\u001b[39;49margs, \u001b[39m\u001b[39;49m\u001b[39m\u001b[39;49mkwargs)\n\u001b[1;32m 183\u001b[0m \u001b[39mexcept\u001b[39;00m \u001b[39mKeyError\u001b[39;00m:\n\u001b[1;32m 184\u001b[0m \u001b[39mreturn\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_create(key, createfunc, \u001b[39m\u001b[39margs, \u001b[39m\u001b[39m\u001b[39m\u001b[39mkwargs)\n\nFile \u001b[0;32m~/gitrepo/gal_dev/.venv/lib/python3.10/site-packages/beaker/util.py:189\u001b[0m, in \u001b[0;36mSyncDict._create\u001b[0;34m(self, key, createfunc, *args, *kwargs)\u001b[0m\n\u001b[1;32m 188\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39m_create\u001b[39m(\u001b[39mself\u001b[39m, key, createfunc, \u001b[39m\u001b[39margs, \u001b[39m\u001b[39m\u001b[39m\u001b[39mkwargs):\n\u001b[0;32m--> 189\u001b[0m \u001b[39mself\u001b[39m[key] \u001b[39m=\u001b[39m obj \u001b[39m=\u001b[39m createfunc(\u001b[39m\u001b[39;49margs, \u001b[39m\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n\u001b[1;32m 190\u001b[0m \u001b[39mreturn\u001b[39;00m obj\n\nFile \u001b[0;32m~/gitrepo/gal_dev/.venv/lib/python3.10/site-packages/beaker/ext/database.py:76\u001b[0m, in \u001b[0;36mDatabaseNamespaceManager.init..\u001b[0;34m()\u001b[0m\n\u001b[1;32m 73\u001b[0m cache_table\u001b[39m.\u001b[39mcreate(bind\u001b[39m=\u001b[39mengine, checkfirst\u001b[39m=\u001b[39m\u001b[39mTrue\u001b[39;00m)\n\u001b[1;32m 74\u001b[0m \u001b[39mreturn\u001b[39;00m cache_table\n\u001b[0;32m---> 76\u001b[0m engine \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m\u001b[39mclass\u001b[39m\u001b[39m.\u001b[39mbinds\u001b[39m.\u001b[39mget(url, \u001b[39mlambda\u001b[39;00m: sa\u001b[39m.\u001b[39;49mengine_from_config(sa_opts, \u001b[39m'\u001b[39m\u001b[39msa.\u001b[39m\u001b[39m'\u001b[39m))\n\u001b[1;32m 77\u001b[0m table \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m\u001b[39mclass\u001b[39m\u001b[39m.\u001b[39mtables\u001b[39m.\u001b[39mget(table_key, \u001b[39mlambda\u001b[39;00m: make_table(engine))\n\u001b[1;32m 79\u001b[0m SqlaNamespaceManager\u001b[39m.\u001b[39m\u001b[39minit\u001b[39m(\u001b[39mself\u001b[39m, namespace, engine, table,\n\u001b[1;32m 80\u001b[0m data_dir\u001b[39m=\u001b[39mdata_dir, lock_dir\u001b[39m=\u001b[39mlock_dir)\n\n\u001b[0;31mAttributeError\u001b[0m: 'NoneType' object has no attribute 'engine_from_config'" }

[amol-]

That seems to suggest that you don't have sqlalchemy installed, which is required for database extension to work.

[claudiofr]

I do have sqlalchemy installed. As I said it works perfectly fine in Beaker V 1.11.0. I was wondering if they have a regression test for type=ext:database because this should have failed.

[claudiofr]

I do have sqlalchemy installed. As I said it works perfectly fine in V 1.11.0.I was wondering if they have a regression test for type=ext:database because this should have failed.

On Saturday, January 28, 2023 at 10:11:13 AM EST, Alessandro Molina ***@***.***> wrote:  

That seems to suggest that you don't have sqlalchemy installed, which is required for database extension to work.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[amol-]

I do have sqlalchemy installed. As I said it works perfectly fine in Beaker V 1.11.0. I was wondering if they have a regression test for type=ext:database because this should have failed.

Yes there are tests for ext:database and they currently pass: https://github.com/bbangert/beaker/blob/master/tests/test_database.py

The error you are reporting seems to be "message": "'NoneType' object has no attribute 'engine_from_config'", That would suggest that sa is None when used here: https://github.com/bbangert/beaker/blob/master/beaker/ext/database.py#L76 sa should be initialised to the sqlalchemy module -> https://github.com/bbangert/beaker/blob/master/beaker/ext/database.py#L21-L30

If that doesn't happen it is probably because _init_dependencies is not being invoked, as _init_dependencies is invoked by the constructor itself, it's probably a regression introduced by the removal of OpenResourceNamespaceManager.__init__(self, namespace) due to the change of inheritance hierarchy for DatabaseNamespaceManager

That would be a reasonable cause because the test suite invokes _init_dependencies explicitly and thus would continue to work.

Can you try to install beaker with pip install git+https://github.com/bbangert/beaker to confirm that the latest version of the codebase works for you?

[claudiofr]

When I install beaker from the github repository using the link you gave it works fine. The 1.12.0 version on PyPI fails with error. Apparently the PyPI 1.12.0 version doesn't match the latest in your github repository.

On Sunday, January 29, 2023 at 06:52:37 PM EST, Alessandro Molina ***@***.***> wrote:  

I do have sqlalchemy installed. As I said it works perfectly fine in Beaker V 1.11.0. I was wondering if they have a regression test for type=ext:database because this should have failed.

Yes there are tests for ext:database and they currently pass: https://github.com/bbangert/beaker/blob/master/tests/test_database.py

The error you are reporting seems to be "message": "'NoneType' object has no attribute 'engine_from_config'", That would suggest that sa is None when used here: https://github.com/bbangert/beaker/blob/master/beaker/ext/database.py#L76 sa should be initialised to the sqlalchemy module -> https://github.com/bbangert/beaker/blob/master/beaker/ext/database.py#L21-L30

If that doesn't happen it is probably because _init_dependencies is not being invoked, as _init_dependencies is invoked by the constructor itself, it's probably a regression introduced by the removal of OpenResourceNamespaceManager.init(self, namespace) due to the change of inheritance hierarchy for DatabaseNamespaceManager

That would be a reasonable cause because the test suite invokes _init_dependencies explicitly and thus would continue to work.

Can you try to install beaker with pip install git+https://github.com/bbangert/beaker to confirm that the latest version of the codebase works for you?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[amol-]

Beaker 1.12.1 with the fix has been released

https://pypi.org/project/Beaker/1.12.1/