sneakers-the-rat / paper-feeds

A FastAPI web server for creating RSS feeds for scholarly journals with the magic of adversarial interoperability
GNU General Public License v3.0
63 stars 4 forks source link

Model schema incompatibility - implement database migrations #9

Closed roaldarbol closed 10 months ago

roaldarbol commented 10 months ago

I am currently getting an error when I try searching on the up-to-date main branch. Seems related to journal.recent_paper_count.

I just run poetry run start and type Journal (or anything else for that matter).

Logs

``` (base) ➜ journal-rss git:(css-styling) ✗ poetry run start INFO: Will watch for changes in these directories: ['/Users/roaldarbol/Filen/Projects/journal-rss'] INFO: Uvicorn running on http://localhost:8000 (Press CTRL+C to quit) INFO: Started reloader process [30343] using StatReload INFO: Started server process [30349] INFO: Waiting for application startup. INFO: Application startup complete. INFO: 127.0.0.1:50826 - "GET / HTTP/1.1" 200 OK INFO: 127.0.0.1:50826 - "POST /search HTTP/1.1" 500 Internal Server Error ERROR: Exception in ASGI application Traceback (most recent call last): File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1969, in _exec_single_context self.dialect.do_execute( File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 922, in do_execute cursor.execute(statement, parameters) sqlite3.OperationalError: no such column: journal.recent_paper_count The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__ return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ await super().__call__(scope, receive, send) File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/starlette/applications.py", line 116, in __call__ await self.middleware_stack(scope, receive, send) File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__ raise exc File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__ await self.app(scope, receive, _send) File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/starlette/_exception_handler.py", line 55, in wrapped_app raise exc File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/starlette/_exception_handler.py", line 44, in wrapped_app await app(scope, receive, sender) File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/starlette/routing.py", line 746, in __call__ await route.handle(scope, receive, send) File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/starlette/routing.py", line 288, in handle await self.app(scope, receive, send) File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/starlette/routing.py", line 75, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/starlette/_exception_handler.py", line 55, in wrapped_app raise exc File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/starlette/_exception_handler.py", line 44, in wrapped_app await app(scope, receive, sender) File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/starlette/routing.py", line 70, in app response = await func(request) ^^^^^^^^^^^^^^^^^^^ File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/fastapi/routing.py", line 299, in app raise e File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/fastapi/routing.py", line 294, in app raw_response = await run_endpoint_function( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function return await dependant.call(**values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/roaldarbol/Filen/Projects/journal-rss/src/journal_rss/app.py", line 51, in search results = crossref.store_journal(results) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/roaldarbol/Filen/Projects/journal-rss/src/journal_rss/services/crossref.py", line 77, in store_journal journals.append(load_journal(r.issn[0].value)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/roaldarbol/Filen/Projects/journal-rss/src/journal_rss/services/crossref.py", line 89, in load_journal db_journal = session.exec(read_statement).first() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/sqlmodel/orm/session.py", line 68, in exec results = super().execute( ^^^^^^^^^^^^^^^^ File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2308, in execute return self._execute_internal( ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2190, in _execute_internal result: Result[Any] = compile_state_cls.orm_execute_statement( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/sqlalchemy/orm/context.py", line 293, in orm_execute_statement result = conn.execute( ^^^^^^^^^^^^^ File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1416, in execute return meth( ^^^^^ File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/sqlalchemy/sql/elements.py", line 517, in _execute_on_connection return connection._execute_clauseelement( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1639, in _execute_clauseelement ret = self._execute_context( ^^^^^^^^^^^^^^^^^^^^^^ File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1848, in _execute_context return self._exec_single_context( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1988, in _exec_single_context self._handle_dbapi_exception( File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2344, in _handle_dbapi_exception raise sqlalchemy_exception.with_traceback(exc_info[2]) from e File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1969, in _exec_single_context self.dialect.do_execute( File "/Users/roaldarbol/Filen/Projects/journal-rss/.venv/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 922, in do_execute cursor.execute(statement, parameters) sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such column: journal.recent_paper_count [SQL: SELECT journal.title, journal.publisher, journal.recent_paper_count, journal.feed, journal.feed_created, journal.id FROM journal JOIN issn ON journal.id = issn.journal_id WHERE issn.value = ?] [parameters: ('1901-6220',)] (Background on this error at: https://sqlalche.me/e/20/e3q8) ```
sneakers-the-rat commented 10 months ago

Db schema incompatibility: looks like we need to introduce migrations at "right about now" levels of complexity.

Short term fix: delete database sqlite file and recreate on next launch :)

edit: i believe on python database migrations mean alembic atm?

roaldarbol commented 10 months ago

Check, deleting the db worked!

sneakers-the-rat commented 10 months ago

Done - https://github.com/sneakers-the-rat/journal-rss/commit/c4413f812d2186e46a0de9953a3080416957c9af