OpenEnergyPlatform / open-MaStR

A collaborative software to download the energy database Marktstammdatenregister (MaStR)
https://open-mastr.readthedocs.io/en/latest/
GNU Affero General Public License v3.0
87 stars 19 forks source link

Feature 403 SQLA 2 compatibility #405

Closed deniztepe closed 1 year ago

deniztepe commented 1 year ago

Connectionless database queries are replaced with explicit connection initializations, since this functionality will be removed with the upcoming SQLAlchemy 2.0 version.

nesnoj commented 1 year ago

Thanks for the fix! If I remember correctly, I also had issues during CSV export. But I assume the review approvals included a (manual) test of this feature?

chrwm commented 1 year ago

I tested it for a few thousand units and all tables in #401 (the branch includes this PR already) and it worked fine. I'm running a full export now just to be sure.

nesnoj commented 1 year ago

I tested it for a few thousand units and all tables in #401 (the branch includes this PR already) and it worked fine. I'm running a full export now just to be sure.

Great, thx!

chrwm commented 1 year ago

When downloading a fresh dump from today, the export of all tables and all rows works fine. When trying to export an older sqlite db, this commit from 04.01.23 350dd1c leads to an error, as the new columns are not in the db.

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such column: gas_storage_extended.DatumDesBetreiberwechsels
[SQL: SELECT gas_storage_extended."DatenQuelle", gas_storage_extended."DatumDownload", gas_storage_extended."EinheitMastrNummer", gas_storage_extended."DatumLetzteAktualisierung", gas_storage_extended."LokationMastrNummer", gas_storage_extended."NetzbetreiberpruefungStatus", gas_storage_extended."NetzbetreiberpruefungDatum", gas_storage_extended."AnlagenbetreiberMastrNummer", gas_storage_extended."Land", gas_storage_extended."Bundesland", gas_storage_extended."Landkreis", gas_storage_extended."Gemeinde", gas_storage_extended."Gemeindeschluessel", gas_storage_extended."Postleitzahl", gas_storage_extended."Ort", gas_storage_extended."Strasse", gas_storage_extended."StrasseNichtGefunden", gas_storage_extended."Hausnummer", gas_storage_extended."Hausnummer_nv", gas_storage_extended."HausnummerNichtGefunden", gas_storage_extended."Laengengrad", gas_storage_extended."Breitengrad", gas_storage_extended."Registrierungsdatum", gas_storage_extended."Inbetriebnahmedatum", gas_storage_extended."EinheitSystemstatus", gas_storage_extended."EinheitBetriebsstatus", gas_storage_extended."NichtVorhandenInMigriertenEinheiten", gas_storage_extended."NameGasspeicher", gas_storage_extended."Speicherart", gas_storage_extended."MaximalNutzbaresArbeitsgasvolumen", gas_storage_extended."MaximaleEinspeicherleistung", gas_storage_extended."MaximaleAusspeicherleistung", gas_storage_extended."DurchschnittlicherBrennwert", gas_storage_extended."Weic", gas_storage_extended."Weic_Na", gas_storage_extended."SpeicherMastrNummer", gas_storage_extended."Gemarkung", gas_storage_extended."FlurFlurstuecknummern", gas_storage_extended."Adresszusatz", gas_storage_extended."DatumBeginnVoruebergehendeStilllegung", gas_storage_extended."DatumDesBetreiberwechsels", gas_storage_extended."DatumRegistrierungDesBetreiberwechsels" 
FROM gas_storage_extended]
(Background on this error at: http://sqlalche.me/e/14/e3q8)