New table `einheitenaenderungnetzbetreiberzuordnungen`

[m-jankowski]

Description of the issue

Since the update of the Martkstammdatenregister on April 18 (release notes: https://www.marktstammdatenregister.de/MaStRHilfe/subpages/releasenotes.html) the open-mastr bulk download("bulk") results in an error. It probably has to do with the "Öffentliche Daten: Neuer Menüpunkt "Änderung der Netzbetreiberzuordnung" - Übersicht der Einheiten mit geänderter Netzbetreiberzuordnung" change of the MaStR.

Error message:

KeyError                                  Traceback (most recent call last)
Cell In[3], line 2
      1 db = Mastr()
----> 2 db.download(method='bulk')

File [/opt/homebrew/Caskroom/miniconda/base/envs/work/lib/python3.11/site-packages/open_mastr/mastr.py:231](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniconda/base/envs/work/lib/python3.11/site-packages/open_mastr/mastr.py:231), in Mastr.download(self, method, data, date, bulk_cleansing, api_processes, api_limit, api_chunksize, api_data_types, api_location_types, **kwargs)
    225     zipped_xml_file_path = os.path.join(
    226         xml_folder_path,
    227         f"Gesamtdatenexport_{bulk_download_date}.zip",
    228     )
    229     download_xml_Mastr(zipped_xml_file_path, date, xml_folder_path)
--> 231     write_mastr_xml_to_database(
    232         engine=self.engine,
    233         zipped_xml_file_path=zipped_xml_file_path,
    234         data=data,
    235         bulk_cleansing=bulk_cleansing,
    236         bulk_download_date=bulk_download_date,
    237     )
    239 if method == "API":
    240     validate_api_credentials()

File [/opt/homebrew/Caskroom/miniconda/base/envs/work/lib/python3.11/site-packages/open_mastr/xml_download/utils_write_to_database.py:34](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniconda/base/envs/work/lib/python3.11/site-packages/open_mastr/xml_download/utils_write_to_database.py:34), in write_mastr_xml_to_database(engine, zipped_xml_file_path, data, bulk_cleansing, bulk_download_date)
     30 for file_name in files_list:
     31     # xml_tablename is the beginning of the filename without the number in lowercase
...
     77 )
     78 # check if the table should be written to sql database (depends on user input)
     79 include_count = include_tables.count(xml_tablename)

KeyError: 'einheitenaenderungnetzbetreiberzuordnungen'

Steps to Reproduce

from open_mastr import Mastr
db = Mastr()
db.download(method='bulk')

Context and Environment

• Version used: open-mastr 0.14.2
• Operating system: MacOS
• Environment setup and (python) version: Python 3.11.4

Workflow checklist

• [X] I am aware of the workflow in CONTRIBUTING.md

[FlorianK13]

@m-jankowski Thanks for your report, I will try to reproduce the error.

[glilienthal]

I have the same error.

[FlorianK13]

I can reproduce the error, there is a new table coming from BNetzA. @chrwm Seems like we really need to fix the tests now for an urgent new release.

[FlorianK13]

The fix is merged on develop and will be included in the next release, v0.14.3 #513

[kknappmann]

Hi @FlorianK13, do you have an update on when the new Version will be available?

[FlorianK13]

Hi @kknappmann, We are in the process of releasing, see #513. If everything works fine, it might be on pypi this afternoon.

[FlorianK13]

New release is on pypi, it should work now!

[kknappmann]

Hi @FlorianK13 I now get the following error:
Line: db.download(data="solar")

File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/open_mastr/mastr.py", line 231, in download write_mastr_xml_to_database(File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/open_mastr/xml_download/utils_write_to_database.py", line 34, in write_mastr_xml_to_database if is_table_relevant( ^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/open_mastr/xml_download/utils_write_to_database.py", line 76, in is_table_relevant tablename_mapping[xml_tablename]["class"] is not None

KeyError: 'einheitenaenderungnetzbetreiberzuordnungen'

Do you encounter this too?

[FlorianK13]

Hi @kknappmann which version of open-mastr do you have installed? You can run pip install --upgrade open-mastr to get version 0.14.3 which should have the bugfix included.

[kknappmann]

I did and am on version 0.14.3 but encounter this error anyway, see my screenshot

[FlorianK13]

@kknappmann On version 0.14.3 I can parse the new table to my sqlite database. Can you try to delete the existing database that you have (or rename it) so that a new database is initiated?

[kknappmann]

that worked, thanks!