teksi / wastewater

[DEV] Future TEKSI wastewater module, adapted data model to fit VSA-DSS 2020 new standard
https://teksi.github.io/wastewater
GNU General Public License v3.0
0 stars 5 forks source link

Improvement of performance of INTERLIS tests #309

Open ponceta opened 1 month ago

ponceta commented 1 month ago

Describe the bug Many errors and warnings are displayed when testing INTERLIS import / export features slowing the whole CI process.

teksi_wastewater/tests/test_interlis.py::TestInterlis::test_minimal_import_export
  /usr/src/plugin/teksi_wastewater/interlis/interlis_model_mapping/interlis_exporter_to_intermediate_schema.py:2278: SAWarning: Cannot correctly sort tables; there are unresolvable cycles between tables "pg2ili_abwasser.abwasserbauwerk, pg2ili_abwasser.bauwerksteil, pg2ili_abwasser.deckel", which is usually caused by mutually dependent foreign key constraints.  Foreign key constraints involving these tables will not be considered; this warning may raise an error in a future release.
    self.abwasser_session.flush()

-- Docs: https://docs.pytest.org/en/stable/warnings.html
================= 4 passed, 312 warnings in 154.48s (0:02:34) ==================

To Reproduce Exact steps to reproduce the behavior:

  1. Run plugin tests
  2. See errors and warnings

Expected behavior CI of INTERLIS imports and exports takes a reasonnable time

Screenshots / data

ponceta commented 1 month ago

@domi4484 could you take a look at it?

sjib commented 1 month ago

That is like that in the data model:

We have wastewater_structure.fk_main_cover and cover.obj_id that points to structure_part.obj_id and from there structure_part.fk_wastewater_structure

We cannot change these cyclic references in the data model. The are needed from a professionsal (fachlich) perspective

ponceta commented 1 month ago

That is like that in the data model:

We have wastewater_structure.fk_main_cover and cover.obj_id that points to structure_part.obj_id and from there structure_part.fk_wastewater_structure

We cannot change these cyclic references in the data model. The are needed from a professionsal (fachlich) perspective

And

teksi_wastewater/tests/test_interlis.py::TestInterlis::test_minimal_import_export
  /usr/src/plugin/teksi_wastewater/interlis/utils/tww_sqlalchemy.py:72: SAWarning: Implicitly combining column baseclass.t_basket with column kanalschaden.t_basket under attribute 't_basket'.  Please configure one or more attributes for these same-named columns explicitly.
    base.prepare(

Is also required or is there any improvement possible?

ponceta commented 1 month ago

Improvement proposals from @domi4484 are here :

sjib commented 1 month ago

@domi4484 Some ideas of documenation that could give hints on that: https://stackoverflow.com/questions/7797403/sqlalchemy-error-please-configure-one-or-more-attributes-for-these-same-named or the sqlalchemy documentation for JOIN https://docs.sqlalchemy.org/en/20/orm/join_conditions.html#overlapping-foreign-keys