Simplify the SQLDeveloper to Django implementation

[RBirdwatcher]

Currently this imlementation uses a Pythion model clled RegDNA, which is n older artefacts of the project. As a first step of simplification we can replace RegDNA library with PYEcore library to remove one dependency.

[benjamin-arfa]

In the regdna.py file, it already uses the ecore package, meaning the above-mentioned "dependency" has already been fixed / removed. Does this issue relate to a full refactoring of the regdna.py file -into> pybirdai_ecore_model.py file using provided classes from the pyecore module ?

[RBirdwatcher]

Hi @benjamin-arfa .

Regdna.py is currently used for only one purpose. CSV files exported from SQLDEveloper related to the BIRD Enriched Input Layer or LDM are converted to be described in the classes described in regdna.py such as ELClass , ELAttribute, ELEnum. This same/similar code existed in the older Java application also which made use of Ecore instead of Django to describe models. This is done in the birds_nest\pybirdai\process_steps\sqldeveloper_import\import_sqldev_il_to_regdna.py and birds_nest\pybirdai\process_steps\sqldeveloper_import\import_sqldev_ldm_to_regdna.py classes. These processes translate from a CSV format to something that looks a lot more like a MetaModel and so is much closer to the Django models format.

Since we moved to use Python Django now and not Ecore, it was easy to then translate from these classes such as ELClass , ELAttribute, ELEnum into A Django models file, this is done in birds_nest\pybirdai\process_steps\sqldeveloper_import\import_sqldev_ldm_to_django.py (that same file deals with BIRD Enriched Input layer or the LDM as the process is the same)

The classes such as ELClass , ELAttribute, ELEnum are almost identical to the classes in the open source PyECore project (EClass, EAttribute, EEnum) and we should just use these instead so that we remove a dependency on the regdna.py file, which we don’t need anymore.

Regdna.py also introduced new classes related to tranformations which we no longer need such as RulesForILTable as we now use the standard SMCubes entity called CUBE_LINK and CUBE_STRUCTURE_ITEM_LINK. Regdna.py introduced new classes related to report cells but we now use the standard SMCubes entity called COMBINATIONS that you will be familiar with. So we can ignore those extra classes in of Regdna.py

Continuing to use PyEcore has an advantage that we can very easily create an ecore version of our model if we want to…this extra format can be nice for interaction with other Ecore tools, but we don’t currently use it.

So I foresee we will just change the lines in import_sqldev_il_to_regdna.py that refer to ELPackage, ELClass , ELAttribute, ELReference, ELEnum in regdna.py to just use EPackage, EClass , EAttribute, EReference, EEnum from the more mature pyecore.ecore class. There maybe some small gotchas, and it will need testing, but that is the approach I am thinking of.

The creation of the regdna.py file was always an extra step in the java/ecore application it will be nice to remove it and rely just on the more mature pyecore file form the PyEcore project.

As a note the classes EClass , EAttribute, EReference, EEnum and a few others are defined in the standard Ecore metamodel..It is nice to use these concepts as they are well tested, and can provide interaction with ecore if we really want it (but our focuss is on Django of course)

Any further questions please feel free to ask. Thanks

[benjamin-arfa]

can I use the sqldeveloper data to test the process ? I will try to repolace the EL classes with Ecore Classes

[RBirdwatcher]

Sure, you can get the SQLDeveloper files by following the instructions here https://github.com/eclipse-efbt/efbt/wiki/Export-Files-from-SQLDeveloper for BIRD 6.4.0 , best to use the Enriched input layer. Also I can email you copy of the files I already extracted if this is easier