search

RobokopU24 / ORION

Code that parses datasets from various sources and converts them to load graph databases.
MIT License
12 stars 13 forks source link

Fix for get_latest_source_version being called unnecessarily. #237

Open DnlRKorn opened 3 months ago

DnlRKorn commented 3 months ago

Made pretty much all calls for source versioning into dynamic calls.

Resolves issue #236 and number 3 of issue #227 .

Modified the DataSource class in kgxmodel.py to make source_version generated when needed instead of at objection creation time. Modified build_manager.py to get strings which describe graph_version as needed for building the graph and dependant subgraphs and not unnecessarily.

EvanDietzMorris commented 2 months ago

Finally got to testing this out. The subgraph logic was broken here because after your changes it would always try to dynamically generate the version for a subgraph using information from the graph spec, but we had and still want the ability to hard code a subgraph version in the graph spec, ignoring latest versions or other stuff in the graph spec. I made some changes to handle that, and generally improved the error catching and messaging around specifying subgraph versions.

I made some other (possibly questionable) changes to the DataSource part, entirely because the way you did it forced us to always assign a function to get_source_version, even when a version was explicitly specified in the Spec, and that sort of felt bad to me. I wanted us to still be able to provide a string value for the version without using a lambda function when we wanted to. The changes I added accomplish that, but honestly might have made it all a bit messier and harder to understand. Take a look and see what you think and we can change it back or to something else if you want.