iiasa / ixmp4

A data warehouse for high-powered scenario analysis in the domain of integrated assessment of climate change and energy systems modeling
https://docs.ece.iiasa.ac.at/ixmp4
MIT License
11 stars 6 forks source link

DRY optimization core layer #96

Open glatterf42 opened 4 months ago

glatterf42 commented 4 months ago

For #94, @danielhuppmann asked in the core layer: "Why does Parameter not inherit from Table"? And sure enough, these optimization items share a lot of common functionality. So before moving on to Variable and Equation, for which this will be the case even more so, I wanted to take another look at this.

@meksor, I expect you'll be the one to take a look at this. Sorry, this description might be a bit lengthy.

Initially, I interpreted the question as for the DB layer, and how to handle inheritance (or rather Composition) there has been recently discussed in #82. Instead, this PR focuses on the core layer, where we also have two classes per optimization item that could potentially benefit from higher reusability (to follow the DRY principle): e.g., the Table (mostly a collection of data attributes) and the TableRespository (functions to handle Tables). For the Tables themselves (and similar), we are currently using Python's property to make data attributes accessible in a controlled way. Unfortunately, these propertys don't mix well with inheritance. This article describes how to still make it work for the getter part, but since we would still have to repeat the same number of lines in the data files, I'm not sure it's worth the effort here. What we stand to gain is a separation of functionality and placeholder code, it seems to me. So when we want to make changes to how e.g. data is returned from an optimization model, we would just need to change one place and all others would inherit these changes. However, code repetition would not be decreased and understanding the models might become more complicated, so I've put this on hold for now. For TableRepository, there is luckily a quite straightforward way: we can employ the same approach as in the DB layer and create an OptimizationBaseRepository in the core layer that can be extended with Creator, Retriever, etc as needed. This seems to work well and every optimization item repo then just needs to clarify which _model_type and _backend_repository it is interested in. For better understanding, I wanted to type hint the _backend_repository in the OptimizationBaseRepository. However, we didn't already have an appropriate object to do so. That's why I have created a BackendBaseRepository in abstract.optimization.base, which simply announces how the actual TableRepository in the DB layer is going to look. I'm not sure if this is the correct location; I've usually worked with abstract models to type hint API/DB layer functionality, but I also don't know where else it would live.

This is the point where I'm not sure I'm naming things correctly anymore. I started from Table, which already has Table.data, which is arguably the largest part of shared functionality, at least in the DB layer, and which is missing from IndexSet and Scalar. So these two items could not use the OptimizationBaseRepository as it is now, which seems like a misnomer. Maybe the data functionality needs to be further separated (allowing us to DRY even more parts of this layer) or maybe we find another name? The same applies for BackendBaseRepository.


[!NOTE] For the curious: there's an issue on CPython about propertys and inheritance from 2012.

meksor commented 1 month ago

@glatterf42 OK so i think the currently proposed changes (removing the commented base model) are good. I also think the rest of the facade layer could also benefit from something like this. Lets figure the @property part out later, for now just explicitly having them in every leaf class is okay imo. Since we are now not depending on sqlalchemys session-synchronization anyway (changing something in a model with id=1 will change it in all models referencing that row) it would also be okay to just reassign the properties (maybe automatically) to the facade class from the api or db model, thereby completely removing the whole property thing for most stuff.