search

NEMRI-org / primo-optimizer

PRIMO - The P&A Project Optimizer Toolkit aims to provide multi-scale, optimization-based, open source computational tools and models to support the Methane Emissions Reduction Program (MERP) and the National Emissions Reduction Initiative (NEMRI).
Other
0 stars 7 forks source link

Long computation times for the pairwise metric matrix computation #54

Open strahlw opened 1 week ago

strahlw commented 1 week ago

In the OptModelInputs object, the _pairwise_matrix requires significant computation time because it computes the distance matrix for the entire well data before then extracting the needed values for the clusters. If the number of wells included in projects is small, then there is no reason to compute the entire distance matrix. Whether or not we will need this to be efficient is still a question (if the re-optimize step of the manual override goes to a worker which will add the operation to a queue, then I think it is less important).

strahlw commented 1 week ago

^^^ As it is implemented, it also will fail for large datasets that require a lot of memory