Store conic duals in arrays instead of separating coordinates - Githubissues

AI4OPT / OPFGenerator

Instance generator for OPF problems

MIT License

2 stars 3 forks source link

Store conic duals in arrays instead of separating coordinates #76

Closed mtanneau closed 2 months ago

mtanneau commented 3 months ago

Please have a look but do not merge just yet, I want to run additional checks on my end to ensure that the data-generation pipeline works smoothly.

mtanneau commented 2 months ago

There is an issue with the conversion from multiple JSON to one H5. I need to update the convert_to_h5! function to fix that. Upstream code should be OK

mtanneau commented 2 months ago

@klamike @ivanightingale PR ready for review. I fixed the issue with conic duals (see tensorize function in commit https://github.com/AI4OPT/OPFGenerator/pull/76/commits/453e9dcb83e6f9b938c7275e959f0f748ce5e5ed)

mtanneau commented 2 months ago

assuming the slurm pipeline runs without error

It does not... I need to update the code that merges multiple H5 files.

mtanneau commented 2 months ago

I hope things are OK now. I am re-running a complete slurm pipeline locally to check that everything works fine. I am some further improvements for merging H5 files, which I will include in a separate PR.

mtanneau commented 2 months ago

OK, now we're good to go. I ran this on a full slurm pipeline and checked the outputs.

ivanightingale commented 2 months ago

Regarding the vector variables, looks like the number of rows (the dimension of the first axis) of the final 3D matrix is equal to the dimension of the variable (e.g. 3 for nu_sm_fr)?

mtanneau commented 2 months ago

Regarding the vector variables, looks like the number of rows (the dimension of the first axis) of the final 3D matrix is equal to the dimension of the variable (e.g. 3 for nu_sm_fr)?

This is how things should be:

the result dictionary (JSON format) has vector-valued conic duals (nu_sm, nu_jabr); no changes otherwise
the h5 result (per instance) stores the conic dual in a Ex3 or Ex4 matrix (whether it's thermal constraints or jabr inequality)
the final H5 file (containing the whole dataset) should store the conic duals in a Ex3xM or Ex4xM 3-dimensional array

Let me triple-check the final bullet.