search

powsybl / powsybl-core

A framework to build power system oriented software
https://www.powsybl.org
Mozilla Public License 2.0
126 stars 42 forks source link

CGMES: use only header to identify CGMES profile instead of looking e.g. for _EQ_ in the file name #3205

Open jeandemanged opened 2 weeks ago

jeandemanged commented 2 weeks ago

Describe the current behavior

CGMES importer looks for _EQ_ or _EQ. string presence in file name to filter for EQ profiles.

Describe the expected behavior

A more robust solution would rather check only the model header md:Model.profile to determine what profile(s) a file contains.

Describe the motivation

Got an issue with non-obvious stack trace with one IGM where files were named (allegedly weirdly):

All of them are then considered as EQ. Because they all have same md:Model.modelingAuthoritySet the import then fails with default importer settings.

Workarounds are:

Extra Information

powsybl-core 6.5.0

DEBUG:powsybl:Duplicate key MAS (attempted merging values [confidential_EQ_confidential_SV.xml] and [confidential_EQ_confidential_TP.xml])
java.lang.IllegalStateException: Duplicate key MAS (attempted merging values [confidential_EQ_confidential_SV.xml] and [confidential_EQ_confidential_TP.xml])
        at java.base@17.0.11/java.util.stream.Collectors.duplicateKeyException(Collectors.java:135)
        at java.base@17.0.11/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:182)
        at java.base@17.0.11/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
        at java.base@17.0.11/java.util.stream.Streams$StreamBuilderImpl.forEachRemaining(Streams.java:411)
        at java.base@17.0.11/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
        at java.base@17.0.11/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
        at java.base@17.0.11/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
        at java.base@17.0.11/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
        at java.base@17.0.11/java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1707)
        at java.base@17.0.11/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.base@17.0.11/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base@17.0.11/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
        at java.base@17.0.11/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base@17.0.11/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
        at com.powsybl.cgmes.conversion.CgmesImport$MultipleGridModelChecker.separateByModelingAuthority(CgmesImport.java:260)
        at com.powsybl.cgmes.conversion.CgmesImport$MultipleGridModelChecker.separate(CgmesImport.java:247)
        at com.powsybl.cgmes.conversion.CgmesImport.importData(CgmesImport.java:163)
        at com.powsybl.iidm.network.Network.read(Network.java:128)
        at com.powsybl.iidm.network.Network.read(Network.java:135)
        at com.powsybl.python.network.NetworkCFunctions.lambda$loadNetwork$6(NetworkCFunctions.java:171)
        at com.powsybl.python.commons.Util.doCatch(Util.java:132)
        at com.powsybl.python.network.NetworkCFunctions.loadNetwork(NetworkCFunctions.java:163)
zamarrenolm commented 2 weeks ago

Agree on the fact that the names may not follow the expected CIMXML pattern used by CGMES exchanges:

<effectiveDateTime>_<businessProcess>_<sourcingActor>_<modelPart>_<fileVersion>.xml

From the names of your files, EQ should be a businessProcess, which I think is not.

Here we were looking only at the filename to have a fast way of splitting a data source that contained multiple IGMs, trying to avoid reading "inside" each XML. But we have to obtain the modelling authority anyway (by reading the header), so it makes sense to check also the profile.