OpenEnergyPlatform / omi

Repository for the Open Metadata Integration (OMI). For metadata definition see metadata repo:
https://github.com/OpenEnergyPlatform/metadata
GNU Affero General Public License v3.0
7 stars 4 forks source link

ValueError if parse_date gets none valid input #12

Closed 4lm closed 4 years ago

4lm commented 5 years ago

Hi all,

I have around 30 JSON files I try to convert from OEP-1.3 to OEP-1.4.

Around ten files are failing, because they have in temporal.reference_date a none valid input (string: "none" instead of a datestring) and produce an error ("Unknown string format:", timestr).

If I have a look at the corresponding file [parse.py] (https://github.com/OpenEnergyPlatform/omi/blob/dev/src/omi/dialects/oep/parser.py) I can see that there are multiple instances in which parse_date is used without a safety net.

I would solve the issue in the concerning code as follows. Also for all the other occurrences of parse_date in parse.py.

BEFORE:

        # filling the temporal section
        temporal = structure.Temporal(
            reference_date=parse_date(json_old["temporal"].get("reference_date")),
            start=None,
            end=None,
            resolution=None,
            ts_orientation=None,
        )

AFTER:

        # filling the temporal section
        try:
            temporal__reference_date = parse_date(json_old["temporal"].get("reference_date"))
        except ValueError:
            temporal__reference_date = json_old["temporal"].get("reference_date")
        temporal = structure.Temporal(
            reference_date=temporal__reference_date,
            start=None,
            end=None,
            resolution=None,
            ts_orientation=None,
        )

What do you think? Is this approach feasible?

MGlauer commented 5 years ago

Yes, this is part of a larget issue, namely me not beeing aware of the fact that we do not have mandatory fields. I am working on it in features/everything_is_optional

christian-rli commented 4 years ago

Was this issue addressed in #13 ? I tried to parse a sample OEP-1.3 file after the merge and for testing purposes changed the date to "none". I got the following (same?) Error:

    raise ValueError("Unknown string format:", timestr)
ValueError: ('Unknown string format:', 'none')
MGlauer commented 4 years ago

Omi is able to parse the string

{
    "id": "id"
}

correctly in v1.3 and v1.4. Therefore I consider this closed