search

wmo-im / iwxxm

XML schema and Schematron for aviation weather data exchange
https://old.wmo.int/wiswiki/tiki-index.php%3Fpage=TT-AvXML
49 stars 22 forks source link

Ensure that minimum information content of failed translation messages is correct #41

Closed braeckel closed 6 years ago

braeckel commented 7 years ago

In ICAO METP working groups it was agreed that the minimum set of metadata for partially translated messages:

METAR: METAR (COR) CCCC YYGGggZ TAF: TAF (COR/AMD) CCCC YYGGggZ SIGMET/AIRMET: CCCC SIGMET | AIRMET ... VALID YYGGgg/YYGGgg VAA: DTG, VAAC TCA: DTG, TCAC

There are existing examples illustrating this (*-translation-failed) but these need to be checked against the minimum information above. Further, schema modifications may need to be made if there is a mismatch.

Also, as part of this activity it would be good to ensure that -NIL messages are closely matched to -translation-failed other than the extra translation metadata.

braeckel commented 6 years ago

After checking the current schemas, the required changes for the failed translation cases would be:

For each of the fields above they should be made optional (0..1) and rules should be added to ensure that they are present in normal (non-failed translation) cases. @blchoy Do you have any thoughts on this proposal before I start making these modifications? I would lean towards having a single rule that checks each of these (rather than separate rules for each) - for example a single rule would ensure that normal cases of SIGMET have sequenceNumber, issuingAirTrafficServicesRegion, and originatingMeteorologicalWatchOffice.

@blchoy Only METAR and TAF have NIL examples currently, is this correct and are these the only reports that can be NILed? If so no changes seem to be required since the contents of NIL and translation failed cases match for METAR and TAF.

blchoy commented 6 years ago

Only METAR and TAF have NIL examples currently, is this correct and are these the only reports that can be NILed?

Yes. It is only meaningful to have "nil" report for routine reports for obvious reason. I agree with you no "translation failed" examples are required.

blchoy commented 6 years ago

After checking the current schemas, the required changes for the failed translation cases would be:

  • AIR/SIGMET: sequenceNumber, issuingAirTrafficServicesRegion, originatingMeteorologicalWatchOffice
  • TCA: tropicalCycloneName, advisoryNumber, observation, forecast, expectedNextAdvisoryTime
  • VAA: volcano, advisoryNumber, informationSource, colourCode, analysis

For SIGMET/AIRMET, I think the sequenceNumber should be retained (this is hidden in the "..." of the minimum requirement :) )

I have no further suggestions to TCA and VAA.

braeckel commented 6 years ago

OK, the following will be optional with rules to ensure their presence in normal cases.

mgoberfield commented 6 years ago

colourCode is an optional field in routinely normally issued VAA. For instance VAAs from Washington do not include them.

braeckel commented 6 years ago

Thanks, Mark. You are correct, I removed it from the list.

I noticed that we have not yet considered SWA failed translation cases, I added my proposed set to the list below. These are somewhat similar to VAA.

If we follow the suggestions of my prior comment it would mean for AIR/SIGMET that in failed translation cases the issuingATS and originatingWMO would be simply missing, while other information like phenomenon and analysis would look like the following because they are nillable but not optional:

     <iwxxm:phenomenon nilReason="missing"/>
     <iwxxm:analysis nilReason="missing"/>

For consistency I think it is worth including all of these properties in the new set of optional fields so that the same approach is used for all information that can be missing in failed translation cases. Here would be the new list (new fields in bold):

The downside of making all of these fields optional is that there will be States or developers that look only at the XML schema (not Schematron) during development who will conclude that there are fewer fields required and they will get validation failures when issuing or validating their messages for the first time. I don't think this is at all an unlikely scenario, but overall I still prefer the option of making everything optional and relying on the Schematron rules to catch these cases so that we don't have a mixture of approaches in IWXXM (some missing, some nil...). It is unfortunate that this change would introduce some new confusion but States will need to be aware of the Schematron rules for many situations beyond this one.

braeckel commented 6 years ago

I've created PR #91 which includes the changes discussed in https://github.com/wmo-im/iwxxm/issues/41#issuecomment-395123782. If issues are identified with this approach I can to update the PR, but it should help with understanding the implications of the changes discussed above.

blchoy commented 6 years ago

Perhaps it is too late to ask this question: Why we need to translate TCA, VAA and SWA? Who is going to translate them? Aren't they prepared at the source?