openAIP / openaip-openair-parser

OpenAIR format parser. Parses openAIR format and outputs in JSON or GeoJSON format.
MIT License
5 stars 7 forks source link

Token 'AL' on line 155 does not allow subsequent token 'AF' #34

Closed llauner closed 1 year ago

llauner commented 1 year ago

This was fixed in 1.10 but it's there again in 1.16

31

Thanks

reskume commented 1 year ago

The tests are passing for this. Can you please post the actual openAIR definition block so I can see whats being parsed actually? Without this, it's not possible to investigate this issue further.

llauner commented 1 year ago

Here is the file[](url) france.txt

reskume commented 1 year ago

Also, one more thing. This is actually not an error in code but points to an eventual error in the openAIR definition. I suppose you read an extended openAIR formatted file/block but the parser is set to parse the default format. You have to specifically tell the parser to parse the extended format using the init option "extendedFormat: true". But this would require you to also use the extended format, e.g. AY/AC tag (which is anyway the better format ;) ).

For import into openAIP, I always rewrite the FR file into the extended format to not loose all type information. If you like I can link it here.

llauner commented 1 year ago

I don't think I'm using an extended format. I'm doing an automated file validation when it's pushed to a repo. The fact that I don't have the "Successfully parsed" message causes my github action to fail. Same when I'm doing it manually, the message I'm reading says "error" in my file. This used to be validated correctly in 1.10 Are you saying that AF is only supported in the extended format ??

reskume commented 1 year ago

Yep, AF,AG,AY are not standard openAIR. They all are "extended" tags that do not occur in the official docs. They are defined in the section "Additional OpenAir fields (differences from original)".

llauner commented 1 year ago

Well, we had this discussion here: https://github.com/planeur-net/airspace/issues/16

The same file validates fine with 1.12 without the -E option. I'll keep using 1.12 as I don't have any other option right now

reskume commented 1 year ago

1.12 is way behind the current release. With 1.16, an important fix has been added to assure correct validation of openAIR tag ordering, especially when dealing with extended format tags. I also think, the fact that the parser is very strict about the used format, is a huge plus. It will pinpoint errors in the file instantly and when the file passes, everybody that uses this file can be sure that it aligns with a specific format definition.

llauner commented 1 year ago

I can't upgrade the file to extended version because client software (xcsoar, seeyou, ...) don't support it. The AF is fine as it's extra information that is unknown and therefore ignored. So I can't use anything > 1.12 Could you add an option that maybe tolerated the non breaking change between the standard and the extended format ?

What's your suggestion ?

llauner commented 1 year ago

Is there a way to automatically convert to extended format so that I can check again if it's supported by xcsoar and seeyou ?

reskume commented 1 year ago

In this case, I would suggest a fork and rewriting the code but this could get tricky due to the nature of the parser. The parser has two formats and they can be configured to validate against. This was one of the main design goals of this parser: to have strict format validation because we ran into so many problems with all the abominations of various openAir files that we wanted a parser that pinpoints errors (and does not ignore them) and outputs reliable data that we can use in our software.

I also assume that Naviter/SeeYou should already support the extended format. The initial proposal from 2018 to have an extension to the openAIR format with the AY/AF/AG tags is actually from them! Please see the official docs. Also, the Austrian Austrocontrol already offers specific SeeYou/Naviter openAIR files explicitly in the extended format for quite some time now. I assume the support for the extended format has been around for long already, at least for Naviter/SeeYou.

Also, I would think that maintaining the "standard" and "extended" version of the file is not much additional overhead. For each airspace you change you simply have to change the extended version of it or vice versa. In fact, it is also easily possible to simply create a standard version from an extended version. That's what we do at openaip.

reskume commented 1 year ago

Is there a way to automatically convert to extended format so that I can check again if it's supported by xcsoar and seeyou ?

Please find the our import file attached. Please note that this file does not contain "overlapping" airspaces in the German/Swiss area. They have been removed from our side. Mapping to AY is done manually depending either on the Type specification in the AN value.

france_20230420_import.txt

llauner commented 1 year ago

OK, that looks reasonable in terms of change. Anyway you could convert the attached file to the extended format (that would save me the trouble to re-import the change since the one you sent). In the meantime I'll check if the extended version is supported by XCSoar france.txt

reskume commented 1 year ago

Maybe @lordfolken is interested in this issue and can answer if XCSoar already can handle the extended tags?

reskume commented 1 year ago

OK, that looks reasonable in terms of change. Anyway you could convert the attached file to the extended format (that would save me the trouble to re-import the change since the one you sent). In the meantime I'll check if the extended version is supported by XCSoar

I can do that, sure! But please note that we are using the AY values that our parser uses (which includes most of the ones that Naviter proposed in the format extension). Also, the conversion is done by hand in an editor and there may be airspaces where we have to simply assume a specific type, set UNCLASSIFIED etc. The number of airspaces where this happens is small but could be even more reduced when somebody with local knowledge does it.

Anyway, I'll give it a shot when you say you want to provide this format.

llauner commented 1 year ago

OK, I've checked SeeYou, SeeYou Navigator and XCSoar, they all seem to be supporting that so I think we can move forward with the change. Could you provide me with the extended version of the file I sent ? and then I'll have it re-checked by somebody with local knowledge for the AC unclassified Thanks

reskume commented 1 year ago

Yep, can do. I'll create a new ticket on the France airspace repo once I'm done and attach the file to that ticket.