search

XCSoar / xcsoar-data-content

XCSoar's data repository
http://download.xcsoar.org
9 stars 45 forks source link

Header for cup files #87

Closed mlep closed 2 years ago

mlep commented 5 years ago

In the description of the cup file format (https://download.naviter.com/docs/CUP-file-format-description.pdf), it is stated that:

"name, code, country, lat, lon, elev, style, rwydir, rwylen, freq, desc" is contained at line number 1 and is ignored when reading. This line is currently missing from (most of/all) the cup files of the repository.

Currently, the cup files contains no metadata. So, after download, one does not know where it came from, when it was created, what to expect from it, how to contribute, etc. What are you thoughts about it? And is it possible to add such metadata in a cup file? ... I could not find anything in the description of the file format...

lordfolken commented 5 years ago

CUP is a terrible file format. It was intended for loggers from the 1990s. Its simplistic, everyone can create it with a spreadsheet, it is loosely defined, for example encoding was not mandated, nor charsets. It supports no metadata, and even the comments are a case of "tolerance" not implementation.

I needed a new source for data, since 8.33khz change in Europe was coming up, and welt2000 was deprecated, without a full replacement. (Both openAIP (incompatible license) and OpenFlightmaps, don't cover soaring turn points.

I also wanted to supply current data, and at least have a way to supply current data to the users. So I created this repository and the continuous integration (testing with aerofiles) / continuous deployment (ssh deploy to download.xcsoar.org) chain in order to have any change live within minutes.

This is by far a sub-optimal solution and i'm certainly far from happy with it. (see also the bug reports on both xcsoar-data-repository and xcsoar-data-content. )

mlep commented 5 years ago

Thank you for your feedback.

On the long run, XCSoar may separate waypoints and airport/landing fields (as a user, I will see this as an improvement It will be easier to maintain too). And XCSoar could also support a more appropriate file format.

With the current cup file format:

By the way, do you confirm that:

lordfolken commented 5 years ago

On the long run, XCSoar may separate waypoints and airport/landing fields (as a user, I will see this as an improvement It will be easier to maintain too). And XCSoar could also support a more appropriate file format.

It should be a generic database with all the waypoint information, plus reliability, source, dates and all other aspects. I talked to the owner of how2soar.de and he uses a database with reputation to filter the sources. He also as a review process involving google earth and determining whether the coordinates are valid. He says that he has waypoints with over 30 different coordinates.. all wrong. With his reputation db, he can quickly determine valid, from less valid waypoints. Ideally i'd like to build something like that. Also the sources should be from different places, such as OFM/OpenAIP, only stuff which is nowhere else should be maintained here... but experience wise that is wishful thinking.

The cup files are just an export format in the end.

With the current cup file format:

  • the charset is specified ("UTF-8 or Windows-1252") Yes, I implored naviter to define it. That is the 2018 update.

  • the description field is kind of official (i.e. "name, code, country, lat, lon, elev, style, rwydir, rwylen, freq, desc "), although there its format is not described...

Its really the first header for any spreadsheet software. It just serves humans editing the file, the interpreters just discard it.

By the way, do you confirm that:

  • XCSoar has full support for UTF-8? (characters like à, ê, ï ß) I'm not sure. I know that French/German "special" characters are matched by the ascii parent. I.e. ê is matched by pressing 'e' on the virtual keyboard. Full support might be stretch..

  • the header ("name, code, country, lat, lon, elev, style, rwydir, rwylen, freq, desc") should be added?

I currently concatenate all files in this repo and add them to a xcsoar_waypoints.cup which is then used in the mapgen as waypoint source. The concatenate command would need to be changed so that the description line only appears at the top of the file and only once. This can be changed easly, see .travis.yml

lordfolken commented 2 years ago

This has been added