OC attributes in GPX files

following5 commented 6 years ago

It has been suggested in https://github.com/opencaching/okapi/pull/511 to add OC attributes to the GPX extensions. This is a complex issue, and I think we may postpone it to a later update of the GPX extension.

OCDE GPX attributes

There already is the OCDE way to store OC attributes in GPX: They are stored as groundspeak:attribute with id set to OC-ID + 100. GS currently uses id 1-66, while OCDE attributes range from 1-62.

I am aware of the following implementations of these OCDE GPX attributes:

The original OCDE implementation.
The OKAPI option gc_ocde:attrs.
OCPL has implemented the "OConly" attribute, id=106
GSAK recognizes the attribute 106 and displays an OConly icon. It stores all of the OCDE attributes and replicates them when re-exporting GPX, but discards the names - just the IDs are exported.

Probably there are also a few tools for Opencaching.de which use those attribs.

Advantages of this method:

Strongest possible promotion of the OC attributes.
Already supported by some applications.
OC Attribs ususually will not get lost when GPX is processed by an application not aware of them.

Disadvantages:

OC attribs can appear in an unusable way in applications.
Groundspeak may add more than 34 new attribs and just overwrite the OC ID space, rendering the OC attribs useless. Hard to say if this will ever happen. Very few GS attribs were added in recent years. We may also try to do an agreement with GS that they skip the 100's. (Probably they will want to skip at least the 106.)

How to proceed?

I am not sure what is best: Adding the OCDE attributes to OCPL, or adding OC attributes to the GPX extension. OCDE may want to keep OCDE GPX attribs for backward compatibility.

If we go for the GPX extension, I would suggest to stick as closely to Groundspeak as possible, so that it's most easy for developers to support the OC attribs:

Add oc:attribute to groundspeak:attributes.
Define it in a way which is backward-compatible to parsers that understand groundspeak:attribute.
Use it only for attributes that cannot be mapped to a groundspeak:attribute.
OKAPI A-Codes may be added as an XML attribute to both groundspeak:attribute and oc:attribute.

wrygiel commented 6 years ago

Use it only for attributes that cannot be mapped to a groundspeak:attribute

It seems safer to output both, I think. Regardless of whether they can be mapped to each other or not.

Note, that it's possible that some A-codes may get mapped to multiple GC-attributes, and vice versa. It's also possible that GC will introduce attributes which will map to A-codes in the future. If you're the author of the app which parses A-codes, how would you proceed? You would probably keep parsing A-codes. So we (OC developers) wouldn't remove these A-codes, despite the fact, that a mapping appeared.

This strategy shouldn't introduce much problems, I believe. Developers could detect if this cache comes from OC (and display only A-codes), or GC (and display only GC-codes).

We can also think about possibly including the English names of A-code attributes directly in the file. I can see both pros and cons for that though...

following5 commented 6 years ago

It seems safer to output both, I think. Regardless of whether they can be mapped to each other or not.

Then it will be a complex task for developers to merge the two attribute lists, because there will be duplicates.

wrygiel commented 6 years ago

That's why above I suggested not to merge them at all.

following5 commented 6 years ago

Attribute sets will need to be merged for global filtering:

gsakfilter

We can also think about possibly including the English names of A-code attributes directly in the file. I can see both pros and cons for that though...

Big pro: It would allow to define oc:attribute backward-compatible to groundspeak:attribute. So any application that can handle GS attributes will easily be updatable to OC attributes.

wrygiel commented 6 years ago

Okay, I understand what you mean.

it will be a complex task for developers to merge the two attribute lists

Note, that in your solution, it still will be equally complex for some developers, in particular, the developers who generate the GPX file (e.g. us). ;) It's okay if we try to do the hard work for someone else, but in this particualar case the work is so complex that this "some else" might be actually mad that we did that work, because he may think that he would have done it better himself.

The main problem is that attributes cannot be mapped "one to one", especially in context of searching. This is a bit problemating even if we limit ourselves to GC attributes only. For example:

If the user is searching for a cache that "doesn't need any additional equipment", then should we automatically exclude the cache that need a "boat"?

This problems grows exponentially with the growing number of attributes. (Do you remember our discussion about S-codes?)

Of course, we could include our "mapping proposal" directly within the GPX file, but:

We should not completely remove the data which the actual end user gave us. Both the user, and the external developer, might be unhappy about us "meddling" with this data, because they would like to "meddle" with it in a different way.
Is our "mapping proposal" mutable? I think it is, at least to some extent (especially if GC introduces new attributes). If so, then wouldn't it be better to ask developers to fetch the mapping directly from OKAPI? Or, even more simply, to read about our proposed mapping on some web page?

wrygiel commented 6 years ago

Let's assign some IDs to all solutions proposed so far:

Solution 1 (misunderstood following5's proposal)

Enforce a mapping, remove mapped A-codes.

<groundspeak:attribute id="11" inc="true">Requires wading</groundspeak:attribute>
<groundspeak:attribute id="25" inc="true">Parking available</groundspeak:attribute>
<oc:attribute a-code="A23">Hilly area<oc:attribute>
<oc:attribute a-code="A29">Historic site</oc:attribute>

Solution 2 (wrygiel - don't embed a mapping at all)

<groundspeak:attribute id="11" inc="true">Requires wading</groundspeak:attribute>
<groundspeak:attribute id="25" inc="true">Parking available</groundspeak:attribute>
<oc:attribute a-code="A22">Requires wading</oc:attribute>
<oc:attribute a-code="A23">Hilly area</oc:attribute>
<oc:attribute a-code="A29">Historic site</oc:attribute>
<oc:attribute a-code="A33">Parking available</oc:attribute>

Solution 3 (wrygiel - embed a mapping, but also keep mapped A-codes)

<groundspeak:attribute id="11" inc="true">Requires wading</groundspeak:attribute>
<groundspeak:attribute id="25" inc="true">Parking available</groundspeak:attribute>
<oc:attribute a-code="A22" gc-equiv-id="11" gc-equiv-inc="true">Requires wading</oc:attribute>
<oc:attribute a-code="A23">Hilly area<oc:attribute>
<oc:attribute a-code="A29">Historic site</oc:attribute>
<oc:attribute a-code="A33" gc-equiv-id="25" gc-equiv-inc="true">Parking available</oc:attribute>

This allows developers to ignore the oc:attributes which have gc-equiv set, if they want to.

wrygiel commented 6 years ago

Solution 4 (wrygiel - don't embed the actual mapping, but inform which were mapped)

<groundspeak:attribute id="11" inc="true">Requires wading</groundspeak:attribute>
<groundspeak:attribute id="25" inc="true">Parking available</groundspeak:attribute>
<oc:attribute a-code="A22" has-gc-equiv="true">Requires wading</oc:attribute>
<oc:attribute a-code="A23">Hilly area<oc:attribute>
<oc:attribute a-code="A29">Historic site</oc:attribute>
<oc:attribute a-code="A33" has-gc-equiv="true">Parking available</oc:attribute>

following5 commented 6 years ago

THIS is what I propose, in GC.com-backward-compatible syntax:

Solution 1.1 (also referred to as Solution 5)

<groundspeak:attribute id="11" inc="1" a-code="A22">Requires wading</groundspeak:attribute>
<groundspeak:attribute id="25" inc="1" a-code="A33">Parking available</groundspeak:attribute>
<oc:attribute id="1023" inc="1" a-code="A23">Hilly area</<oc:attribute>
<oc:attribute id="1029" inc="1" a-code="A29">Historic site</oc:attribute>

Developers just need to add "oc:attribute" to their existing parser, and just need to split off attribs >= 1000 to "oc:attribute" when generating GPX. That's it. I think this is the only way which gives us a slight chance that e.g. GSAK will ever implement it.

(Even this would be the biggest OC support GSAK ever implemented, because they would need to re-layout their filtering window for alle the new attribs, and maybe change database type for attribute IDs from 8-bit to 16-bit.)

following5 commented 6 years ago

On the other hand - it is likely that OCDE will stay backward compatible with the OCDE GPX attribute hack as described in the initial posting. Then my proposal would not be an option for OCDE - but your proposals would fit.

So far, I would even prefer to map the remaining OCPL attributes (besides of ID 6) to 100+ groundspeak:attribute in OCPL code, compatible to the OCDE IDs. This would guarantee the highest possible support by developers.

wrygiel commented 6 years ago

(added an ID to your proposal)

wrygiel commented 6 years ago

I don't understand solution 5. What does "1023" mean? How is it defined? How can developers learn what it means?

following5 commented 6 years ago

That's not solution 5 but the correct solution 1. ;)

The 1023 is just some ID which allows to simply merge OC and GS attribute lists. It happens to be 1000 + (A)23, but must not be defined that way.

How can developers learn what it means?

By a definition table for all OC IDs that we provide in the XSD. But many developers (e.g. GASK) will not need to know what it means, but just handle it by ID and present it by the English string to the user.

But as my proposal would not be backward-compatible to the existing OCDE GPX attributes, it may be useless.

wrygiel commented 6 years ago

Now we have two "solution 1"s. Confusing...

But many developers (e.g. GASK) will not need to know what it means, but just handle it by ID and present it by the English string to the user.

But GSAK (and all other apps) will ignore <oc:attribute> elements anyway. If you want older versions of GSAK to not ignore it, then you would need to use <groundspeak:attribute>.

If you want GSAK to support solution 5, then they need to do additional work. Why do you think that they will do it for solution 5, but not, say, solution 4? It seems to be an equal amount of work.

wrygiel commented 6 years ago

BTW - I forgot to include the English names in Solutions 1-4, but they should be there of course.

wrygiel commented 6 years ago

By a definition table for all OC IDs that we provide in the XSD.

We already expose this list, and even in a parsable format, via attribute_index method (we can evem make it require Level 0 Authentication). Introducing new IDs, and keeping a copy in the XSD - both seem bad ideas.

following5 commented 6 years ago

Regarding my propsal ...

So far, I would even prefer to map the remaining OCPL attributes (besides of ID 6) to 100+ groundspeak:attribute in OCPL code, compatible to the OCDE IDs. This would guarantee the highest possible support by developers.

see https://github.com/opencaching/okapi/issues/346

following5 commented 6 years ago

Now we have two "solution 1"s. Confusing...

Have renamed it to 1.1. ;)

If you want GSAK to support solution 5, then they need to do additional work. Why do you think that they will do it for solution 5, but not, say, solution 4? It seems to be an equal amount of work.

Solution 1.1 would fit into their existing database structure and API, while 2-4 would not. I am not sure if you are proposing string attribute IDs - that's not available in GSAK database and API; but there also is no way to maintain two separate attribute sets. The latter one also does not fit to the GSAK logic and frontend.

It would be a huge step if GSAK adds a boolan field for "requires password" - the first OC field ever in their database. But supporting a set of alternative attributes - I can't imagine that. And I guess it's similar for most other GPX tools.

Another issue is that that we forgot to allow tri-state A-Codes, because the OC sites so far only know boolean attributes. This may change, e.g. OCDE is considering some more negative attributes, and it would make sense to go for tri-state attribs then. Any OC GPX scheme should include support for tri-state attribs. The OCDE GPX attribs and solution 1.1 both do it, they have the "inc" field. The A-Code system would need some extension for that.

following5 commented 6 years ago

A backward compatible way to allow tri-state A-Codes could be: Define some A-code property which allows grouping two A-codes into positve-negative. And taking care that OKAPI never delivers both at once, the positive and the negative attrib. Currently this is possible, because OCDE allows to combine things like "only by day" and "only at night".

wrygiel commented 6 years ago

Can you name some apps which really recognize OCDE's 100+ codes? As far as I understand you, GSAK doesn't really "recognize" them, it only "doesn't remove them" from GPX files (and that's currently the only feature it offers us).

Introducing three-state attributes in OC won't solve the actual problem. There are many other, more complex combinations that don't make much sense when put together, the one you mentioned ("only by day" and "only at night") is only one of the simplest ones.

BTW, there are some apps which already use A-codes (e.g. c:geo). I think that reusing them is the best course of action.

Solution 6 (oc:acode attr + map A-codes to 1000+ GC-codes)

<groundspeak:attribute id="11" inc="true" oc:acode="A22">Requires wading</groundspeak:attribute>
<groundspeak:attribute id="25" inc="true" oc:acode="A33">Parking available</groundspeak:attribute>
<groundspeak:attribute id="1023" inc="true" oc:acode="A23">Hilly area</groundspeak:attribute>
<groundspeak:attribute id="1029" inc="true" oc:acode="A29">Historic site</groundspeak:attribute>

following5 commented 6 years ago

Can you name some apps which really recognize OCDE's 100+ codes?

I am only aware of GSAK recognizing the attribute 106. But as these attribs are out there now for at least 6 years, it is likely that there are more tools that use them. And on the other hand: If these attributes - which are most simple to implement for any Groundspeak-aware tool - are rarely recognized, then the acceptance of anything more different from groundspeak:attribute may even less accepted by developers.

There are many other, more complex combinations that don't make much sense when put together,

Sure. But virtually ALL applications out there can handle tri-state GPX attributes, but none can handle more complex attributes. In theory you are right, but I think we have no chance to change those implementations.

Regarding solution 6 and also 1.1, it just came into my mind what I wrote at #11: This also may break some crude text parsers. Maybe we should create some sample files and let users test them.

following5 commented 6 years ago

Just noticed that OCPL has completely adopted the OCDE GPX attribute schema. @andrixnet implemented that in April 2017. However, there are some overlaps and mismatches to OCDE IDs. Will open an issue for that at OCPL.

When the issues are resolved, https://github.com/opencaching/okapi/issues/346 could be implemented.

andrixnet commented 6 years ago

@wrygiel @following5 please take another look at https://github.com/opencaching/opencaching-pl/issues/1251#issue-263040828

following5 commented 6 years ago

<groundspeak:attribute id="11" inc="true" oc:acode="A22">Requires wading</groundspeak:attribute>

XML validator says: "Attribute 'oc:acode' Is Not Allowed To Appear In Element 'groundspeak:log." (tested against this scheme)

opencaching / gpx-extension-v1