Implement internal references between attribute names in different OC nodes

[GoogleCodeExporter]

Attributes are very important to support.

Original issue reported on code.google.com by o.di...@arcor.de on 30 Aug 2011 at 11:27

• Blocking: #190

[GoogleCodeExporter]

This issue was updated by revision r760.

Original comment by following09 on 19 Apr 2013 at 11:36

[GoogleCodeExporter]

(docs update in r761)

I have implemented my proposal in your branch. If you do not agree, I can 
revert my changes.

This adds the necessary fields for 1:1 attribute mapping and assumes that only 
1:1 relations are defined. When a 1:many relation is added to attributes.xml, 
attrs/allmeta will throw an exception, and the distinction beween primary and 
secondary attrib relations must be added to attributes.xml etc.

Original comment by following09 on 19 Apr 2013 at 11:43

[GoogleCodeExporter]

>  When a 1:many relation is added to attributes.xml, attrs/allmeta will throw 
an exception,

... and will continue throwing exceptions until the local cached instance of 
attributes.xml is refreshed. This needs to be fixed.

Original comment by following09 on 20 Apr 2013 at 9:29

[GoogleCodeExporter]

> ... and will continue throwing exceptions until the local cached instance of 
attributes.xml is refreshed.

No, it won't. When an exception is thrown while parsing an updated 
attributes.xml, it will not be cached. So the exceptions will cease as soon as 
attributes.xml is repaired in the repo.

Original comment by following09 on 24 Apr 2013 at 9:46

[GoogleCodeExporter]

Your change is effectively prohibiting many-to-many relationships (forcing them 
to be one-to-one). If you want to deny them, what's the point of having them?

My branch was going for this ERD http://i.imgur.com/Uy08rgV.png - I'm not 
saying that it is the best choice, but I thought you were okay with that, but 
your change contradicts this design.

What is the ERD are you aiming at?

Original comment by rygielski on 25 Apr 2013 at 7:34

[GoogleCodeExporter]

> Your change is effectively prohibiting many-to-many relationships 

NO. It is *prepared* for many-to-many relationships, and they can be enabled as 
soon as they are needed, by defining the primary/secondary property in 
attributes.xml.

I did not enable them immediately, because
- you said that you don't know if they will be needed, so it may be wasted time 
to enable them already now,
- I don't know if you can accept my idea of implementing *both*, many:many AND 
the 1:1 "fallback", so it would be wasted time if I do it and then you say "no".

>  your change contradicts this design

No. My change
- retains this design
- allows to give the relations between OKAPI ATTR and OCxx-ATTR an additional 
*property* - let's call it "priority" - which can be "primary" or "secondary" 
(I don't know how relation properties can be expressed in ER model)
- disables many:many as long as this property has not been defined.

Though it is not necessary yet and may be waste of time, I will complete the 
implementation now and define the primary/secondary property in attributes.xml. 
I hope that you can undestand it then and it won't be a 100% waste of time.

Original comment by following09 on 25 Apr 2013 at 9:23

[GoogleCodeExporter]

I don't know if ER model formally allows this, but technically it's possible, 
and this is what I am doing:

http://bit.ly/17jYHyx

Define both many:many and 1:1 relationships, and the user can decide which of 
them to use.

- the <opencaching> elements in attributes.xml define the many:many relation
- the "priority" properties of the <opencaching> elements define the 1:1 
relation
- allmeta.attributes.internal_ids and geocaches.attr_ids expose the many:many 
relation
- allmeta.attributes.primary_internal_id and geocaches.primary_attr_ids expose 
the 1:1 relation

Original comment by following09 on 25 Apr 2013 at 10:49

[GoogleCodeExporter]

(The many:many relations are supersets of the 1:1 relations.)

Original comment by following09 on 25 Apr 2013 at 10:51

[GoogleCodeExporter]

To be precise, it's not 1:1 but 1:1/0 (OKAPI-ATTR : OCxx-ATTR):

Original comment by following09 on 25 Apr 2013 at 11:03

Attachments:

• attribs.png

[GoogleCodeExporter]

This issue was updated by revision r778.

Original comment by following09 on 25 Apr 2013 at 12:07

[GoogleCodeExporter]

This was easier than I thought. ;)

Currently there are only primary relations in attributes.xml, i.e. 
geocaches.attr_ids and geocaches.primary_attr_ids are always the same. If 
multiple local<->global relations are needed for any attribute, the additional 
<opencaching> elements must be tagged with prio > 1, e.g. prio="2".

Original comment by following09 on 25 Apr 2013 at 12:12

[GoogleCodeExporter]

This issue was updated by revision r779.

Original comment by following09 on 25 Apr 2013 at 3:10

[GoogleCodeExporter]

Another option would be to discard allmeta.attributes.primary_internal_id and 
order .internal_ids by prio, i.e. the first one would be the primary.

Original comment by following09 on 25 Apr 2013 at 3:13

[GoogleCodeExporter]

Okay. Let's try to test our schema in practice. I found some use cases
(maybe you can see better ones?). If we cannot find the right answers
for these use cases, then perhaps we should take the simple approach
and leave it for the developers to sort out?

Simple approach = remove the attributes.xml file and expose internal
attributes only, along with their descriptions and categories - all
fed from the database.

=== USE CASE 1 ===

Let's take a look at A70 and A71.

A70 - used in OCPL, meaning: you can take your kids with you, even
really small children, but kids 10-12 yo are cool too. All easy
multi-caches and quizes are marked with this attribute (so that kids
have fun with them).

A71 - used in OCDE, meaning: this is kind of a subset of A70, but it
has almost the same meaning. It is said to be cool specificly for kids 
10-12 yo (not all kids in general).

If someone is *viewing* the cache on OCPL, I guess we should include
A70 only (not both). But. if someone from Germany is *searching* for
A71 on OCPL, should we include caches with A70 too? I guess we should,
what do you think?

If so, then perhaps we should document that all primary attributes
should be used when viewing the cache? And all non-primary attributes 
will be additionally used for searching?

=== USE CASE 2 ===

A53 - used in OCDE - Climbing gear required.
A56 - used in OCPL and OCDE - Special tool required.

Obviously, caches which require climbing gear will be marked with A56 
in OCPL.

If someone is searching for A53 in OCPL, we cannot return caches with 
A56, because A56 is too broad.

But, perhaps, when someone is searching for A56 in OCDE, we should 
return caches with A53? I guess we should (climbing gear is a subset 
of special tools).

In fact, A56 is a superset for many other attributes: A55, A54, A53,
A57... I think that all of these should show up when we're searching
for A56. What do you think?

Original comment by rygielski on 29 Apr 2013 at 9:00

[GoogleCodeExporter]

I would be a pity to discard the whole concept just because of a few attribs 
which we may not be able to handle well. :-(  That would be a lose-lose-lose 
situation for OC nodes, client developers and end users.

A70/A71:

> If someone is *viewing* the cache on OCPL, I guess we should include
A70 only (not both).

Viewing generally includes only those attributes which are present at a certain 
cache - no application displays "placeholdes" for non-properties of a cache. So 
for OCPL caches only A70 can pop up, and for OCDE caches only A71. We do not 
need to make any further provision for this.

>  if someone from Germany is *searching* for A71 on OCPL, should we include 
caches with A70 too?

No, because A71 means that the cache is a challenge for 10-12yo children - many 
of the A70 caches will be just boring for them. I see that with my own 
children, 13 and 15, who are bored of caches which they liked with 8.

So these are to separate attributes, which just fall into the same category. 
It's up to the developers which of them to present for search.

A53/A56:

OCDE attribute 46 means: A tool is required which is no compass, no flashlight, 
no climing gear, no cave equipment, no diving equipment and no boat.
OCPL attribute 83 probably means: A tool is required which is no compass, no 
pen, no shovel, no flashlight and no boat.
GC attrib 51 will be another variation of that.

I think it is ok to map this all to one OKAPI attribute, but it does not make 
sense to directly search for it. But it can be used as search component for "no 
special tool required", which would include the other tool attribs as well.

Overall, I think the best solution would be to distinguish between a "view 
attriute set" and a "search attribute set". The view attribute set is that 
which is currently defined in attributes.xml. 

The search attribute set will consist of logical combinations of view 
attributes:

[S1]
  positive search: "only listed at opencaching" = A1
  negative search: ["also listed at non-opencaching sites" - does not make sense*]

[S2] 
  positive search: "can be done at daylight" = (A39 & ~A43) | A41
  negative search: "cannot be done at daylight" = A43

[S3]
  positive search: "compass required" = A49
  negative search: ["no compass required" - does not make sense*]

[S4]
  positive search: "special tool required" = A49 | A51 | A52 | A53 | A54 | A55 | A56 | A57
    (I omitted A50 here, because a pen is standard equipement for geocachers like a GPS device.)
  negative search: ["no special tool required" - does not make sense*]

*These options do not make sense, because the attributes often are not set. We 
cannot know if a cache is listed somewhere else or if NO compass is required.

Another example: A29/A30

A30 includes A29, as any historic site is a point of interest. So PL-61 may be 
added as secondary local attribute to A30 - but that won't bring any benefit 
for viewing, it is just redundant. And searching may be done via these search 
attribs:

[S5]
  positive search: "historic place" = A29
  negatice search: [does not make sense]

[S6]
  positive search: "point of interest" = A29 | A30
  negative search: [does not make sense]

It looks like this "search attributes" concept makes the many:many relations 
obsolete. This concept is also in my todo list for OC.de; it will replace the 
current "direct attribute search" function.

Original comment by following09 on 29 Apr 2013 at 12:13

[GoogleCodeExporter]

For the "search attribute set", we may discard many attributes, e.g. the 
breeding-season or the dangerous-animals. Both do not make sense when searching 
for geocaches, neither positive nor negative search.

(But we may still provide a direct attribute-search function, which is 
deprecated for general use but may be used for special purposes - like a NP 
organization which wants to check what caches are marked with breeding-season.)

Original comment by following09 on 29 Apr 2013 at 12:20

[GoogleCodeExporter]

I think I like this idea. Indeed, many-to-many relations will not be needed in 
such case. Would you like to try to implement it?

Will the S-attributes also be defined via the XML file? It might be difficult 
to express all possible rules in XML (e.g. "(A39 & ~A43) | A41").

As negative searches often won't make sence, perhaps we should allow only 
positive-searches for S-attributes?

Original comment by rygielski on 29 Apr 2013 at 1:55

[GoogleCodeExporter]

I can try to implement it, but it may take some weeks because there are other 
high-prio issues at OC.de. I will tell when I am ready to start it, maybe you 
can do it earlier. :)

The logical expressions may be replaced by simpler must-have / must-not-have 
lists, which can be fed into the existing algorithm of 
get_common_search_params(). For "can be done at daylight", this would be: Must 
have A39 (24/7) or A41 (not recommended at night), and must not have A43 (only 
at night).

> As negative searches often won't make sence, perhaps we should allow only 
positive-searches for S-attributes?

We could allow only "unary" seaches, yes (which may still include negations of 
the "view attributes", i.e. "no access fee" = ~A26). This would nicely fit the 
OC attributes concept, which knows only one-state attributes.

Original comment by following09 on 29 Apr 2013 at 2:37

[GoogleCodeExporter]

Probably I won't be able to spare much time for OKAPI until I finish urgent 
work-related projects (at least a month). BTW, that's also the reason why I 
answer your emails with a lag - sorry about that.

BTW, I'm not sure, but I think S-attribute conditions might not be "SQL 
friendly". We might need to create an extra table (with aggregated S-values) to 
support attribute searches in decent runtimes. Your idea seems a good one, but 
it will take some time to implement properly. I think it's worth it though.

Original comment by rygielski on 29 Apr 2013 at 3:45

[GoogleCodeExporter]

I will start the implementation now.

Original comment by following09 on 5 Jun 2013 at 5:02

[GoogleCodeExporter]

OK! Good luck, it's a tough one! ;)

BTW, if all goes well, I will be able to spare more time for OKAPI after ~23rd 
of June.

Original comment by rygielski on 5 Jun 2013 at 6:15

[GoogleCodeExporter]

This issue was updated by revision r811.

Original comment by following09 on 6 Jun 2013 at 9:57

[GoogleCodeExporter]

I first did a simple implementation, which allowed attribute definitions like
  S1 = (A1 or A2) and not (A3 or A4)

This could be accomplished with remarkably few lines of searching.inc.php code. 
Nested queries move most of the search logic into SQL code and make it 
performant. Then I decided to pimp it and

- allow AND expressions of included A-Codes, e.g. the A3 here:
    S1 = (A1 or A2) and A3 and not (A4 or A5)

- allow to include cache types in the search attribute definitions, which is 
especially useful for the OC.de cachetype/attribute duplicates like "webcam" 
and "moving"; e.g.
    S1 = (A12 or type-Webcam)

- store the or-terms in human-readable form in attributes.xml.

This did not add much complexity, but inflated the amount of code.

I also considered to allow and-terms for exclusions, e.g.
  S1 = (A1 or A2) and not (A3 and A4)
but that would either add lots of complexity or have performance penalties. And 
instantly I did not find any meaningful example where it would be needed. So I 
discarded that idea.

Now there is a "cache attribute set" (A-codes) and a "search attribute set" 
(S-codes) defined in attributes.xml, which can be separately queried via 
attrs/allmeta. I defined just a few search attribs for demonstration and 
testing, and I did not update yet the comments in attributes.xml.

Regarding the "or-expressions", I considered to pre-compile them in 
attr_helper.inc.php, including translation of OKAPI to local IDs. That would 
make the searching PHP code somewhat more efficient, but it would generate 
complex data structures which would need to be documented for the allmeta 
method. Altogether, I don't think it is worth the effort.

I hope that my implementation is basically ok, and now expect tons of change 
requests from you. ;)

Original comment by following09 on 6 Jun 2013 at 10:26

[GoogleCodeExporter]

Fuck, there is an error in the sql statement which combines attribute and cache 
type conditions. And in further tests, I hit a nesting limit for subqueries.

Now I must rethink the whole thing. :-(

Original comment by following09 on 6 Jun 2013 at 11:55

[GoogleCodeExporter]

I reverted my changes and maybe will give it another shot next week. It looks 
like this cannot be efficiently done by some standard SQL statments. Instead it 
will need an Optimizer:

Either some PHP code which arranges the attribute terms in a way which produce 
a few well-arranged "where cache_id in (select ...)" conditions, or a caching 
table for search attributes which works similar like caches_attributes.

Original comment by following09 on 7 Jun 2013 at 12:41

[GoogleCodeExporter]

Nested queries are usually very inefficient in MySQL. Usually it is *much* 
faster to make N queries than one query with N subqueries.

Original comment by rygielski on 7 Jun 2013 at 7:59

[GoogleCodeExporter]

all cache types need to be defined and implemented properly, so that the 
cachetype option in search attribute is usable without risking exceptions

Original comment by following09 on 13 Jun 2013 at 1:56

• Now blocked on: #220

[GoogleCodeExporter]

The definition and handling of the types not known to all OC sites needs to be 
fixed, but documentation stays a separate issue.

Original comment by following09 on 13 Jun 2013 at 3:50

[GoogleCodeExporter]

> Nested queries are usually very inefficient in MySQL. 

Confirmed. This statement

select * from caches where cache_id in
(select cache_id
from caches_attributes
where attrib_id in (6,38)
group by cache_id 
having count(*)=2)

takes so long on my developer machine that I decided to restart the mysqld. :-O 
When I eliminate the subquery, it takes a few milliseconds.

Original comment by following09 on 13 Jun 2013 at 5:52

[GoogleCodeExporter]

This issue was updated by revision r819.

Original comment by following09 on 13 Jun 2013 at 10:52

[GoogleCodeExporter]

Compared to my first implementation, I did these changes:

- Allowed AND-aggregated exclude ("mustnothave") terms in S-Attribute 
definitions, e.g. "not (A1 and A2)".

- The search attribute definitions are pre-parsed and translated into arrays of 
local IDs after loading attributes.xml. This makes the search attributes 
directionary more complex, but the searching implementation faster and cleaner.

- Lots of SQL optimizations and moving data around in PHP, which would not have 
been necessary if the MySQL subqueries were not broken. :( This has inflated 
the complexity of the search code, and it is more expensive than a sound 
subquery.

I am not happy about discarding the "negative search" concept. There are some 
attributes where both positive and negative search makes sense; e.g. "climbing 
gear required" and "safari cache". Therefore I would like to re-add this 
concept in a way which keeps it simple: Define a naming convention for 
S-Attributes, where attribute -S72 is the complement of attribute S72. It's 
just a documentation change; attributes are defined separately with these IDs 
in attributes.xml. Then application developers can decide to group these two 
into one three-state option. 

Original comment by following09 on 13 Jun 2013 at 11:28

[GoogleCodeExporter]

I am done so far and need some input now if the general concept is ok and on 
the idea of complementary attributes. Then I would go on defining some more 
attributes.

I know that you are out of time right now and will try to be patient. ;)

Original comment by following09 on 14 Jun 2013 at 11:55

[GoogleCodeExporter]

> Therefore I would like to re-add this concept in a way which keeps
> it simple: Define a naming convention for S-Attributes, where
> attribute -S72 is the complement of attribute S72. It's just a
> documentation change; attributes are defined separately with these
> IDs in attributes.xml.

If I was an external developer (unaware of all the complexity of
this), I would expect that positive and negative searches should
always "sum up" to the whole set of all caches. That is, "S1 or not
S1" should evaluate to "true". And such assumption will often be
wrong.

But perhaps you're right. If we explain it properly, it could be a
good thing. It's hard to tell...

Original comment by rygielski on 15 Jun 2013 at 10:04

[GoogleCodeExporter]

Perhaps it's safer to put them in alternative-groups? Most groups will contain 
1 element, some will contain 2 elements (like "climbing gear required" + 
"climbing gear NOT required"), some will contain even more! (like "can be found 
by night", "can be found ONLY by night", "can by found by day", etc.)

We could persuade the developers to allow the users to choose at most one 
option from each alternative-group.

Original comment by rygielski on 15 Jun 2013 at 10:27

[GoogleCodeExporter]

I just had a look at the code. (I won't have time to deeply analyze it
anytime soon...)

1.

I'm quite sure that we should NOT expose the S-code internal logic via
the public allmeta method ("musthave" and "mustnothave" dictionaries).
All of these internal stuff can change (we can re-implement it
differently), so I'd like to keep it private, in order to remain
backward-compatible later. External developers shouldn't need it
anyway.

BTW, perhaps it's better to hardcode S-conditions directly in the 
searching.inc.php file, than to parse them from the attributes.xml 
file? This way we can access all the data we want, instead of being 
limited by our simple XML query language.

2.

I think it will be difficult to keep up the ID sequence between A and
S attributes. There is no reason why we shouldn't name them in
separate sequences, like this http://i.imgur.com/4ppyBv6.png

3.

I think that's a bug. http://i.imgur.com/QjuQZAi.png - if something
fails during refresh_from_string, the lock won't be released, which
can make your site unstable. BTW, in your implementation the lock can
be also released two times (once in the catch block earlier, and once
in the end of the func), the second release might throw exceptions I
guess.

That's all I can tell you right now, sorry. :(

BTW, you have some mixed indentation in some files, e.g.
http://i.imgur.com/yDH6k8I.png, http://i.imgur.com/VhIWCcW.png,
http://i.imgur.com/N9RXMif.png and some others (you can find them with
regexp)

Original comment by rygielski on 15 Jun 2013 at 10:55

[GoogleCodeExporter]

> I would expect that positive and negative searches should
always "sum up" to the whole set of all caches. That is, "S1 or not
S1" should evaluate to "true". And such assumption will often be
wrong.

There are different cases for negative search:

1. Complementary attributes are present on the OC node. Then we can define 
complementary searches which will rarely return wrong data, but never will sum 
up to the whole set, because attributes are often missing in the listings.

2. Only one of the two cases is defined as attribute on the OC node, e.g. 
"climbing gear needed". Then we may define a whole-set complementary search by 
looking for caches which do NOT have this attribute, but which will return many 
wrong hits for the negative case.

ad 1: It is a general problem that attributes are missing in the listings. If 
the user searches for caches which can be done at night, he will get only a few 
of them. (Still, this is a useful search.) So I think that's no argument 
against complementary searches, as the returned cache set is always incomplete.

ad 2: I think we need to decide carefully for each S-attribute if a negative 
search is useful tough it returns wrong hits. E.g. if someone does not like 
climbing-gear caches, it will be useful if he can filter at least some of them 
out. It's the same like the search for terrain ratings: Searching for terrain < 
4 will still return T >= 4 caches which have not been properly rated, but it is 
useful to exclute at least some of them. On the other hand, inverting danger 
attributes is not desirable because each wrong hit will be dangerous.

> Perhaps it's safer to put them in alternative-groups?

Alternative-groups look like a useful feature to me. Currently there are two of 
them: The length of way to the cache, and the steepness of the terrain (both 
incomplete, some alternatives are missing). The time attributes are no 
alternative group, because e.g. "can be found at night" includes "can only be 
found at night". They just fall into the same _category_.

If we give these groups a flag which says if they are logically complete, then 
this would replace my idea of defining complements.

> I'm quite sure that we should NOT expose the S-code internal logic via
the public allmeta method ("musthave" and "mustnothave" dictionaries).

Ok, I have changed this.

> perhaps it's better to hardcode S-conditions directly in the 
searching.inc.php file, than to parse them from the attributes.xml 
file?

Hard-coding has some severe disadvantages. It is harder to understand and to 
maintain, and much harder to optimize than a set of conditions based on the 
same formal rules. Therefore I would rather avoid this, or just enable it as an 
*option* for the very few attributes where it is needed. E.g. the length-of-way 
attributes would benefit from using the trip_distance field.

> I think it will be difficult to keep up the ID sequence between A and
S attributes.

I did not intend so. It is just convenient for the moment when defining the 
S-attributes dictionary. They can be renumbered later.

> I think that's a bug. http://i.imgur.com/QjuQZAi.png - if something
> fails during refresh_from_string, the lock won't be released,

Fixed. Such bugs like to pop up because the concept of throwing exceptions is 
error-prone by design  - I can't understand why it is so popular. You can never 
be sure if a function may throw an exception or not; the whole code gets 
intransparent. Therefore I never use that in my own projects, but pass 
exceptions as (typically self-defined) objects up the call stack, preferably as 
return value of functions, which is indicated by the function name.

Now, it looks like it will take a while until we can finish the attribute 
search functions. What aboult releasing the A-codes as a first step, and then 
continue work on the searching? 

Original comment by following09 on 16 Jun 2013 at 12:07

[GoogleCodeExporter]

> 1. Complementary attributes are present on the OC node.
> 2. Only one of the two cases is defined as attribute on the OC node, 

Replace "on the OC node" by "as A-Codes".

Original comment by following09 on 16 Jun 2013 at 1:40

[GoogleCodeExporter]

> Hard-coding has some severe disadvantages. It is harder to
> understand and to maintain, and much harder to optimize than a set
> of conditions based on the same formal rules.

It depends on how we implement it. I think that's a good use case for
inheritance. If each S-attribute is a class, then 90% of them could be
simple one-liners, e.g.

class S123 extends SimpleSAttribute {
    function getRule() { return "(A13 && A51) || A20"; }
}

This way, all the rule-parsin logic would be implemented inside
SimpleSAttribute class, and all of this logic could be overriden - if
S123 needed some non-standard behavior.

> Now, it looks like it will take a while until we can finish the
> attribute search functions. What aboult releasing the A-codes as a
> first step, and then continue work on the searching? 

I will have some more time soon. Why could try. AFAIR, there were some
issues with A-codes that prompted the discussion about S-codes. I'm
quite sure we can publish A-codes to be viewed, but I don't remember
if we came to a definite conclusion about A-code searching. Do we want
to allow it?

Original comment by rygielski on 19 Jun 2013 at 11:32

[GoogleCodeExporter]

1. You defined n:m relations between OKAPI A-Codes and local attributes to 
allow some primitive OR operations when searching.

2. I insisted that we need 1:1 mapping between local attributes and A-codes for 
some applications and implemented an _additional_ 1:1 mapping - called "primary 
attributes" in addition to the n:m mapping.

3. You did not like and/or not undestand that.

4. I supposed to introduce S-Codes instead which allow combinations of A-Codes 
for searching, so that the n:m mapping between local and A codes gets obsolete.

5. You said that you like that idea.

6. I implemented it.

The amount of time that I can put into this issue is exausted. I will go on 
with other things at OC.de now where I can achive lots of more improvements for 
the users with the same effort.

Original comment by following09 on 19 Jun 2013 at 12:39

[GoogleCodeExporter]

Okay, I'll take it from here. I'll try to divide it into three separate parts:

1. User should be able to get the list of caches' A-codes. Also, there should 
be a method which will expose the names and descriptions of A-codes (e.g. 
allmeta). The mapping between internal attributes and A-codes will be 1:1.

2. Possibly (?) allow searching by A-codes (not S-codes). This has some 
disadvantages, so we may need to discuss that a little more. But let's do that 
after point 1 is published and tested.

3. Implement S-codes.

If all goes well, I will have the time to complete all three by the end of 
July. I think I will publish point 1 by the end of this week.

Original comment by rygielski on 24 Jun 2013 at 10:59

[GoogleCodeExporter]

thumbs-up :-)

1b. Add groundspeak:attributes to GPX files (Issue 190)

Original comment by following09 on 24 Jun 2013 at 2:50

[GoogleCodeExporter]

This issue was updated by revision r829.

Original comment by rygielski on 25 Jun 2013 at 7:02

[GoogleCodeExporter]

I suggest to add

- an option to the attribute_index method which determines if *all* attributes 
are to be returned, or only those which are known on the specific OC site;
- a boolean field to the attribute(s) methods which tells if the attribute is 
known on the OC site.

This will help developers to understand the OC attributes concept and to 
optimize attribute usage. It should be noted then that attributes may be added 
locally, i.e. the flag "is locally known" can change from false to true, but 
not the other way round.

Original comment by following09 on 25 Jun 2013 at 11:53

[GoogleCodeExporter]

I think that the attributes A42 and A43 should be merged to one attribute 
called "night cache", because that is what the description of both tell. A 
merged description would be:

"The Cache has been designed to be searched at night; you may not be able to 
find it during daylight. I.e. there might be some light-reflecting surfaces 
involved which are usually invisible during daylight."

Original comment by following09 on 25 Jun 2013 at 12:12

[GoogleCodeExporter]

> It should be noted then that attributes may be added locally, i.e.
> the flag "is locally known" can change from false to true, but not
> the other way round.

It can change the other way around too. Once an attribute gets
deprecated (or replaced by other attribute), it may disappear from OC
nodes.

> I think that the attributes A42 and A43 should be merged to one
> attribute called "night cache", because that is what the description
> of both tell.

I believe I propesed it earlier, but you insisted they are not exactly 
the same ;) I.e. they have two different GC equivalents.

Original comment by rygielski on 25 Jun 2013 at 12:34

[GoogleCodeExporter]

> I think that the attributes A42 and A43 should be merged to one
> attribute called "night cache", because that is what the description
> of both tell.

Either way, I agree. I will change it later.

Original comment by rygielski on 25 Jun 2013 at 12:40

[GoogleCodeExporter]

> It can change the other way around too. Once an attribute gets deprecated (or 
replaced by other attribute), it may disappear from OC nodes.

They should not disappear from any OC node nor have their meaning substancially 
changed, for the same reason as with OKAPI A-Codes: backward compatibility with 
existing data, which may be externally stored. OCDE always complied to this 
rule; we set a deprecated flag instead (cache_attrib.search_default = false, 
should be renamed).

But for the case someone would do such unsane changes, we can define the flag 
as "is or was known on the local OC node". I think that makes more sense than 
bothering users with disappearing local attribs.

Original comment by following09 on 25 Jun 2013 at 1:32

[GoogleCodeExporter]

I don't see it. Can you give me an example of when it's backwards incompatible?

Original comment by rygielski on 25 Jun 2013 at 2:12

[GoogleCodeExporter]

The way I see it, A-codes can't disappear (to be backward compatible), but 
internal IDs can (since they are not part of any API).

Original comment by rygielski on 25 Jun 2013 at 2:13

[GoogleCodeExporter]

Internal attribute IDs from OCDE they are included in the "XML interface" API 
(http://www.opencaching.de/doc/xml/) and in GPX files. Also there are tools 
which spider listings directly from the website, like Ocprop: 
http://wiki.opencaching.de/index.php/Ocprop. Maybe there are no such things at 
OCPL sites ...

Original comment by following09 on 25 Jun 2013 at 2:25