obs: filter on fields: Sex, Life Stage, Alive

Thanks to Nudge on #bot-stuff on iNat Discord server for suggesting this:

Using the iNat bot, is there a way to view, say, juvenile observations of a species? (Or sex etc)?

My idea for this is to add something similar to the recently added filtered displays: [p]obs from home by me, a display filtered by observations from the user's home place, and with their own observations listed as the first row. That is, the title of the display should reflect any filter criteria and the rows would further narrow it down by reaction buttons to add either users or places, depending on which kind of embed it is.

It would be nice if there were a short way to enter both field names and values, much like we have abbreviations for place names defined, making possible filters like sex=m or stage=juv. Then an example command might be:

[p]obs sex=m white-cheeked jumping spiders

And the expected output would be an embed with the title: "Genus Pelegrina (Sex: Male)".

For now, we could concentrate on filters for https://api.inaturalist.org/v1/controlled_term/for_taxon and https://api.inaturalist.org/v1/controlled_term, accepting partial matches on names. It's up to the user to know the number of unique characters that will give them the desired result. The first matching term will be used.

So, for example, for Sciuridae (Squirrels), there is only a single "controlled field" returned for the taxon lookup, which is "Sex", and that can have one of two values, "Female" or "Male". Thus, "sex=f", "sex=fem", "sex=female", "s=fem", "s=f" should all be acceptable inputs.

{
    "page": 1,
    "per_page": 30,
    "results": [
        {
            "excepted_taxon_ids": [
                47170,
                333586,
                63081,
                47113
            ],
            "id": 9,
            "is_value": false,
            "label": "Sex",
            "multivalued": false,
            "ontology_uri": "",
            "uri": "",
            "values": [
                {
                    "blocking": false,
                    "id": 10,
                    "label": "Female",
                    "ontology_uri": "",
                    "uri": ""
                },
                {
                    "blocking": false,
                    "id": 11,
                    "label": "Male",
                    "ontology_uri": "",
                    "uri": ""
                }
            ]
        }
    ],
    "total_results": 1
}

Compare the other API call: https://api.inaturalist.org/v1/controlled_terms which doesn't have a taxon_id required parameter. The Life Stage field appears in there.

I'm not sure why, in the specific case of Squirrels, only Sex is returned in the for_taxon query response, and yet the generic /v1/controlled_terms response contains the exact same thing. Perhaps something worth asking about on the iNat forums before starting work on this issue.

Note in the output from both calls there are taxon_ids & excepted_taxon_ids which must be respected so only the values relevant to the taxon being display are available as filters.

For fields observation fields that are not controlled terms, use the old API, e.g. https://www.inaturalist.org/observation_fields.json?q=own+property instead. See allowed_values for the values allowed in the field.

Since there are so many of these, it might be a good idea to allow field names to be defined using abbreviations for them, much in the same way places are defined, so the Discord community can just add a useful subset of them for their own purposes. Without that, the usual rules for handling values with blanks in them apply here, e.g. to find one's own observations with "Observed on own property?": "yes", this might be needed:

[p]obs insects "own property"=y by me

But if we had user-defined fields:

[p]field add ownprop 11722

The expected output for that would be an embed confirming addition of ownprop as an abbreviation for "Observation field: Observed on own Property?" with the URL https://www.inaturalist.org/observation_fields/11722 (shows details for the property as well as matching observations).

And then the user could type this simpler query:

[p]obs insects ownprop=y by me

As for controlled terms, I got a little further by asking on the forums. pleary gave me the answer we need to make it work: https://forum.inaturalist.org/t/how-do-the-v1-controlled-terms-calls-work/13089

Just specify all ancestor_ids as a comma-delimited list for taxon_id= and then only take the values that have one of the taxon's ancestor_ids in its taxon_ids, and doesn't have any of the taxon's ancestor_ids in its excepted_from_taxon_ids.

For example, for Ephemeroptera (48011), to determine the Life Stage (controlled terms id: 1) values:

https://api.inaturalist.org/v1/controlled_terms/for_taxon?taxon_id=1,47120,372739,47158,184884,48011

Adult included: matches 1 in taxon_ids & has no exceptions
Teneral excluded: is only for Odonata (47792)
Pupa excluded: is for [47208,48763,49369,47157,47822,47201,62164]; set intersection with [1,47120,372739,47158,184884,48011] (48011's ancestor_ids) is empty
Nymph included: [47793,81769,56834,47744,47792,47651,47504,48011,48112,53795,47198,83187,83201,83200] includes 48011
Larva excluded: only for [20978,47208,47822,47201,47157,49369,47864,48763,47794,83204,83202,62164]
Egg included: [43233,47120,47114,47459,85497,47178,20978,3,49231,47273,60450,49099,26036] includes 47120
Juvenile excluded: is for Animalia (1) but excepted_from_taxon_ids is [184884,47504], i.e. Pterygota (184884) excludes it
Subimago included: is for 48011

This is substantially complete, and has been for a while, via the with qualifier, e.g. ,obs with sex female, or ,obs with s f (abbreviations are allowed). There are some other good ideas in here (principally, lookup on user-defined fields) but those should be filed as a separate wishlist, since this one just covers the controlled terms.

dronefly-garden / dronefly

obs: filter on fields: Sex, Life Stage, Alive #86