Tooltip - Need to display multiple Areas of Operation

[tonysecurityforcemonitor]

@sam-ffctn - Currently the tooltip only lists one Area of Operation - even if the Organization has multiple areas of operation listed in its dossier - example: http://ffctn.com/clients/on/sfm/M19/#country/ng/o/358c956b-29ca-4dc4-b1ff-847d040d0fbc!date=2014-01-01&focus=map

Can we have the tooltip list all the Areas of Operation for an Organization with each Area comma separated?

[sam-ffctn]

@tonysecurityforcemonitor @evz

For the tooltip I display the data from the Map endpoint: ex: https://sfm.datamade.us/api/countries/ng/map/?tolerance=0.01&at=2014-01-01

here I can find the following for each organization:

"area_current": {
"name": "Bauchi",
"geometry": {GeoJSON},
"id": "0230abb5-a4b2-4b7c-877e-d0dacc2d19c8",
"last_cited": "2012-02-01",
"osm_name": "Bauchi",
"osmid": 3722233,
"first_cited": "2004-08-30"
}

the name property is what I display in the tooltip. I'm a bit confused as to why this name isn't representative of all of the areas related to the organization. Is this a bug or am I missing something? I thought area_current was some kind of union between all the areas of an organization.

Ideally that is how it would behave (unless I'm misunderstanding something).

[tonysecurityforcemonitor]

@sam-ffctn @evz I'm not sure if this is connected or not - but when I filter through not all of the areas of operation are showing - for instance the various Army organizations all have enough data to cover the entire map of Nigeria with various AOOs, however when I filter to only display the Army I get this: http://ffctn.com/clients/on/sfm/M20/#country/ng!date=2014-01-01&classifications=Army&focus=map

As compared to Police organizations: http://ffctn.com/clients/on/sfm/M20/#country/ng!date=2014-01-01&classifications=Police&focus=map

[tonysecurityforcemonitor]

Just following up on this one - not sure where the issue lays @sam-ffctn @evz ...

[sam-ffctn]

@tonysecurityforcemonitor I display all the area data I get from the API, If there is missing some additional data than I think the ball lies in @evz's court.

[evz]

@tonysecurityforcemonitor So .... it looks like part of the problem with not showing all the areas is that I am using the date for the Site to do the initial filtering. That means if the organization was at a given site on the date, it'll return that but we're not checking if they were in a given area on the date. Should I check both? It's just a matter of adding an OR condition on the end of the query that also checks the Area table.

[tonysecurityforcemonitor]

@evz Yes please add an OR -- there isn't always a neat match between a Site and Area (the normalization issue grows even more complex!) so I think having an OR should adjust for that.

[evz]

OK, I added the OR to the end of that query. It still looks like filtering by "Army" is not covering the entire country but, upon inspection, it's only returning 30 unique areas of operation (for 43 organizations) when you are looking at January 1, 2014. I'm not sure if that is what you expect but comparing that to the "Police" filter where you get 108 organizations in 76 unique areas makes me think there might not actually be full coverage for that filter.

[tonysecurityforcemonitor]

@evz Hmmm that's strange - for instance the 1 Mechanized Division should have an Area that covers several states from about 2000 onwards. I just cleaned the Master Import Sheet - there were some issues with dates due to Google Sheets formatting years like 2012 into 7/12/1905. Perhaps reloading the data will fix this issue? The Army should have the entire map covered from at least 2008 onwards.

[evz]

Oh, so what you're saying is that an organization's area of operations for a given date is the union of all the area shapes that are valid areas of operation for that date? So, in the case of the 1 Mechanized Division, it should be everything in the areas of operation list on their detail page (since all of those areas don't have an end date)?

If that is the case, I can certainly implement that kind of a union. Since, in order to create the correct shape for any given date we'll want to do the union-ing on the fly, this might slow things down a bit. I'll see what I can do to make it faster. At the very least, I'm caching the responses on the server side so once we've made the response once, we won't have to create it over again.

[evz]

Oh, so, with that in mind, what is an organization's current site? Should there only ever be one on a given date?

[tonysecurityforcemonitor]

Oh, so, with that in mind, what is an organization's current site? Should there only ever be one on a given date?

An organization can have multiple Sites on the same date

[tonysecurityforcemonitor]

Oh, so what you're saying is that an organization's area of operations for a given date is the union of all the area shapes that are valid areas of operation for that date? So, in the case of the 1 Mechanized Division, it should be everything in the areas of operation list on their detail page (since all of those areas don't have an end date)?

I see the Area as more of a list - so with the tooltip we have each Area seperated by a comma idealy. This is because Areas can be non-contigious and also change over time.

Displaying Area on the map doesn't seem to working as I'd expect.

On the map on 1 Jan 2014 the only Area displayed for the 1 Mechanized Division Area is Bauchi - but the Division also has Jigawa, Kaduna, and other Areas which aren't displayed.

Additionally - the 1 Mechnized Division had 2000 as the date_of_first_citation for Bauchi and the Area had N for Assume_to_current_date? so I'd expect any date before 1 Jan 2000 or after 31 Dec 2000 would not have Bauchi displayed as an Area for the 1 Division.

If the functionality is better achived through viewing Areas as an union then that works for me. The one question I would have is how the tooltip and dossier would seperate out the various Areas from one another (as they have different confidence scores, sources, dates).

[evz]

So, @sam-ffctn it seems like, in light of what @tonysecurityforcemonitor is talking about above, the right thing to do would be to switch the area_current and site_current portions of those responses to lists of objects. I was trying to figure out how to do it by just aggregating the geographies into one MultiPolygon (or Multipoint) but the problem is that we then lose the ability to be able to tell the difference between the different geographies (so, all parts would have the same name or the same list of names or whatever).

If I change those two properties to lists, does that cause tremendous, breaking changes for you? Is there a better way of doing that? What if I were to leave the response as a single object but just aggregated the properties of that object into lists of things. So, right now those properties look like:

{
...
    "area_current": {
    "osm_name": "Degema",
    "name": "Degema",
    "osmid": 3720751,
    "id": "03a4b135-fb9b-49f7-a96c-2a985c8ee577",
    "last_cited": null,
    "first_cited": "2006-10-02",
    "geometry": { ... }
    }
... 
}

I could change them to just aggregate the various properties (including the geometries):

area_current": {
    "id": "002d6aff-44ea-42a0-bb50-2f62d8c20a4b",
    "geometry": {
        "geometries": [ ... ],
        "type": "GeometryCollection"
    },
    "first_cited": [
        "2009-01-23",
        "2009-01-23",
        "2009-01-23",
        "2009-03-11",
        "2006-12-13"
    ],
    "last_cited": [
        null,
        null,
        null,
        null,
        null
    ],
    "osm_name": [
        "Apatzingán",
        "Buenavista",
        "Ziracuaretiro",
        "Tancítaro",
        "Coalcomán de Vázquez Pallares"
    ],
    "osmid": [
        5605517,
        5605596,
        5606676,
        5606427,
        5605673
    ]
}

All the properties are listed in the same order so unpacking them would be possible.

Another option would be to just make a list of the kind of objects that we have currently. Any thoughts? I think all three of these would be about as simple to implement. Oh, and, whatever we do here is going to effect the same property on the /organizations/:id/chart/ endpoint (since it also has area_current and site_current properties)

[evz]

Oh, that'll also effect the /countries/:id/search/organizations/ endpoint.

[sam-ffctn]

@tonysecurityforcemonitor @evz The easier solution for me would be to union all the geometry and provide a concatenated string for all the names of the areas. I don't think we really need to differentiate between the different areas, since we would want to highlight/select all the areas of an organization simultaneously, never individually.

[evz]

@sam-ffctn That sounds good. There are a couple kinds of unions that I could do. One will physically combine all of the shapes into one shape. This can be somewhat computationally expensive and would erase the boundaries in between shapes if they physically touch one another. Another option would be to just offer a kind of collection of all the shapes (or points) that preserves the boundaries between the shapes and is much faster because it's just a simpler operation. My preference would be to offer the latter mostly because the shapes are, under the hood, distinct and it'd mean the performance on the server side would not take too much of a hit. @tonysecurityforcemonitor Do you have a preference?

[tonysecurityforcemonitor]

@evz I don't have an informed preference - I think having shapes be less resource intensive for the app is always a good thing.

[evz]

OK, I'll try the "collection" method first and we'll go from there.

[evz]

@sam-ffctn I just pushed up a change that implements that "collection" approach. Let me know how that works.

[sam-ffctn]

@tonysecurityforcemonitor @evz It was a bit more painful than I thought it would be to fix the map for the new collections, but it's working now.

http://ffctn.com/clients/on/sfm/M22/#country/ng!date=2014-01-01

[tonysecurityforcemonitor]

Sorry for the pain @sam-ffctn - but the tooltip is working as expected!