Async selections interfere with each other

[flipper44]

I am tracing through a large network, I need to select 100s-1000s of features across multiple layers using an in statement on each with a list of ids. So in order to avoid delays I call them async with a promise when they have processed..it seems the persist=true and append=true causes them to contaminate each other and I get the proper Jon results from all the calls, but the selection layer only shows a mixed bag of items selected..selection overview also shows that the majority of features on certain layers are no longer there

[flipper44]

PS I meant 100s-1000s of features on each layer...think about a small electric utilities entire grid

[jumpinjackie]

Do you wait for the selection response to arrive first before starting the next batch of ids?

[flipper44]

No..Way too long 35 seconds vs async 6secs for huge traces

[flipper44]

Guess I assumed append would work with multiple requests running...Mapguide must reselect for each request..I can run the same trace and get different results. Problem is things like Primary and secondary lines and meters are big selections while transformers and fuses and others are smaller, so the get to fly through while the others are chugging away when it's async.

[jumpinjackie]

I think this is a MapGuide-level problem. I don't think concurrent writing to session resources is thread safe.

[flipper44]

I'm guessing append and persist don't just make a running queue, it must do something odd like grab existing then append and send back...does that make sense?

[jumpinjackie]

I think it's because at the moment you save the appended selection back to the session repository, another async request might have started appending from a pre-append selection state. MapGuide would not know anything about this.

If you wait for the selection response first, that is at least acknowledgment from the server that the next batch is "safe" to append to the current selection.

[flipper44]

I guess that's what I was trying to say..seems like it grabs current selection appends then replaces the old selection with the new instead of just leaving a static selection object that it can queue into through multiple threads..which is odd because copying and replacing per request makes little sense..that explains why I only get results from 3 or 4 layers of 12 returned in the overview..I wonder if I can run it without persist or append and just convert the json response to a selectionXML object and send it back again and at least save some time? My guess is that the SelectionXML will be just as slow for the same reasons

[jumpinjackie]

I'm not quite comfortable with modifying mapguide-rest to cater to this one particular scenario, but here's something you could try.

Have a .net/Java/PHP helper that takes your list of IDs and spits out the selection XML for that particular set of ids.

Fire off your batches of async requests to this helper script and with each fragment you get back stash in a temp array. Once all your batches complete, assemble the final selection XML from the collected fragments and then do the big selection update with persist/append=true

[jumpinjackie]

Another thought just entered my mind as I was writing the previous comment:

Is the "select" done by user interaction (user clicks or box selects on a map) or a FDO query?

If it's a FDO query, I imagine that you are using some attribute/spatial filter to get this list?

If that's the case, I could bake something into mapguide-rest to do this easily. It'd be a new representation for features source: selection XML!

So a request like:

GET /library/path/to/your.FeatureSource/features.selectionxml?filter=<some filter>

And it returns a the selection XML for the matching features. Then you could hammer this endpoint in parallel (this operation would not mutate the current selection) and assemble the final selection XML from the returned fragments to do the big selection update at the end.

[flipper44]

After a trace has been run on the data I have JSON structure telling it what to select/display on the map that has been traced. Here is a very short trace results example: {
"SelectionMethod":"TRACE", "ID":"Trace", "SelectionObjects":[
{
"LayerName":"FUSE", "LayerField":"gisid", "LayerServiceID":"5bd743e0-146c-4f3a-a443-6e2b560c5f61", "FieldValues":"'F00864'", "TYPE":null }, {
"LayerName":"METER", "LayerField":"gisid", "LayerServiceID":"5bd743e0-146c-4f3a-a443-6e2b560c5f61", "FieldValues":"'M00482','M00483','M00461'", "TYPE":null }, {
"LayerName":"PRIMARY LINE", "LayerField":"gisid", "LayerServiceID":"5bd743e0-146c-4f3a-a443-6e2b560c5f61", "FieldValues":"'P00370','P00368','P00367','P00369','P00366','P00365'", "TYPE":null }, {
"LayerName":"SECONDARY LINES", "LayerField":"gisid", "LayerServiceID":"5bd743e0-146c-4f3a-a443-6e2b560c5f61", "FieldValues":"'S04642','S04641','S04643'", "TYPE":null }, {
"LayerName":"TRANSFORMER", "LayerField":"gisid", "LayerServiceID":"5bd743e0-146c-4f3a-a443-6e2b560c5f61", "FieldValues":"'T03246','T03244','T03245','T03259'", "TYPE":null } ], "IgnoreSmartReporting":true, "Recenter":true, "ZoomExtents":true, "ZoomBuffer":"80%" }

[flipper44]

I was able to get around this using:

<SelectionUpdate>
    <Layer>
        <Name>METER</Name>
        <SelectionFilter>gisid IN ('M02356','M02357','M02355')</SelectionFilter>
    </Layer>
    <Layer>
        <Name>PRIMARY LINE</Name>
        <SelectionFilter>gisid IN ('P02906','P02904','P02905','P02903')</SelectionFilter>
    </Layer>
    <Layer>
        <Name>SECONDARY LINES</Name>
        <SelectionFilter>gisid IN ('S02615','S02614','S02616')</SelectionFilter>
    </Layer>
    <Layer>
        <Name>TRANSFORMER</Name>
        <SelectionFilter>gisid IN ('T01917','T01916','T01918')</SelectionFilter>
    </Layer>
</SelectionUpdate>

It doesn't zip through like Async but it is faster. Jackie, I am on an older version of REST, can you validate that this works in the newest version as well?

append:"true",
featurefilter:"<SelectionUpdate><Layer><Name>METER</Name><SelectionFilter>gisid IN ('M02356','M02357','M02355')</SelectionFilter></Layer><Layer><Name>PRIMARY LINE</Name><SelectionFilter>gisid IN ('P02906','P02904','P02905','P02903')</SelectionFilter></Layer><Layer><Name>SECONDARY LINES</Name><SelectionFilter>gisid IN ('S02615','S02614','S02616')</SelectionFilter></Layer><Layer><Name>TRANSFORMER</Name><SelectionFilter>gisid IN ('T01917','T01916','T01918')</SelectionFilter></Layer></SelectionUpdate>",
format:"json",
geometry:null,
layerattributefilter:"2",
layernames:null,
maxfeatures:-1,
persist:"true",
requestdata:0,
selectionvariant:"INTERSECTS",
selectionxml:"false",

[jumpinjackie]

Should be fine

[flipper44]

This does Not work in 2.5.2 Unfortunately! So I am now checking the version and if it is less than 2.6 I run it synchronously. However I get this weird Result when trying to append. Request USING /session/{session}/{mapName}.Selection BODY: {"geometry":null,"selectionvariant":"INTERSECTS","persist":"true","layerattributefilter":"2","requestdata":0,"featurefilter":"<SelectionUpdate><Layer><Name>AMR</Name><SelectionFilter>gisid IN ('A00002')</SelectionFilter></Layer></SelectionUpdate>","selectionxml":"false","format":"json","append":"true","layernames":"AMR"} RESULT: {"d":"{\"FeatureInformation\":{\"@xmlns:xsi\":\"http://www.w3.org/2001/XMLSchema-instance\",\"FeatureSet\":{\"@xsi:noNamespaceSchemaLocation\":\"FeatureSet-1.0.0.xsd\",\"Layer\":[{\"@id\":\"e20d2202-5b66-11e7-8001-68b599521206\",\"Class\":{\"@id\":\"dbo:amr\",\"ID\":[\"AgAAAA==\"]}}]},\"Tooltip\":null,\"Hyperlink\":null}}"} THEN I APPEND AGAIN a query from another Layer: BODY: {"geometry":null,"selectionvariant":"INTERSECTS","persist":"true","layerattributefilter":"2","requestdata":0,"featurefilter":"<SelectionUpdate><Layer><Name>FUSE</Name><SelectionFilter>gisid IN ('F00817','F00808','F00818','F00819','F00820','F00823','F00804','F00805','F00828','F00824','F00795','F00796','F00798','F00825','F00826','F00797','F00829','F00806','F00827','F00800','F00837','F00838','F00841','F00842','F00832','F00833','F00835','F00831','F00830')</SelectionFilter></Layer></SelectionUpdate>","selectionxml":"false","format":"json","append":"true","layernames":"FUSE"} RESULT: {"d":"{\"FeatureInformation\":{\"@xmlns:xsi\":\"http://www.w3.org/2001/XMLSchema-instance\",\"FeatureSet\":{\"@xsi:noNamespaceSchemaLocation\":\"FeatureSet-1.0.0.xsd\",\"Layer\":[{\"@id\":\"e20cacd2-5b66-11e7-8002-68b599521206\",\"Class\":{\"@id\":\"dbo:fuse\",\"ID\":[\"AgAAAA==\"]}},{\"@id\":\"e20d2202-5b66-11e7-8001-68b599521206\",\"Class\":{\"@id\":\"dbo:amr\",\"ID\":[\"AgAAAA==\"]}}]},\"Tooltip\":null,\"Hyperlink\":null}}"} WHY DOES it get the same ID in Both and Not actually Add features and Append? Did I set something wrong in the Selection Body?

[jumpinjackie]

Ok that definitely looks like a bug now. I'll take a look when I have time.