koopjs / koop

Transform, query, and download geospatial data on the web.
http://koopjs.github.io
Other
671 stars 128 forks source link

duplicate features rendering #347

Closed mapsgeek closed 5 years ago

mapsgeek commented 5 years ago

image

the endpoint http://koopserver.herokuapp.com/file-geojson/rest/services/country/FeatureServer

the map on arcgis online http://mapsgeek.maps.arcgis.com/home/webmap/viewer.html?webmap=da1d36c8661d476fbd68718d74788168

not sure what's wrong but also can we make koop compatible with esri-leaflet ? there's an issue currently with the fitbounds function meanwhile the feature service from arcgis online works just fine with it, not sure if should go to a different issue but just asking

rgwozdz commented 5 years ago

@mapsgeek - can you make the webmap public. It's asking me to login.

Can you give more context? Have you seen the duplicate features in the request list or in the response bodies? Or is it just the darker rendering of certain countries on the map that makes you suspect duplicate feature rendering? Have you made any customizations to the provider?

Please add esri-leaflet as a separate issue and I will try to take a look.

mapsgeek commented 5 years ago

@rgwozdz my bad http://arcg.is/SG0OO, actually i couldn't figure out from the requests list since the queries are quite long to follow but it's the darker colour is what i'm suspecting specially if the layer has many features, this doesn't seem to be happening on other provided samples, i haven't test with my own data yet just doing prototyping. i haven't customized the provider by any means i'm using koop as middleware in nestjs framework

i will be providing more details for the esri-leaflet issue

rgwozdz commented 5 years ago

@mapsgeek - Thanks. I am noticing that the duplicate rendering is intermittant. On some zoom levels, the problem is visible; on others all is normal.

It looks like you are relying on Koop to generate your OBJECTIDs. The first thing I would do is assign your own object IDs either in the data source or in the provider (via 'idField', see https://koopjs.github.io/docs/usage/provider#setting-provider-metadata-in-getdata). Koop can generate the OBJECTIDs for you, but as the console warnings note, it can sometimes lead to unpredictable results (such as this) in ArcGIS clients, and isn't recommended for production.

It might be something else, but we need to eliminate the OBJECTID issue first.

mapsgeek commented 5 years ago

thank you for assistants honestly i thought it's something i can just ignore, my bad i will try what you suggested and let you know.

mapsgeek commented 5 years ago

well i updated the country.geojson file with added field of idField but the behavior still the same and my best guesses for what produces this is that when you pan the map so the bounding box changes and send fetching a query but at the same time there's an intersection area which was being fetched by a previous query before the pan even happens !.

rgwozdz commented 5 years ago

@mapsgeek - I've taken another look at this. I don't think you have idField set properly in your provider. When I look at the payloads of the AGO requests to your Koop I see the fields array:

  "fields": [
    {
      "name": "OBJECTID",
      "type": "esriFieldTypeOID",
      "alias": "OBJECTID",
      "sqlType": "sqlTypeInteger",
      "domain": null,
      "defaultValue": null
    },
    {
      "name": "idField",
      "type": "esriFieldTypeInteger",
      "alias": "idField",
      "sqlType": "sqlTypeOther",
      "domain": null,
      "defaultValue": null
    },
  ...
  ...
  ],

Notice that you have both idField and OBJECTID and that OBJECTID has been assigned type esriFieldTypeOID. That means Koop is still creating and OBJECTID and AGO is still using it as the primary key for the data.

Moreover, the requests to Koop for each tile area are bringing back the same country feature, but with a different on-the-fly generated OBJECTID (this is why we recommend setting idField in GeoJSON metadata). This explains why you are seeing duplicate features. Since AGO is getting different OBJECTIDs for the same country it doesn't think they are the same and so it renders both.

From what I can tell, you have added an idField to your raw data. However to use that as your data's primary key, you also have to flag it in your GeoJSON metadata in the provider. In the getData function of your provider you would have to do:

geojson.metadata.idField = 'idField'.

(Note that the idField doesn't need to be named idField; this is just ArcGIS's way of figuring out which property to treat as the primary key)

A different way to fix this is editing your GeoJSON; rename all the idField keys as OBJECTID. Koop will see that the data already has an OBJECTID property and preferentially use that.

Hope this helps.

mapsgeek commented 5 years ago

Thank you for your patience it works now just fine, i renamed the idField in my raw geojson to OBJECTID since i honestly don't understand how to set the geojson.metadata.idField = 'idField' from my backend code in away other than taking the provider folder from node_modules to the project source and edited the geojson.metadata.idField = 'idField' in there and referred to it in my provider import instead of referring to @koopjs/..., however that didn't seem to work right but anyway it works anyway with just renaming the field in the geojson, i will take it !