How to resolve the no "idField" assignment warning?

[bmulcahy]

WARNING: requested provider has no "idField" assignment. This can cause errors in ArcGIS clients

What is the correct way to resolve this warning? How should our data be formatted?

[rgwozdz]

This warning stems from the need for ArcGIS clients usually requiring an OBJECTID field. In addition to requiring this field, OBJECTIDs that are not integers or outside the range of 0 - 2,147,483,647 can break features in some ArcGIS clients.

How you resolve this warning depends on your data. If your data already has a field that can serve as a unique ID and is an integer in the range of 0 - 2,147,483,647, you can instruct koop to use it as the OBJECTID by adding idField:<your-field-name> to the geojson metadata set in your provider.

If you don't set idField and don't have a field called OBJECTID in your data, koop will create one using a numeric hash of the feature, scaled to a range of 0 - 2,147,483,647. There is a chance of collisions using this method (perhaps a small one) - hence the warning you are seeing.

[bmulcahy]

Thanks for the explanation. Could we add this to the docs or readme?

[rgwozdz]

@bmulcahy if you go to this page and look in the "Metadata" section, you'll see a similar explanation. I'll admit that it is somewhat buried - we're in the early stages of reorganizing our docs. Hope to publish a new site soon.

[bmulcahy]

Ah my fault I should have looked closer

[simon-fisher]

I am getting this warning, but don't want to assign an idField and just want to let koop create the objectid field. I do see this being created successfully. However I can't seem to query using the objectid. ie. where=OBJECTID=1234567. No features are returned. Is this expceted? Do I need to provide my own objectid field?

[rgwozdz]

No, not expected. I think you have found a bug, but it's with koop-core. Working to fix it now.

[rgwozdz]

@simon-fisher - see https://github.com/koopjs/winnow/issues/115

[rgwozdz]

@simon-fisher - I think I have fixed your noted issue using OBJECTID in the where parameter when those OBJECTIDs are created by Koop. You will need to upgrade your deployment so that Koop uses newly release Winnow v1.16.7.

[simon-fisher]

Sorry I just came back to this now. Even after upgrading winnow I still get the warning and OBJECTID where queries fail. Is there any chance this issue came back? Or do I need to upgrade koop to latest version as well?

[rgwozdz]

I'd upgrade everything first, then check. Please let me know if you still have issues

[simon-fisher]

I upgraded everything to latest, koop at 3.17.1. and same issue appears.

Not sure if this is related, but when searching through for winnow I did notice FeatureServer still uses "winnow": "^1.16.5" https://github.com/koopjs/FeatureServer/blob/master/package.json but elsewhere it references 1.16.11

[rgwozdz]

It's fine that the package uses ^1.16.5; the ^ should ensure the most recent minor release is installed. You can double check by looking in the lock file or the node_modules/winnow file.

I think you may have some aspect of implementation wrong. I just tested this with the socrata provider, and the request http://localhost:8080/koop-provider-socrata/data.ct.gov/y6p2-px98/FeatureServer/0/query?where=OBJECTID=2096926232 returned the expected feature. My advice would be to do the same, i.e., create a fresh Koop instance using the socrata provider and attempt the same request - just so we know we're on the same page. Then I'd ask you to shared your provider code and sample of your geojson; maybe I will spot something.

[simon-fisher]

I have been playing around with this some and it seems I can get some simple queries using OBJECTID to work, as well as the orderByFields works with OBJECTID. Some more complex queries don't work with OBJECTID, including a Sql IN statement.

Example: https://localhostdev/my-provider/client1/FeatureServer/0/query/?id=parcels&where=OBJECTID in (4767223,2083799)&orderByFields=OBJECTID

Does Sql IN work for you?

[rgwozdz]

IN is currently not supported for OBJECTID queries. See winnow. Is there any way you can define your own field for the unique identifier (using idField to point to it)? It's probably the most reliable solution. Otherwise you could try to PR support for IN.

[simon-fisher]

That's actually what I am trying to do now. I added a field named OBJECTID to my data, but I get 2 fields, mine as objectid and the koop generated field as OBJECTID. I guess I need to code or configure koop to use mine correct?

[rgwozdz]

Can you paste a sample of your geojson?

[simon-fisher]

{ "objectIdFieldName": "OBJECTID", "uniqueIdField": { "name": "OBJECTID", "isSystemMaintained": true }, "globalIdFieldName": "", "hasZ": false, "hasM": false, "spatialReference": { "wkid": 4326 }, "fields": [ { "name": "OBJECTID", "type": "esriFieldTypeOID", "alias": "OBJECTID", "sqlType": "sqlTypeInteger", "domain": null, "defaultValue": null }, { "name": "objectid", "type": "esriFieldTypeInteger", "alias": "objectid", "sqlType": "sqlTypeOther", "domain": null, "defaultValue": null }, { "name": "parid", "type": "esriFieldTypeString", "alias": "parid", "sqlType": "sqlTypeOther", "domain": null, "defaultValue": null, "length": 128 }, { "name": "jur", "type": "esriFieldTypeString", "alias": "jur", "sqlType": "sqlTypeOther", "domain": null, "defaultValue": null, "length": 128 }, { "name": "taxyr", "type": "esriFieldTypeInteger", "alias": "taxyr", "sqlType": "sqlTypeOther", "domain": null, "defaultValue": null }, { "name": "gispin", "type": "esriFieldTypeString", "alias": "gispin", "sqlType": "sqlTypeOther", "domain": null, "defaultValue": null, "length": 128 } ], "features": [ { "attributes": { "objectid": 4, "parid": "001001 E00006", "jur": "000", "taxyr": 2008, "gispin": "001001 E00006", "OBJECTID": 343002794 }, "geometry": { "x": 0, "y": 0 } }, { "attributes": { "objectid": 5, "parid": "001001 E00007", "jur": "000", "taxyr": 2008, "gispin": "001001 E00007", "OBJECTID": 739650346 }, "geometry": { "x": 0, "y": 0 } } ], "exceededTransferLimit": false, "geometryType": "esriGeometryPoint" }

[rgwozdz]

Sorry, I mean the geojson that your provider is passing into koop (into the the callback of the getData function); not the Esri JSON that ends up being served by Koop

[simon-fisher]

I figured it out by setting metadata.idField to my OBJECTID. All OBJECTID queries seem to work now including Sql IN. I think I'm all good now.

Thanks so much for your help on this. Simon