Closed jkerr5 closed 5 years ago
@jkerr5 thanks for posting this and providing the above link. I will put together a PR for 64 bit support in both FeatureServer and Winnow.
Perhaps we could have the OBJECTID always be 64 bit?
That would be fine with me.
@rgwozdz any thoughts on when this might make it into a release?
Hopefully relatively soon. I'm juggling it with a few other things. It's going to require a companion update/release of Winnow and then testing on all the ArcGIS clients, so it's a bit of work to coordinate - unless someone else wants the PR, I probably can't get to it until end of the week.
Thanks for the update @rgwozdz
@jkerr5 - I'm realizing there is an impediment to implementation. JavaScript represents numbers using IEEE-754 double-precision (64 bit) format. It turns out that that give sus only 53 bits precision, which is around fifteen to sixteen decimal digits. When you get to numbers that large or larger, you end up getting approximations:
var bigNumber = 9023372036854775807
console.log(bigNumber); // 9023372036854776000
So while the link you provide indicates ArcGIS support of 64-bit OBJECTIDs, Koop can't readily deliver them due to limitations of JavaScript. A provider could supply a 64-bit integer as a string, but it would have to be converted to an integer prior to serving it to ArcGIS clients (as OBJECTIDs appear to still have to be integers for ArcGIS clients). That conversion could lead to a loss of precision, with the potential result being duplicate OBJECTIDs.
One way forward would be to have FeatureServer set the OBJECTID length to 8 (indicating 64-bit) but print a warning if a provider pushed through an OBJECTID that was greater that the unsigned 53-bit max value (9007199254740991).
How many digits where you hoping to support? Wondering if a max OBJECTID value of 9007199254740991 will suit your needs prior to undertaking the work.
The 53-bit max value of 9007199254740991 would be good. Anything more than the 32-bits we have now would be great :)
On Fri, Oct 12, 2018 at 7:47 PM Rich Gwozdz notifications@github.com wrote:
@jkerr5 https://github.com/jkerr5 - I'm realizing there is an impediment to implementation. JavaScript represents numbers using IEEE-754 double-precision (64 bit) format. It turns out that that give sus only 53 bits precision, which is around fifteen to sixteen decimal digits. When you get to numbers that large or larger, you end up getting approximations:
var bigNumber = 9023372036854775807 console.log(bigNumber); // 9023372036854776000
So while the link you provide indicates ArcGIS support of 64-bit OBJECTIDs, Koop can't readily deliver them due to limitations of JavaScript. A provider could supply a 64-bit integer as a string, but it would have to be converted to an integer prior to serving it to ArcGIS clients (as OBJECTIDs appear to still have to be integers for ArcGIS clients). That conversion could lead to a loss of precision, with the potential result being duplicate OBJECTIDs.
One way forward would be to have FeatureServer set the OBJECTID length to 8 (indicating 64-bit) but print a warning if a provider pushed through an OBJECTID that was greater that the unsigned 53-bit max value (9007199254740991).
How many digits where you hoping to support? Wondering if a max OBJECTID value of 9007199254740991 will suit your needs prior to undertaking the work.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/koopjs/FeatureServer/issues/121#issuecomment-429406195, or mute the thread https://github.com/notifications/unsubscribe-auth/AAP8Ais_yHOr_9otDds9PLHVkz8dCYPhks5ukNXLgaJpZM4Wu-S8 .
Hi @rgwozdz Any update on an ETA for this? We are working a project that needs bigger OBJECTIDs so were hoping this would make it in soon. Thanks.
Hey @jkerr5 - sorry, I've been consumed with a few other big projects so I haven't been able to get to it. I may be freed up starting next week. Sorry, if you guys want to start your own PR, feel free. But it's still in my queue.
@jkerr5 - finally got around to prototyping this, but I am finding that it doesn't work despite the documentation you noted in https://developers.arcgis.com/rest/services-reference/layer-feature-service-.htm.
I find that even when the length of the idField
is set to 8
, values over the 2147483647 max 32bit limit cause buggy behavior in ArcGIS Pro. Specifically, I'm finding that in the attribute table, the idField
is assigned a value of 0 even though koop is sending a value of say 2147483650. Really large numbers appear to cause Pro to crash.
That's unfortunate to hear. I posted a question to the community https://community.esri.com/message/821822-can-arcgis-pro-support-64-bit-objectids
Ideally, we would have an example feature service that implements the objectIdField length property and pumps out 64 bit integer IDs. Then we could analyze the responses, test in clients, and simulate the behavior in Koop.
@jkerr5 - closing this given the response at https://community.esri.com/message/821822-can-arcgis-pro-support-64-bit-objectids
According to https://developers.arcgis.com/rest/services-reference/layer-feature-service-.htm
Can you add support for specifying the length of the ObjectId? Maybe by allowing the
idFieldLength
in the metadata?