koopjs / koop-provider-elasticsearch

A provider for koop that can connect to one or more elastic search instances and turn indices/aliases into individual feature services.
Apache License 2.0
13 stars 4 forks source link

Data access issue in webmap opening attribute table #59

Closed AndrewGriffinGIT closed 1 year ago

AndrewGriffinGIT commented 1 year ago

Opening attribute table creates 500 internal server error. Postman get for the query FeatureServer/0/query?f=json&returnIdsOnly=true&where=(DT_FIELD > timestamp '2023-01-02 05:59:59') AND (1=1)&returnGeometry=false&spatialRel=esriSpatialRelIntersects&outFields=OBJECTID&outSR=102100 returns {    "error": "Cannot read properties of undefined (reading 'OBJECTID')"}

If using this FeatureServer/0/query?f=json&returnIdsOnly=true&where=(DT_FIELD > timestamp '2023-01-02 05:59:59') AND (1=1)&returnGeometry=false&spatialRel=esriSpatialRelIntersects&outFields=OBJECTID

the query works.

why is outSR=102100 causing OBJECTID error

ohh through some quotes around outSR='102100' works too, or drop it entirely also works

AndrewGriffinGIT commented 1 year ago

MicrosoftTeams-image (1)

rgwozdz commented 1 year ago

@AndrewGriffinGIT - thanks for reaching out. Which ArcGIS client is being used here?

AndrewGriffinGIT commented 1 year ago

Enterprise 10.9.1

From: Rich Gwozdz @.> Sent: Wednesday, February 1, 2023 9:35 AM To: koopjs/koop-provider-elasticsearch @.> Cc: Griffin, Andrew @.>; Mention @.> Subject: [EXT]:Re: [koopjs/koop-provider-elasticsearch] Data access issue in webmap opening attribute table (Issue #59)

EXTERNAL EMAIL. Do not click or open attachments unless you recognize the sender and know the content is safe.


@AndrewGriffinGIThttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_AndrewGriffinGIT&d=DwMCaQ&c=nGJ3DvCCJ_qjA9N0MZvrjA&r=_GAp2WNc-b3y5GEA9_s2mPPRZ4i6HpzVZAmYFrHFXOA&m=FuNYvkqDS0BI2ino4FNke6zFsnduWe2puQrMT3J2q5Iolz1vqyaELe8FZvwwcZPT&s=JYBX5ho9IrBQlEMivY5_mLlleSO63vCCVkhKvRRyErU&e= - thanks for reaching out. Which ArcGIS client is being used here?

— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_koopjs_koop-2Dprovider-2Delasticsearch_issues_59-23issuecomment-2D1412260795&d=DwMCaQ&c=nGJ3DvCCJ_qjA9N0MZvrjA&r=_GAp2WNc-b3y5GEA9_s2mPPRZ4i6HpzVZAmYFrHFXOA&m=FuNYvkqDS0BI2ino4FNke6zFsnduWe2puQrMT3J2q5Iolz1vqyaELe8FZvwwcZPT&s=4Xr8Y4h8iNYhWzg5DVAngcHYP-R13A_sRx4nuMgcAL4&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AZAS6O5RH6NBJEJY75OPSGDWVJ7AZANCNFSM6AAAAAAUNEDZNY&d=DwMCaQ&c=nGJ3DvCCJ_qjA9N0MZvrjA&r=_GAp2WNc-b3y5GEA9_s2mPPRZ4i6HpzVZAmYFrHFXOA&m=FuNYvkqDS0BI2ino4FNke6zFsnduWe2puQrMT3J2q5Iolz1vqyaELe8FZvwwcZPT&s=DYVJMDPOt_1Lory9UenbtNVNS6Yao8nRmoYpb1ZiOEQ&e=. You are receiving this because you were mentioned.Message ID: @.**@.>>


This email communication is intended as a private communication for the sole use of the primary addressee and those individuals listed for copies in the original message. The information contained in this email is private and confidential and if you are not an intended recipient you are hereby notified that copying, forwarding or other dissemination or distribution of this communication by any means is prohibited. If you are not specifically authorized to receive this email and if you believe that you received it in error please notify the original sender immediately. We honor similar requests relating to the privacy of email communications

rgwozdz commented 1 year ago

Can you provide the version number of Koop that you are using to serve the data? As of now, I'm not totally clear where this is coming from. Given that it has something to do with the outSR parameter (which asks that geometry be returned in a specified CS), I would have guessed the source of the issue is within the Geoservices output plugin or one of its dependencies. But with a different data provider, I can't reproduce. However I am using the latest Koop packages.

AndrewGriffinGIT commented 1 year ago

"dependencies": { @./elasticsearch": "^7.13.0", @./koop-core": "^7.1.0", @./logger": "^2.0.6", @./provider-elasticsearch": "^3.4.1", @./geojson-rewind": "^0.5.0", @./tilebelt": "^1.0.2", "flat": "^5.0.0", "h3-js": "^3.7.2", "koop": "^4.2.2", "moment": "2.20.1", "ngeohash": "^0.6.3", "node-cmd": "^5.0.0", "polygon-splitter": "0.0.8", "proj4": "2.4.4", "sqlite-parser": "1.0.1" }, "repository": { "type": "git", "url": "git+https://github.com/koopjs/koop-provider-elasticsearch.git" }, "devDependencies": { @.***/cli": "^1.1.2", "gh-release": "^6.0.0" } }

From: Rich Gwozdz @.> Sent: Wednesday, February 1, 2023 9:49 AM To: koopjs/koop-provider-elasticsearch @.> Cc: Griffin, Andrew @.>; Mention @.> Subject: [EXT]:Re: [koopjs/koop-provider-elasticsearch] Data access issue in webmap opening attribute table (Issue #59)

EXTERNAL EMAIL. Do not click or open attachments unless you recognize the sender and know the content is safe.


Can you provide the version number of Koop that you are using to serve the data? As of now, I'm not totally clear where this is coming from. Given that it has something to do with the outSR parameter (which asks that geometry be returned in a specified CS), I would have guessed the source of the issue is within the Geoservices output plugin or one of its dependencies. But with a different data provider, I can't reproduce. However I am using the latest Koop packages.

— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_koopjs_koop-2Dprovider-2Delasticsearch_issues_59-23issuecomment-2D1412282377&d=DwMCaQ&c=nGJ3DvCCJ_qjA9N0MZvrjA&r=_GAp2WNc-b3y5GEA9_s2mPPRZ4i6HpzVZAmYFrHFXOA&m=rXvcZMOh_yJTloOe59dtvDZJBO9UEFfoVnBBmXKLWemBlPP4kAoDNMotmqyD0OMT&s=GQpbLutfmVdJ6pHmjH6Ltn88MuZvUws6N3CfHrnm3XY&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AZAS6OZIS3M4N6B32GA2Q6DWVKAWVANCNFSM6AAAAAAUNEDZNY&d=DwMCaQ&c=nGJ3DvCCJ_qjA9N0MZvrjA&r=_GAp2WNc-b3y5GEA9_s2mPPRZ4i6HpzVZAmYFrHFXOA&m=rXvcZMOh_yJTloOe59dtvDZJBO9UEFfoVnBBmXKLWemBlPP4kAoDNMotmqyD0OMT&s=xhA5WgCDmH0PabZGsLgaJJMfHiyqSGjo1uZsw-lPG4U&e=. You are receiving this because you were mentioned.Message ID: @.**@.>>


This email communication is intended as a private communication for the sole use of the primary addressee and those individuals listed for copies in the original message. The information contained in this email is private and confidential and if you are not an intended recipient you are hereby notified that copying, forwarding or other dissemination or distribution of this communication by any means is prohibited. If you are not specifically authorized to receive this email and if you believe that you received it in error please notify the original sender immediately. We honor similar requests relating to the privacy of email communications

AndrewGriffinGIT commented 1 year ago

Also using Danny Hatcher code. Do you have a github repo where I can just configure my elasticsearch instance and get koop to serve an esri feature service

From: Rich Gwozdz @.> Sent: Wednesday, February 1, 2023 9:49 AM To: koopjs/koop-provider-elasticsearch @.> Cc: Griffin, Andrew @.>; Mention @.> Subject: [EXT]:Re: [koopjs/koop-provider-elasticsearch] Data access issue in webmap opening attribute table (Issue #59)

EXTERNAL EMAIL. Do not click or open attachments unless you recognize the sender and know the content is safe.


Can you provide the version number of Koop that you are using to serve the data? As of now, I'm not totally clear where this is coming from. Given that it has something to do with the outSR parameter (which asks that geometry be returned in a specified CS), I would have guessed the source of the issue is within the Geoservices output plugin or one of its dependencies. But with a different data provider, I can't reproduce. However I am using the latest Koop packages.

— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_koopjs_koop-2Dprovider-2Delasticsearch_issues_59-23issuecomment-2D1412282377&d=DwMCaQ&c=nGJ3DvCCJ_qjA9N0MZvrjA&r=_GAp2WNc-b3y5GEA9_s2mPPRZ4i6HpzVZAmYFrHFXOA&m=rXvcZMOh_yJTloOe59dtvDZJBO9UEFfoVnBBmXKLWemBlPP4kAoDNMotmqyD0OMT&s=GQpbLutfmVdJ6pHmjH6Ltn88MuZvUws6N3CfHrnm3XY&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AZAS6OZIS3M4N6B32GA2Q6DWVKAWVANCNFSM6AAAAAAUNEDZNY&d=DwMCaQ&c=nGJ3DvCCJ_qjA9N0MZvrjA&r=_GAp2WNc-b3y5GEA9_s2mPPRZ4i6HpzVZAmYFrHFXOA&m=rXvcZMOh_yJTloOe59dtvDZJBO9UEFfoVnBBmXKLWemBlPP4kAoDNMotmqyD0OMT&s=xhA5WgCDmH0PabZGsLgaJJMfHiyqSGjo1uZsw-lPG4U&e=. You are receiving this because you were mentioned.Message ID: @.**@.>>


This email communication is intended as a private communication for the sole use of the primary addressee and those individuals listed for copies in the original message. The information contained in this email is private and confidential and if you are not an intended recipient you are hereby notified that copying, forwarding or other dissemination or distribution of this communication by any means is prohibited. If you are not specifically authorized to receive this email and if you believe that you received it in error please notify the original sender immediately. We honor similar requests relating to the privacy of email communications

rgwozdz commented 1 year ago

@dhatcher - can you see if you can reproduce this error?

AndrewGriffinGIT commented 1 year ago

Oddly attribute tables opens in ArcGIS Pro. Only webmaps in portal are creating error when opening attribute table for service

From: Rich Gwozdz @.> Sent: Wednesday, February 1, 2023 1:32 PM To: koopjs/koop-provider-elasticsearch @.> Cc: Griffin, Andrew @.>; Mention @.> Subject: [EXT]:Re: [koopjs/koop-provider-elasticsearch] Data access issue in webmap opening attribute table (Issue #59)

EXTERNAL EMAIL. Do not click or open attachments unless you recognize the sender and know the content is safe.


@dhatcherhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_dhatcher&d=DwMCaQ&c=nGJ3DvCCJ_qjA9N0MZvrjA&r=_GAp2WNc-b3y5GEA9_s2mPPRZ4i6HpzVZAmYFrHFXOA&m=HafFCJK__5HkTwxfKAjZtNdv4jebohlgdmv0IgSIsbecg_m5CHFMUh65Vnscj5lp&s=hipnwvn_fezLlP6uD4XNZveQXAf8BxYFsic6CIjbruI&e= - can you see if you can reproduce this error?

— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_koopjs_koop-2Dprovider-2Delasticsearch_issues_59-23issuecomment-2D1412610617&d=DwMCaQ&c=nGJ3DvCCJ_qjA9N0MZvrjA&r=_GAp2WNc-b3y5GEA9_s2mPPRZ4i6HpzVZAmYFrHFXOA&m=HafFCJK__5HkTwxfKAjZtNdv4jebohlgdmv0IgSIsbecg_m5CHFMUh65Vnscj5lp&s=i0qAB0hnBHw0_6h3vQxoKexKxcNFoPE-_CQW7DlfoO0&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AZAS6O6EP35WXDWK76LWIXLWVK2ZRANCNFSM6AAAAAAUNEDZNY&d=DwMCaQ&c=nGJ3DvCCJ_qjA9N0MZvrjA&r=_GAp2WNc-b3y5GEA9_s2mPPRZ4i6HpzVZAmYFrHFXOA&m=HafFCJK__5HkTwxfKAjZtNdv4jebohlgdmv0IgSIsbecg_m5CHFMUh65Vnscj5lp&s=0hRQ3XChN2NJxcCu8ksouTuL4PwhXX-bTqLC0SwImMs&e=. You are receiving this because you were mentioned.Message ID: @.**@.>>


This email communication is intended as a private communication for the sole use of the primary addressee and those individuals listed for copies in the original message. The information contained in this email is private and confidential and if you are not an intended recipient you are hereby notified that copying, forwarding or other dissemination or distribution of this communication by any means is prohibited. If you are not specifically authorized to receive this email and if you believe that you received it in error please notify the original sender immediately. We honor similar requests relating to the privacy of email communications

AndrewGriffinGIT commented 1 year ago

Request URL: https://myserver.mydomain.com:3000/myapp/es/rest/services/myes/myservice/FeatureServer/0/query?f=json&returnIdsOnly=true&where=(DT_TIMEFIELD%20%3E%20timestamp%20%272023-01-02%2005%3A59%3A59%27)%20AND%20(1%3D1)&returnGeometry=false&spatialRel=esriSpatialRelIntersects&outFields=OBJECTID&outSR=102100 Request Method: GET Status Code: 500 Internal Server Error Remote Address: myipadd:3000 Referrer Policy: strict-origin-when-cross-origin

From: Rich Gwozdz @.> Sent: Wednesday, February 1, 2023 1:32 PM To: koopjs/koop-provider-elasticsearch @.> Cc: Griffin, Andrew @.>; Mention @.> Subject: [EXT]:Re: [koopjs/koop-provider-elasticsearch] Data access issue in webmap opening attribute table (Issue #59)

EXTERNAL EMAIL. Do not click or open attachments unless you recognize the sender and know the content is safe.


@dhatcherhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_dhatcher&d=DwMCaQ&c=nGJ3DvCCJ_qjA9N0MZvrjA&r=_GAp2WNc-b3y5GEA9_s2mPPRZ4i6HpzVZAmYFrHFXOA&m=HafFCJK__5HkTwxfKAjZtNdv4jebohlgdmv0IgSIsbecg_m5CHFMUh65Vnscj5lp&s=hipnwvn_fezLlP6uD4XNZveQXAf8BxYFsic6CIjbruI&e= - can you see if you can reproduce this error?

— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_koopjs_koop-2Dprovider-2Delasticsearch_issues_59-23issuecomment-2D1412610617&d=DwMCaQ&c=nGJ3DvCCJ_qjA9N0MZvrjA&r=_GAp2WNc-b3y5GEA9_s2mPPRZ4i6HpzVZAmYFrHFXOA&m=HafFCJK__5HkTwxfKAjZtNdv4jebohlgdmv0IgSIsbecg_m5CHFMUh65Vnscj5lp&s=i0qAB0hnBHw0_6h3vQxoKexKxcNFoPE-_CQW7DlfoO0&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AZAS6O6EP35WXDWK76LWIXLWVK2ZRANCNFSM6AAAAAAUNEDZNY&d=DwMCaQ&c=nGJ3DvCCJ_qjA9N0MZvrjA&r=_GAp2WNc-b3y5GEA9_s2mPPRZ4i6HpzVZAmYFrHFXOA&m=HafFCJK__5HkTwxfKAjZtNdv4jebohlgdmv0IgSIsbecg_m5CHFMUh65Vnscj5lp&s=0hRQ3XChN2NJxcCu8ksouTuL4PwhXX-bTqLC0SwImMs&e=. You are receiving this because you were mentioned.Message ID: @.**@.>>


This email communication is intended as a private communication for the sole use of the primary addressee and those individuals listed for copies in the original message. The information contained in this email is private and confidential and if you are not an intended recipient you are hereby notified that copying, forwarding or other dissemination or distribution of this communication by any means is prohibited. If you are not specifically authorized to receive this email and if you believe that you received it in error please notify the original sender immediately. We honor similar requests relating to the privacy of email communications

rgwozdz commented 1 year ago

@AndrewGriffinGIT - I'm going to publish an update to Koop later today/tonight that will allow you to see the stack trace if you are in debug mode. That will allow us to pinpoint the error. Stay tuned.

AndrewGriffinGIT commented 1 year ago

So new map viewer no problems, classic map viewer in portal is the issue with opening attribute table

rgwozdz commented 1 year ago

@AndrewGriffinGIT thanks for the update. If you want to debug this, update to the latest version of koop (7.1.2) and set your koop log-level to "debug". You can do this quickly without any code changes by setting an env variable LOG_LEVEL=debug. Fire off the request again and look at your server log and you should see an error with stack trace.

AndrewGriffinGIT commented 1 year ago

Seems this resolved the issues with classic map viewer attribute table

npm uninstall koop

npm install @koopjs/koop-core

From: Rich Gwozdz @.> Sent: Wednesday, February 1, 2023 5:19 PM To: koopjs/koop-provider-elasticsearch @.> Cc: Griffin, Andrew @.>; Mention @.> Subject: [EXT]:Re: [koopjs/koop-provider-elasticsearch] Data access issue in webmap opening attribute table (Issue #59)

EXTERNAL EMAIL. Do not click or open attachments unless you recognize the sender and know the content is safe.


@AndrewGriffinGIThttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_AndrewGriffinGIT&d=DwMCaQ&c=nGJ3DvCCJ_qjA9N0MZvrjA&r=_GAp2WNc-b3y5GEA9_s2mPPRZ4i6HpzVZAmYFrHFXOA&m=wetqoF0YJObItLEQfUJW2IZe0uMrR_QV6MObaKSEtO20ixHakIaZ04SkuUYQx1on&s=2JfG37kxWfiEtjr4emhyLWxuO_7t52TURI3L685FPIs&e= thanks for the update. If you want to debug this, update to the latest version of koop (7.1.2) and set your koop log-level to "debug". You can do this quickly without any code changes by setting an env variable LOG_LEVEL=debug. Fire off the request again and look at your server log and you should see an error with stack trace.

— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_koopjs_koop-2Dprovider-2Delasticsearch_issues_59-23issuecomment-2D1412893295&d=DwMCaQ&c=nGJ3DvCCJ_qjA9N0MZvrjA&r=_GAp2WNc-b3y5GEA9_s2mPPRZ4i6HpzVZAmYFrHFXOA&m=wetqoF0YJObItLEQfUJW2IZe0uMrR_QV6MObaKSEtO20ixHakIaZ04SkuUYQx1on&s=RDtxCtBFpbW2kZO1fV_wrEor4-LCo3UndM9xuXc2-iU&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AZAS6OZGPIZWROZYOPD52OLWVLVPDANCNFSM6AAAAAAUNEDZNY&d=DwMCaQ&c=nGJ3DvCCJ_qjA9N0MZvrjA&r=_GAp2WNc-b3y5GEA9_s2mPPRZ4i6HpzVZAmYFrHFXOA&m=wetqoF0YJObItLEQfUJW2IZe0uMrR_QV6MObaKSEtO20ixHakIaZ04SkuUYQx1on&s=-o6JTb3VIc38YSn8Tuy-leiQD5arIYAralTHrmcqMl8&e=. You are receiving this because you were mentioned.Message ID: @.**@.>>


This email communication is intended as a private communication for the sole use of the primary addressee and those individuals listed for copies in the original message. The information contained in this email is private and confidential and if you are not an intended recipient you are hereby notified that copying, forwarding or other dissemination or distribution of this communication by any means is prohibited. If you are not specifically authorized to receive this email and if you believe that you received it in error please notify the original sender immediately. We honor similar requests relating to the privacy of email communications

rgwozdz commented 1 year ago

Great. Thank you for the update.