agrc / palletjack

A library for updating AGOL data from various external sources
https://agrc.github.io/palletjack/palletjack/
MIT License
12 stars 0 forks source link

REST feature layer extractor #48

Closed jacobdadams closed 1 year ago

jacobdadams commented 1 year ago

Adding a new extractor to load data from a feature layer in a REST map or feature service endpoint.

codecov[bot] commented 1 year ago

Codecov Report

Patch coverage: 99.25% and project coverage change: +1.09% :tada:

Comparison is base (8cfe1e9) 90.75% compared to head (5230fa3) 91.85%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #48 +/- ## ========================================== + Coverage 90.75% 91.85% +1.09% ========================================== Files 7 7 Lines 898 1031 +133 Branches 128 146 +18 ========================================== + Hits 815 947 +132 Misses 74 74 - Partials 9 10 +1 ``` | [Files Changed](https://app.codecov.io/gh/agrc/palletjack/pull/48?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=agrc) | Coverage Δ | | |---|---|---| | [src/palletjack/extract.py](https://app.codecov.io/gh/agrc/palletjack/pull/48?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=agrc#diff-c3JjL3BhbGxldGphY2svZXh0cmFjdC5weQ==) | `92.14% <99.22%> (+4.52%)` | :arrow_up: | | [src/palletjack/utils.py](https://app.codecov.io/gh/agrc/palletjack/pull/48?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=agrc#diff-c3JjL3BhbGxldGphY2svdXRpbHMucHk=) | `94.84% <100.00%> (+0.06%)` | :arrow_up: |

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

jacobdadams commented 1 year ago

Ok, I think I've addressed all the different ideas and suggestions (minus creating a properties class...). Do you guys know of a non-OBJECTID service I can test the arbitrarily-named object id stuff against?

stdavis commented 1 year ago

Do you guys know of a non-OBJECTID service I can test the arbitrarily-named object id stuff against?

Here's one that I just put up. Let me know when you are done testing so that I can clean it up. I just download the dataset as a shapefile and then re-uploaded it.

https://services1.arcgis.com/99lidPhWCzftIe9K/ArcGIS/rest/services/Utah_County_Boundaries_TEST/FeatureServer/0

jacobdadams commented 1 year ago

Do you guys know of a non-OBJECTID service I can test the arbitrarily-named object id stuff against?

Here's one that I just put up. Let me know when you are done testing so that I can clean it up. I just download the dataset as a shapefile and then re-uploaded it.

https://services1.arcgis.com/99lidPhWCzftIe9K/ArcGIS/rest/services/Utah_County_Boundaries_TEST/FeatureServer/0

Ok, duh, I could have done that :) But thank you! It works like it should.

jacobdadams commented 1 year ago

Is authenticated access to private layers on the radar?

I hadn't thought about that, but that would be a good idea. I'll add it to the list while I'm rethinking/refactoring.

jacobdadams commented 1 year ago

Ok, I think this is finally ready. I added access to the retry values as global variables that can be set by the client programs in case the default time/retries are not enough for a given application.

jacobdadams commented 1 year ago

@stdavis and @steveoh, I'd love a final look at this and any last thoughts you've got.