search

ClasicRando / web_rest_scraper

Single page static web application to scrape data from an ArcGIS REST service
https://ClasicRando.github.io/web_rest_scraper/
MIT License
1 stars 0 forks source link

Rest Server link does no work #7

Closed ZacZhu0819 closed 2 years ago

ZacZhu0819 commented 2 years ago

The rest server link below does not work and the data can not be scraped https://maps.isgs.illinois.edu/arcgis/rest/services/ILWATER/Water_and_Related_Wells2/MapServer/2

The link was retrieved from the online map: https://prairie-research.maps.arcgis.com/apps/webappviewer/index.html?id=e06b64ae0c814ef3a4e43a191cb57f87

ClasicRando commented 2 years ago

Hi Zac,

The issue you are trying to mention is a server issue since another CLI tool I have also does not work after 20 or so requests have been made. You would either need to change the way you are scraping the data (ie scrape in chunks using a where clause to avoid the server errors) or attempt a more manual approach. Either way, it's not an issue with the webpage but rather the service you are trying to scrape so I will be closing this issue.

ZacZhu0819 commented 2 years ago

Good Morning Steven,

Thank you so much for the information provided!

I will try some other ways and reach out to the database owner if I still can't get the data by scraping.

Thanks again and have a great day.

Best Regards,

Zac Zhu

Research Data Analyst | ERIS

P: 416.510.5204 ext.83503| W: erisinfo.comhttps://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.erisinfo.com%2F&data=05%7C01%7Czzhu%40erisinfo.com%7Ce5a4d9c139d24f13afce08da7b9c1a12%7C5a6f30998a9543e9a941e2c7022c9f11%7C0%7C0%7C637958207660332875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=1edJImHq%2F79IvF0X%2BdV0aFvCCCa%2BO2m5%2Bz1cPygmuAc%3D&reserved=0

@.https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2FERISInformation&data=05%7C01%7Czzhu%40erisinfo.com%7Ce5a4d9c139d24f13afce08da7b9c1a12%7C5a6f30998a9543e9a941e2c7022c9f11%7C0%7C0%7C637958207660332875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=PzsHiLzpWH1AmCBfggtKdu05uqssvVH0NSQgSNq7Cbg%3D&reserved=0 | @.https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fenvironmental-risk-information-services%2F&data=05%7C01%7Czzhu%40erisinfo.com%7Ce5a4d9c139d24f13afce08da7b9c1a12%7C5a6f30998a9543e9a941e2c7022c9f11%7C0%7C0%7C637958207660332875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=pW2zrJUQCXdg3kayIE0p%2FPaiLMjvsH9QJtOw8rYpNYg%3D&reserved=0

[cid:821a810d-dac3-4127-9d6b-d39d16d87466]

From: Steven Thomson @.> Sent: Thursday, September 1, 2022 5:49 PM To: ClasicRando/web_rest_scraper @.> Cc: Zac Zhu @.>; Author @.> Subject: Re: [ClasicRando/web_rest_scraper] Rest Server link does no work (Issue #7)

Hi Zac,

The issue you are trying to mention is a server issue since another CLI tool I have also does not work after 20 or so requests have been made. You would either need to change the way you are scraping the data (ie scrape in chunks using a where clause to avoid the server errors) or attempt a more manual approach. Either way, it's not an issue with the webpage but rather the service you are trying to scrape so I will be closing this issue.

— Reply to this email directly, view it on GitHubhttps://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FClasicRando%2Fweb_rest_scraper%2Fissues%2F7%23issuecomment-1234817055&data=05%7C01%7Czzhu%40erisinfo.com%7C07f31c57e8444a4e211308da8c63cdb8%7C5a6f30998a9543e9a941e2c7022c9f11%7C0%7C0%7C637976657531756336%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=SirGEphEHawA4kbQL5oVB5o6zsJXtpUTGsEc7YRBW1Q%3D&reserved=0, or unsubscribehttps://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FA2TQC77YODE5T45NH77IBY3V4EQFNANCNFSM57BY6QZQ&data=05%7C01%7Czzhu%40erisinfo.com%7C07f31c57e8444a4e211308da8c63cdb8%7C5a6f30998a9543e9a941e2c7022c9f11%7C0%7C0%7C637976657531912523%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=WrAsI4NVjO2hqEgkn04M7T%2FTUARzRVGH1YWt8uxaUxw%3D&reserved=0. You are receiving this because you authored the thread.Message ID: @.***>