Closed sentry-io[bot] closed 1 month ago
The user provided the IDs of alerts they believe are not working. Upon reviewing the queries associated with these alerts, I noticed a common thread: most of the queries include references to the following courts: ny
, nyappdiv
and nyappterm
I will work on this.
All these Sentry issues are for nyappdiv
(nyappdiv_1st, nyappdiv_2nd...). If I find any more I will link them here
On a first look, it seems our scraper server IP is blocked. I can access the forbidden URLs without problem
https://www.nycourts.gov/reporter/slipidx/aidxtable_1.shtml https://www.nycourts.gov/reporter/slipidx/aidxtable_2.shtml https://www.nycourts.gov/reporter/slipidx/aidxtable_3.shtml https://www.nycourts.gov/reporter/slipidx/aidxtable_4.shtml
We indeed have no data since May 17, 2024 for nyappdiv
This happens for most New York courts...
No data for ny
since May 16, 2024. There are 9 decisions from that day to today on the source
No data for nyappterm
since May 16, 2024. There are 12 opinions from that date to today on the source
nytrial
"New York Other Courts" seem to be working fine
@flooie perhaps you can talk with the courts?
Sent a message to this contact form: https://iapps.courts.state.ny.us/webteam/webteam.jsp
I called the New York State Law Reporting Bureau and was transferred to a voicemail - but I did not catch the name - will update the CRM when I hear back. In the meantime - we wait.
the apps.coruts.state.ny.us website is now working -
As an update: we have fresh data for:
nyappdiv. Latest, July 26th
nytrial, latest July 26th
nyappterm, we have data from May - June, and none for July. This may be due to the date range of the query, which we should make smaller https://github.com/freelawproject/juriscraper/blob/a28e884b986b6651393c612adfab6399e6212bb1/juriscraper/opinions/united_states/state/nyappterm_1st.py#L46-L55
ny; unchanged; no data since May. But maybe the scraper is buggy?
Given this, maybe they are simply not blocking us anymore? @flooie
This is still pending to test since our contact in the Court is out on vacation until September 3rd. However, he did clarify something, that the site we target in the ny
scraper (example) will not work even if everything goes OK with the API key. So, we will probably have to re-write that scraper.
If your referring to the https://iapps.courts.state.ny.us/lawReporting/Search and https://www.nycourts.gov/ctapps/Decisions pages your right the api-key would not work from the information provided me from our security team. Only sites the api-key should work on is the lrb.nycourts.gov/* and www.nycourts.gov/reporter/*. Are you finding this to be the case?
What do you think @flooie , there is more detail on the email thread
Changes are working, we got our first ny
opinion since May https://www.courtlistener.com/opinion/10118643/stefanik-v-hochul/?q=court_id%3Any&type=o&order_by=dateFiled+desc&stat_Published=on
While debugging a user's report concerning daily search alerts for records originating from NY courts, I found the following Sentry events:
403 Error: Forbidden for url: https://www.nycourts.gov/reporter/slipidx/aidxtable_1.shtml
403 Error: Forbidden for url: https://www.nycourts.gov/reporter/slipidx/aidxtable_2.shtml
403 Error: Forbidden for url: https://www.nycourts.gov/reporter/slipidx/aidxtable_3.shtml
403 Client Error: Forbidden for url: https://www.nycourts.gov/reporter/slipidx/aidxtable_4.shtml
Filed by: @Erosendo