search

eshaham / israeli-bank-scrapers

Provide scrapers for all major Israeli banks and credit card companies
MIT License
591 stars 160 forks source link

amex: timeout navigating to login page #878

Closed chaosite closed 1 month ago

chaosite commented 2 months ago

Hello, I'm using this library through moneyman, and recently it's failing to scrape Amex cards.

Strangely, it's scraping Isracard cards just fine; I thought it's basically the same code?

These are logs from a recent scrape:

2024-08-10T16:12:06.855Z moneyman:data scraping account #2 (type=amex)
2024-08-10T16:12:06.855Z moneyman:scrape started
2024-08-10T16:12:06.856Z moneyman:scrape ***amex*** START_SCRAPING
2024-08-10T16:12:06.856Z moneyman:scrape ***amex*** INITIALIZING
2024-08-10T16:12:06.856Z israeli-bank-scrapers:base-scraper-with-browser initialize scraper
2024-08-10T16:12:06.856Z moneyman:scrape ***amex*** INITIALIZING
2024-08-10T16:12:06.856Z israeli-bank-scrapers:base-scraper-with-browser launch a browser with headless mode = true
2024-08-10T16:12:07.159Z israeli-bank-scrapers:base-scraper-with-browser browser has already pages open, use the first one
2024-08-10T16:12:07.159Z israeli-bank-scrapers:base-scraper-with-browser set viewport to width 1024, height 768
2024-08-10T16:12:07.162Z israeli-bank-scrapers:base-isracard-amex navigate to login page
2024-08-10T16:12:37.180Z israeli-bank-scrapers:base-scraper-with-browser terminating browser with success = false
2024-08-10T16:12:37.180Z moneyman:scrape ***amex*** TERMINATING
2024-08-10T16:12:37.185Z israeli-bank-scrapers:base-scraper-with-browser Request failed: net::ERR_ABORTED https://he.americanexpress.co.il/personalarea/Login
2024-08-10T16:12:37.242Z moneyman:scrape ***amex*** END_SCRAPING
2024-08-10T16:12:37.242Z moneyman:scrape error: GENERIC Navigation timeout of 30000 ms exceeded
2024-08-10T16:12:37.242Z moneyman:scrape ended
2024-08-10T16:12:37.242Z moneyman:data scraping took 30.4s
2024-08-10T16:12:37.242Z moneyman:data scraping account #2 ended
chaosite commented 2 months ago

I'm running moneyman on GitHub, could Amex be blocking GH's IP range or something? Is anyone else seeing this?

daniel-hauser commented 2 months ago

Forget to open an issue here, both amex and isracard blocked traffic from the US since ~20/07. I contacted the customer service and provided details. on the 06/08 isracard fixed the issue, but amex is still blocked.

daniel-hauser commented 1 month ago

Another update: I'm still trying to convince the customer service that they have a problem 🤦and it's not an issue with my isp.

daniel-hauser commented 1 month ago

Update: Amex is now working for me

chaosite commented 1 month ago

Confirming that Amex is working again. Thank you very much!