discount scrapping return error

[liranelisha]

scrapping Discount bank successfully login, but then the API return

Error: GENERIC - כרגע לא ניתן להציג את המידע. אפשר לנסות שוב מאוחר יותר או לפנות למוקד התמיכה בטלפון: 03-9439191 ובטלבנק 6111*

[baruchiro]

And when you enter the website as a user, can you see the transactions there?

Maybe they have an issue with their website.

[liranelisha]

confirmed that i enter normally and can view the site as a user from the same machine

[liranelisha]

after further investigation it seems the call to txnsUrls here is causing the issue, due to the date too far into the past, raising the default it to 6months can solve the issue.

for myself i've added startDate to the scrapper option to solve the issue in my implementation

[baruchiro]

@galbarm I will be happy if you can join this discussion.

[baruchiro]

@liranelisha I see the current maximum date is one year before: https://github.com/eshaham/israeli-bank-scrapers/blob/74479a1d9537039098d5f714471d6926da897b45/src/scrapers/discount.ts#L78-L80

Is one year too far? Is the 6 months the most far date we can retrieve?

[liranelisha]

i can do some binary search to find the most far date, though it might to do more to the amount of data returned then to the date... i don't know, and i'm guessing they aren't releasing this information

[baruchiro]

Nevermind, we need to change our approach.

If when browsing as a regular user we receive the data for one year, we need to check what was changed in the website behavior and change it accordingly in our code.

[EzzatQ]

Discount and Mercantile API will return data from 1 year in the past at most. If requesting more than that, you will get the above error message

[liranelisha]

i think the issue is that the default start date is bigger then their max duration, as you can see in the thread above

[EzzatQ]

After testing with Mercantile scraper (uses same system as discount) It looks like the earliest start date that will not return an error is moment().subtract(1, 'years').add(2, 'day').toDate() as opposed to the current implementation moment().subtract(1, 'years').add(1, 'day').toDate()

Changing the default start date to today - 1 year + 2 days would guarantee no error is returned if the scraper is called without an override start date

[github-actions[bot]]

:tada: This issue has been resolved in version 4.4.2 :tada:

The release is available on GitHub release

Your semantic-release bot :package::rocket: