Can't run inside github actions

[robinnarsinghranabhat]

I was trying to test a github workflows to scrape floorsheets daily and save to git.

https://github.com/robinnarsinghranabhat/nepse-floorsheet-daily-scrape/blob/master/scrape_nepse.py

But for some reaason, data isn't fetched when requested from github's server.

Other thing I did notice was we could add expose set of apis to make asynchronous request.

[basic-bgnr]

1. Does it work on your local PC and errors out on github ?
2. Can you post the error log

Edit: After going through github action documentation, I found the following limitation on free tier github account.

API requests - You can execute up to 1,000 requests to the GitHub API in an hour across all actions within a repository. If requests are exceeded, additional API calls will fail which might cause jobs to fail.

There are other limitations as well, are you sure you're not hitting them?

https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration

[robinnarsinghranabhat]

ok. its working. Added time.sleep before request. I should probably rotate user-agents and proxies.

[robinnarsinghranabhat]

@basic-bgnr This is so strange. it's coming again.

  time.sleep(random.uniform(4,6)) # Deliberate Throttling
  data = nepse.getFloorSheetOf(symbol=symbol, business_date=date)

This data object is empty When I run this code inside github actions.

https://github.com/robinnarsinghranabhat/nepse-floorsheet-daily-scrape

Repo is pretty straightforward.

[robinnarsinghranabhat]

@basic-bgnr This is so strange. it's coming again.

  time.sleep(random.uniform(4,6)) # Deliberate Throttling
  data = nepse.getFloorSheetOf(symbol=symbol, business_date=date)

This data object is empty When I run this code inside github actions.

https://github.com/robinnarsinghranabhat/nepse-floorsheet-daily-scrape

Repo is pretty straightforward.

Okay. Now I am rotating user-agents. And doing time.sleep of 5 secs for each : data = nepse.getFloorSheetOf(symbol=symbol, business_date=date)

And it looks like it's working now.

[basic-bgnr]

Glad its working 👍