GoogleNews on EC2

[claudia0juarez]

I have been running this script in AWS EC2 through a virtual environment running on python34 and coming back empty; when I run it locally in PyCharm I am getting a result, do you know why is this the case?

import requests from bs4 import BeautifulSoup from GoogleNews import GoogleNews

googlenews = GoogleNews() googlenews.search('forbes') url = googlenews.getlinks() name = googlenews.result() print(url[0])

[HurinHu]

The problem is, google knows you are using a cloud server through IP, and it will block the IP as it is recognized as a robot.

[claudia0juarez]

Thanks! Do you know a way to bypass this error?

[mjlabe]

I have mine running on Digital Ocean. Maybe they just block AWS?

On Sun, Feb 23, 2020, 11:13 PM Claudia notifications@github.com wrote:

Thanks! Do you know a way to bypass this error?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/HurinHu/GoogleNews/issues/15?email_source=notifications&email_token=AG3DIIKEYAZROWIJDD4BIBDRENCOPA5CNFSM4K2A632KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMWSOVI#issuecomment-590161749, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG3DIILPSFC5QHM2LA6HMQTRENCOPANCNFSM4K2A632A .

[HurinHu]

If it is not necessary to run on server, you can run it locally and pass the results to the server rather than running on the server directly. And don’t call the method too frequently, it also may cause this issue.

[claudia0juarez]

Thanks mjlabe/Hurin, I do need it to run over EC2 as it needs to run daily on a cron job

[claudia0juarez]

BTW I tried running it in GoogleCloud same results