Closed ghost closed 6 months ago
Hi, thanks for the contribution. Everything looks good except for the session request, having a session is not actually an standard. having a session on the requests could make your bot vulnerable to be blocked becuase it's easy identifiable by the server throught the cookies, if no session is stablished the server won't be able identify the requests are from a bot(this could be done for example checking when was the last time the bot made a requests, if it's too fast, block the bot) Sessions are usefull were cookies are strictly required, like for example the website requires login, or some websites requires an initial cookie to authenticate subsequent requests
Makes sense. I have removed the prepere_session()
method and pushed the code.
Thank you very much Sir
Changes :
utils.prepare_session()
to prepare a session before sending request to the website. It is not going to make huge difference, but a standard practice to maintain same session. I would recommend starting session inSearch_all
as it is making multiple requests.dict.get(key, default)
for fetching data from dict instead ofdict[key]
. Latter might break in absence of the key..gitignore
file