search

rajatomar788 / pywebcopy

Locally saves webpages to your hard disk with images, css, js & links as is.
https://rajatomar788.github.io/pywebcopy/
Other
520 stars 105 forks source link

How to set cookies on a url #126

Closed tadam98s closed 4 months ago

tadam98s commented 4 months ago

For nayone tha needs, the code below adds coockies:

cookie_1 = requests.cookies.create_cookie(domain=url,name='name-1',value='value-1')
cookie_2 = requests.cookies.create_cookie(domain=url,name='name-2',value='value-2')
config = get_config(url, project_folder, project_name=projectName, bypass_robots=True, debug=True, delay=None, threaded=False)
session = config.create_session()
session.cookies.set_cookie(cookie_1)
cookies = requests.utils.dict_from_cookiejar(session.cookies)  # turn cookiejar into dict
session.cookies.set_cookie(cookie_2)
cookies = requests.utils.dict_from_cookiejar(session.cookies)  # turn cookiejar into dict
print(cookies)
crawler = config.create_crawler()
crawler.get(url)
session = crawler.session
session.cookies.set_cookie(cookie_1)
cookies = requests.utils.dict_from_cookiejar(session.cookies)  # turn cookiejar into dict
session.cookies.set_cookie(cookie_2)
cookies = requests.utils.dict_from_cookiejar(session.cookies)  # turn cookiejar into dict
print(cookies)
crawler.save_complete(pop=True)
cookies = requests.utils.dict_from_cookiejar(session.cookies)  # turn cookiejar into dict
print(cookies)
rajatomar788 commented 4 months ago

Hmm. Well done. Should I close this?

tadam98s commented 4 months ago

Yes, info only. -- Mickey Cohen Shanit Ltd. CEO POB 23410, Jerusalem 9123302, Israel M: +972-54-758-6312 Skype: tadam_98 @.***

On April 7, 2024 9:20:13 AM GMT+03:00, Raja Tomar @.***> wrote:

Should I close this?

-- Reply to this email directly or view it on GitHub: https://github.com/rajatomar788/pywebcopy/issues/126#issuecomment-2041332604 You are receiving this because you authored the thread.

Message ID: @.***>

tadam98s commented 4 months ago

no need to publish the following comment: Eventually setting the cookies did not help me to save the site because there is a JavaScript that the site runs to check for presence of cookies and I think that executing the JavaScript could be outside the scope of this application. -- Mickey Cohen Shanit Ltd. CEO POB 23410, Jerusalem 9123302, Israel M: +972-54-758-6312 Skype: tadam_98 @.***

On April 7, 2024 9:20:13 AM GMT+03:00, Raja Tomar @.***> wrote:

Should I close this?

-- Reply to this email directly or view it on GitHub: https://github.com/rajatomar788/pywebcopy/issues/126#issuecomment-2041332604 You are receiving this because you authored the thread.

Message ID: @.***>