Closed kernyan closed 1 year ago
Awesome! This looks fantastic. Only thing is I don't know how to use it now 🙈. Previously I was using a
cookies_headers.py
file described in theREADME.md
. I managed to generate a.json
version using https://www.scrapingbee.com/curl-converter/json/ but it crashes still (once I fixed the problem below) with this:Choice: 11 30 16 Traceback (most recent call last): File "/home/cph/city-strides-hacking/./download_node_csv.py", line 156, in <module> nodes, cache, cache_file = download_nodes_of_city(city, cookies) File "/home/cph/city-strides-hacking/./download_node_csv.py", line 135, in download_nodes_of_city response = requests.get( File "/home/cph/.local/lib/python3.10/site-packages/requests/api.py", line 73, in get return request("get", url, params=params, **kwargs) File "/home/cph/.local/lib/python3.10/site-packages/requests/api.py", line 59, in request return session.request(method=method, url=url, **kwargs) File "/home/cph/.local/lib/python3.10/site-packages/requests/sessions.py", line 573, in request prep = self.prepare_request(req) File "/home/cph/.local/lib/python3.10/site-packages/requests/sessions.py", line 484, in prepare_request p.prepare( File "/home/cph/.local/lib/python3.10/site-packages/requests/models.py", line 370, in prepare self.prepare_cookies(cookies) File "/home/cph/.local/lib/python3.10/site-packages/requests/models.py", line 627, in prepare_cookies cookie_header = get_cookie_header(self._cookies, self) File "/home/cph/.local/lib/python3.10/site-packages/requests/cookies.py", line 147, in get_cookie_header jar.add_cookie_header(r) File "/usr/lib/python3.10/http/cookiejar.py", line 1375, in add_cookie_header attrs = self._cookie_attrs(cookies) File "/usr/lib/python3.10/http/cookiejar.py", line 1334, in _cookie_attrs self.non_word_re.search(cookie.value) and version > 0): TypeError: expected string or bytes-like object
What did you do for it to work?
oh sorry, ya i didn't explain. so i learned that the default header works so i removed it. the cookies.json
only contains these two keys like below
{
"remember_user_token": "...",
"_citystrides_session": "..."
}