search

scrapinghub / frontera

A scalable frontier for web crawlers
BSD 3-Clause "New" or "Revised" License
1.29k stars 216 forks source link

Frontera Request/Response decoding/encoding doesnt work between python 2/3 versions #293

Open whalebot-helmsman opened 7 years ago

whalebot-helmsman commented 7 years ago

There is an error when you encode Request or Response in one python (e.g. python2) and decode it in another(e.g. python3).

How to reproduce? All information in one gist

I split tests/test_codecs.py in three parts - common.py(request shared between read and write for comparison), read.py(requests read from msgpack and compare with requests from common) and write.py(write to msgpack requests from common)

if I write/read in same python, all is work. But if switch between versions error appears. Look at attached to gist write-2-read-3.txt and write-3-read2.txt files.

sibiryakov commented 6 years ago

Thank you for the report. Could track down where the problem is, and submit a PR?