search

twitter-archive / kestrel

simple, distributed message queue system (inactive)
http://twitter.github.io/kestrel
Other
2.77k stars 312 forks source link

Kestrel ignores memcache flags #4

Closed elephantum closed 14 years ago

elephantum commented 15 years ago

Module python-memcached uses memcache ability to add meta-data (flags) to store type of data, and uses it for parsing.

For example it sets flags=0x1 to say that data is pickled and should be unpickled.

Kestrel silently drops flags so meta-data is lost which leads us to unexpected behavior: we write python tuple "(1000, 100)", we get "(I1000 I100 tp1 ." as a result.

As far as I understand Kestrel code, to achieve this functionality we should:

  1. add .flags member to QItem
  2. move instantiation of QItem from PersistentQueue.add to KestrelHandler.set
  3. use item.flags in formatting response in KestrelHandler.get

If these are acceptable changes I'm going to code them and present patch for review shortly.

robey commented 15 years ago

that sounds fine to me. you will also need to add a new variant of ADD in the journal -- maybe ADD_FLAGS. and just always write ADD_FLAGS but still be able to read ADD and ADDX.

robey commented 14 years ago

send me the patch and i'll merge it. :)