pingles / clj-hector

Simple Cassandra client for Clojure
42 stars 19 forks source link

Add support for composite column types #8

Closed mstump closed 12 years ago

mstump commented 12 years ago

This is in reference to gh-7, adding composite column support to clj-hector.

It's not quite ready to be merged yet, but I wanted to show you the direction I was headed so you could give feedback early on. It's been several years since I've last written Clojure code so any criticisms would be welcome.

The major issue outstanding is that the Component values that are returned as part of a Composite column when performing a query are always HeapByteBuffers instead of the expected type. After this is fixed I think I might be done.

pingles commented 12 years ago

Thanks for all this Matt- looks very interesting.

Apologies I've not taken a look through so far- I've been pretty busy with work but I'm hoping to get a day or two over the next week or so to catch up.

From checking your repository it looks like you've progressed further than these commits?

mstump commented 12 years ago

No problems regarding the delay. I'm not in a huge to push out a new release of clj-hector. I'm pulling in my branch as a leiningen sub project. I just want to be as transparent as possible.

I've completed support for DynamicComposite, and tests for both row fetch, column slice, and column range queries pass tests.

The Composite issue I was running into was most likely a bug in Hector, as other people are seeing it too. I might be able to work around it.

I'm working on keyspace and column family creation with Composite columns, I'm not sure if the issue I'm running into is my bug or Hector's.

I'm going to spend some time working on it today, but starting tomorrow I need to run off and spend time with the relatives so I'm not sure how much progress I'll make.

Adding this feature is taking a little bit longer than I had hoped. I'm paying two penalties, one for coming up to speed on Clojure, and another because Hector isn't particularly well documented.

pingles commented 12 years ago

Hi Matt,

I just merged in a bunch of changes + refactorings that Nick had done- it changes some of the assumptions around using maps for super-columns and sounds like it may make some of the composite stuff easier (https://github.com/pingles/clj-hector/pull/12).

I'll see if I can get an hour or two tomorrow to play around with the changes you've been working on. Thanks again for your work!

mstump commented 12 years ago

Alright, so after a bunch of back and froth about how Composite deserialization is supposed to work on hector-users we finally came to the conclusion that it doesn't which explains the behavior I was seeing.

http://groups.google.com/group/hector-users/browse_thread/thread/ba03b9435689dbbc

As a result I added an option called :c-serializer which allows the user to specify an array of serializers to use for deserializing the Component values of a Composite.

All tests are passing and this pull request is ready for inclusion.