davidsantiago
commented
13 years ago Yeah, I've been meaning to get on top of this 1.3 stuff. I'll try to get something going later tonight and hopefully can get something solid this weekend.
- David
Closed seancorfield closed 13 years ago
davidsantiago
commented
13 years ago Yeah, I've been meaning to get on top of this 1.3 stuff. I'll try to get something going later tonight and hopefully can get something solid this weekend.
seancorfield
commented
13 years ago Thanx David. Feel free to ping me if you want someone to test stuff or lend a hand!
davidsantiago
commented
13 years ago Sean, I took a stab at this, it's in the feature/clojure-1.3 branch. I also uploaded it to clojars as 1.3.0-SNAPSHOT. Let me know how that works for you.
David
seancorfield
commented
13 years ago It looks good so far David: I have accounting files from a Power MTA server that I need to analyze - they're about 200MB each and so far it seems to be powering thru them on Clojure 1.3.0! Thank you!
davidsantiago
commented
13 years ago Great. I'll probably make a 1.3.0 release on monday or tuesday if you still haven't found a problem.
David
seancorfield
commented
13 years ago I've run this on all the PMTA accounting files we have and it seems rock solid. Using pmap to apply our DB lookup process gets thru each 200MB file in about 60 seconds so I'm very pleased with that. Thank you!!
davidsantiago
commented
13 years ago Very glad to hear it. Out of curiosity, what kind of performance were you getting before, for comparison? I ran some tests while trying to add some of the performance improvements and put the results here: https://github.com/davidsantiago/clojure-csv/blob/feature/clojure-1.3/bench.clj.txt
I've come to realize that it could probably get a lot faster still if I changed the API. Perhaps after 1.3 soaks for a while and seems solid, I will see if I can make some more gains in a 2.0 with a slightly different API.
David
seancorfield
commented
13 years ago Before you made these changes, I couldn't use the library at all - contrib 1.2.0 won't even load on Clojure 1.3.0, as far as I can tell, so any library that relies on it simply isn't usable with 1.3.0. I've been going thru this with a few 3rd party libraries and I gather Chas Emerick has too - since his book relies on Clojure 1.3.0 and he's had to ask a number of library authors to upgrade their code to work with 1.3.0.
davidsantiago
commented
13 years ago Ah, I thought you were upgrading an app from Clojure 1.2 that used the previous versions.
David
seancorfield
commented
13 years ago No, we started out on early builds of 1.3.0 and we're currently live on Alpha 8 with Beta 1 going to production "real soon now". I'm a big advocate of using bleeding edge builds to take advantage of the latest technology :)
Since clojure-csv depends on contrib 1.2.0, it won't run on Clojure 1.3.0 (because contrib 1.2.0 won't run on Clojure 1.3.0).
I just went thru the process with CongoMongo, making that compatible with Clojure 1.3.0 so I'd be willing to help do the same for clojure-csv (which I'd need to do in order to use clojure-csv since we're wedded to Clojure 1.3.0 at work :)