search

nathanmarz / cascalog

Data processing on Hadoop without the hassle.
Other
1.38k stars 178 forks source link

Can't capture result of filter ops anymore #275

Closed ipostelnik closed 9 years ago

ipostelnik commented 9 years ago

Cascalog 1.x allowed you to capture output of filter op or match it to a value:

(def D [[1] [2]])
;; works
(??<- [?x ?z] (D ?x) (odd? ?x :> ?z))
;; doesn't
(??<- [?x ?z] (D ?x) ((filterop odd?) ?x :> ?z))
;; doesn't
(??<- [?x ?z] (D ?x) ((filterop odd?) ?x :> false))

Somehow we lost this in 2+

sritchie commented 9 years ago

Huh, that should still be around for sure. What error are you seeing? What do those queries produce?

— Sent from Mailbox

On Fri, Mar 27, 2015 at 5:37 PM, ipostelnik notifications@github.com wrote:

Cascalog 1.x allowed you to capture output of filter op or match it to a value:

(def D [[1] [2]])
;; works
(??<- [?x ?z] (D ?x) (odd? ?x :> ?z))
;; doesn't
(??<- [?x ?z] (D ?x) ((filterop odd?) ?x :> ?z))
;; doesn't
(??<- [?x ?z] (D ?x) ((filterop odd?) ?x :> false))

Somehow we lost this in 2+

Reply to this email directly or view it on GitHub: https://github.com/nathanmarz/cascalog/issues/275

ipostelnik commented 9 years ago

First failing query is:

RuntimeException Only able to build to ["?x"] but need ["?x" "?z"]. Missing ["?z"] jackknife.core/throw-runtime (core.clj:159)

Second:

RuntimeException Could not apply all operations: (#cascalog.logic.predicate.FilterOperation{:op #clojure.lang.AFunction$1@4d8fc030, :input [false "!G__6710"]}) jackknife.core/throw-runtime (core.clj:159)

sritchie commented 9 years ago

Okay, this is fixed by https://github.com/nathanmarz/cascalog/pull/276. You okay if I push that as 3.0.0-SNAPSHOT? I can also do a full 3.0.0 release. It shouldn't be source incompatible at all, it'll just add the optional in memory platform.

ipostelnik commented 9 years ago

Thanks for fixing this problem. I'd suggest holding off on 3.0 release for a bit. This issue has an easy workaround. I think I've got a couple of other issues coming. Finally migrating our biggest, baddest cascalog project off 1.x.

-Igor

On Mon, Mar 30, 2015 at 11:02 AM, Sam Ritchie notifications@github.com wrote:

Closed #275 https://github.com/nathanmarz/cascalog/issues/275.

— Reply to this email directly or view it on GitHub https://github.com/nathanmarz/cascalog/issues/275#event-268642781.

sritchie commented 9 years ago

Nice! Let me know as these come up. We should be able to get some rapid fire fixes in.

ipostelnik mailto:notifications@github.com March 30, 2015 at 10:10 AM Thanks for fixing this problem. I'd suggest holding off on 3.0 release for a bit. This issue has an easy workaround. I think I've got a couple of other issues coming. Finally migrating our biggest, baddest cascalog project off 1.x.

-Igor

On Mon, Mar 30, 2015 at 11:02 AM, Sam Ritchie notifications@github.com wrote:

Closed #275 https://github.com/nathanmarz/cascalog/issues/275.

— Reply to this email directly or view it on GitHub https://github.com/nathanmarz/cascalog/issues/275#event-268642781.

— Reply to this email directly or view it on GitHub https://github.com/nathanmarz/cascalog/issues/275#issuecomment-87737496.

Sam Ritchie (@sritchie) Paddleguru Co-Founder 703.863.8561 www.paddleguru.com http://www.paddleguru.com/ Twitter http://twitter.com/paddleguru// Facebook http://facebook.com/paddleguru