tolitius / cbass

adding "simple" to HBase
Eclipse Public License 1.0
24 stars 11 forks source link

Made delete-by always lazy to avoid OOM exceptions #5

Closed chrishowejones closed 8 years ago

chrishowejones commented 8 years ago

There's no reason for delete-by to have a fully realised collection in the intermediary stage of the scan as this is opaque to the user so make the scan always lazy.

tolitius commented 8 years ago

ah.. good point. as far the PR goes, we might want to concise it up a little:

instead of

(defn delete-by [conn table & by]
  "Delete by using scan's syntax for the selection criteria."
  (let [criteria (if by (vec by) [])
        row-keys (->>
                  (conj criteria :lazy? true) ;; always fetch keys lazily to avoid OOM errors
                  (apply scan conn table)
                  (map first))

we can just

(defn delete-by [conn table & by]
  "Delete by using scan's syntax for the selection criteria."
  (let [row-keys (-> (conj by true :lazy?)
                     (apply scan conn table)
                     (map first))
chrishowejones commented 8 years ago

Tidied up as suggested and retested.

tolitius commented 8 years ago

looks good. thanks!

chrishowejones commented 8 years ago

Thanks