noprompt / meander

Tools for transparent data transformation
MIT License
921 stars 55 forks source link

StackOverflowError when nesting the maybe pattern #141

Closed lucywang000 closed 3 years ago

lucywang000 commented 3 years ago

Say we have a list of maps, where there is an optional :b key, and inside :b there is an optional :b2 key.

(m/search [{:a 1}
           {:a 1 :b {:b1 1}}
           {:a 1 :b {:b1 1 :b2 2}}]
  (m/scan {:a ?a
           :b (m/or (m/and nil ?b1 ?b2)
                    {:b1 ?b1
                     :b2 (m/or (m/and nil ?b2)
                               ?b2)
                     })})
  {:a ?a :b1 ?b1 :b2 ?b2})

But this kind of nesting throwed a StackOverflowError.

btw: I wanted to ask in slack, but I think maybe github is a better place in case others would search for similar questions here.

timothypratley commented 3 years ago

could you please try the solution to #133 -- setting a larger stack size in JVM options (add :jvm-opts ["-Xss2m"] to your project.clj)

jimmyhmiller commented 3 years ago

Ignore this comment. Left a comment below. I thought there was a cata here when there was not. The m/somes I have here will still probably give you the results you really expected.

@timothypratley Not the problem in this case.

I can explain a bit more later, but here is the fix for this particular example

(m/search [{:a 1}
           {:a 1 :b {:b1 1}}
           {:a 1 :b {:b1 1 :b2 2}}]
  (m/scan {:a ?a
           :b (m/or (m/and nil ?b1 ?b2)
                    {:b1 (m/some ?b1)
                     :b2 (m/or (m/and nil ?b2)
                               (m/some ?b2))
                     })})
  {:a ?a :b1 ?b1 :b2 ?b2})

Basically, search goes down both paths if the pattern matches. So you need to guard against that or else you are infinitely recursing.

jimmyhmiller commented 3 years ago

Sorry, I shouldn't comment when busy. This doesn't stackoverflow for me. So maybe it is related to Tim's issue? How are you running this code?

jimmyhmiller commented 3 years ago

I am able to recreate the issue now. While we have a work around, this is something we need to fix and I will start trying to look into the issue.

noprompt commented 3 years ago

PR GH-141 is open for this ticket.