Dict Equality Checks rapidly exceeds max recursion

[joe-kimmel-vmw]

What steps did you take: I wanted to compare two (yaml) structures that had been read into data.values and turned into dictionaries. I used the == operator as is our traditional way.

What happened: - comparison exceeded maximum recursion depth

What did you expect: a boolean

Anything else you would like to add: the culprit is here: https://github.com/vmware-tanzu/carvel-ytt/blob/develop/vendor/github.com/k14s/starlark-go/starlark/value.go#L1134 because whenever they made starlark they were really conservative re. the stack limits for equality comparisons.

IMO this would be magical to expose via command line flag and/or hardcode several orders of magnitude larger.

(note that if this is exposed via CLI flag, I think there's another place where depth/maxdepth is also hardcoded, but I'll let you-all grep the codebase and consider implementation details like if i'm raising the recursion limit in one place do i probably also want to raise it in all the places? probably?)

Environment: Here is a repro:

#@ def buildNest(mm, d):
#@ if d == 0:
#@   return mm
#@ end
#@ 
#! "i pity d foo"; see below
#@ i = d
#@ mm[d] = {i: buildNest({'pity': 'foo'}, d-1)}
#@ return mm
#@ end

#@ depth = 4

#@ map1 = buildNest({}, depth)
#@ map2 = buildNest({}, depth)

map: #@ map1
#! this exceeds recursion depth when depth >= 5. I pity d foo who exceeds max recursion depth!
nested_map: #@ map1 == map2

(i suspect this exists in all versions of ytt released up til today)

---

Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible" 👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help working on this issue.

[pivotaljohn]

Related upstream: https://github.com/google/starlark-go/issues/360

[pivotaljohn]

Until recently, Starlark as a language operated (and through many integrations) under this comparison limit.

Hypotheses:

• given that Starlark is meant to be used as a configuration language, this domain — in practice — does not need such depth.
• folks ran into this limit and opted-out of using Starlark
• ???

What brought this up, @joe-kimmel-vmw?

[pivotaljohn]

This is Starlark behaving as designed: limit depth to avoid "infinite recursion". Increasing or making this limit greater is more of an enhancement than a bug. 🤷🏻

[pivotaljohn]

Update: as part of vmware-tanzu/carvel-ytt#593, the required upstream support for setting this value (noted above) will be coming in to the next release of our Starlark fork. 🥳