47ng / prisma-field-encryption

Transparent field-level encryption at rest for Prisma
https://github.com/franky47/prisma-field-encryption-sandbox
MIT License
223 stars 27 forks source link

fix: `Maximum call stack size exceeded` in `traverseTree` #73

Closed Qrtn closed 11 months ago

Qrtn commented 11 months ago

If prisma-field-encryption is used as middleware:

When working with a table (that may or may not use prisma-field-encryption) that has lots of rows/data, calling stack.unshift(...children) may fail with:

RangeError: Maximum call stack size exceeded

This is because children is too large to be passed on the stack as arguments to unshift.

I noticed this happening when calling prisma.<table>.findMany() and expecting a large amount of data back:

The presence of prisma-field-encryption in the middleware will cause the call to fail with the error above.

franky47 commented 11 months ago

This is probably a trade-off with memory consumption, recreating the stack array with large numbers of elements on every node visit will probably lead to OOM errors on RAM-limited containers, would you be able to emulate that in a test?

Qrtn commented 11 months ago

@franky47 we can also just call .push() in a for loop instead. Would that be preferable?

  const stack: StackItem[] = [
    {
      path: [],
      type: typeOf(input),
      node: input,
      state: initialState
    }
  ]

  while (stack.length > 0) {
    const { state, ...item } = stack.pop()!
    const newState = callback(state, item)
    if (!isCollection(item.node)) {
      continue
    }
    const children: StackItem[] = Object.entries(item.node).map(
      ([key, child]) => ({
        key,
        node: child,
        type: typeOf(child),
        path: [...item.path, key],
        state: newState
      })
    )
    for (let i = children.length - 1; i >= 0; --i) {
      stack.push(children[i])
    }
  }

Basically if you have a large table and you call findMany() on it right now with prisma-field-encryption enabled, it will fail, regardless of whether or not there are encrypted fields in that table.

https://stackoverflow.com/questions/61740599/rangeerror-maximum-call-stack-size-exceeded-with-array-push

franky47 commented 11 months ago

That may work, indeed.

Edit: disregard my comment about ordering, I got confused between the original implementation and your changes.

Qrtn commented 11 months ago

Awesome, thanks @franky47 !

github-actions[bot] commented 8 months ago

:tada: This PR is included in version 1.5.1-beta.1 :tada:

The release is available on:

Your semantic-release bot :package::rocket:

github-actions[bot] commented 6 months ago

:tada: This PR is included in version 1.5.1 :tada:

The release is available on:

Your semantic-release bot :package::rocket: