search

Budibase / budibase

Low code platform for building business apps and workflows in minutes. Supports PostgreSQL, MySQL, MariaDB, MSSQL, MongoDB, Rest API, Docker, K8s, and more 🚀
https://budibase.com
Other
22.76k stars 1.58k forks source link

Bypass transformers that are trivial, allow users to configure bson buffer size. #15032

Closed samwho closed 3 days ago

samwho commented 3 days ago

Description

This PR checks if transformers are simply just returning the data they are given and if that's the case bypasses them entirely. This avoids the serialization/deserialization overhead of passing data into and out of the JS isolate, especially for MongoDB which does BSON serialization and deserialization.

I'm also making the BSON internal buffer size configurable with an environment variable so that we can change it to whatever we want as can self-host users. Currently you cannot return more than 17 megabytes of data from a MongoDB query and this limitation is imposed by the BSON library because 16 megabytes is the MongoDB document size limit. Queries could be returning multiple documents so this limit could be quite constricting for no good reason.

qa-wolf[bot] commented 3 days ago

QA Wolf here! As you write new code it's important that your test coverage is keeping up. Click here to request test coverage for this PR!