When profiling loading a form with many repeating group rows, as well as when filling out data in a large repeating group, one of the largest contributors to timing and memory allocation is useExpressionDataSources and useValidationDataSources. Both of these hooks use a lot of simple hooks within them as well as a lot of delayed selectors. When having a thousands of nodes these hooks are called A LOT, and so optimizing these makes a big difference even though you may not think these are expensive in regular use. There are primarily two optimizations introduced here, one for simple hooks, and one for delayed selectors.
For regular hooks where everyone just needs the exact same value, I created a createHookContext method that allows you to run the hook once for the entire node generator and then reuse the value with a useContext instead. It turns out that even a direct subscription to a zustand store via a useSelector is much more expensive than using a simple context value. Some other hooks like useExternalAPI (useQuery) is also expensive when used at this scale.
Delayed selectors previously used 3x useRef, 1x useState, 1x useEffect, and 2x useCallback. For expression data sources we probably had something like 10 delayed selectors (there were some overlap as some other hooks used them internally), we use something like up to 4 expressiondatasources for each node (markhidden, runexpressions, options, sourceoptions). With 10 000 nodes, this becomes a cool couple million hooks 🤯 (of just delayed selectors). Now its more like 80 000 hooks for 10 000 nodes of useExpressionDataSources (of just delayed selectors). Delayed selectors are now implemented using only 1x useRef, and 1x useSyncExternalStore. Additionally, you can create an arbitrary number of delayed selectors with useMultipleDelayedSelectors that also just has a fixed 1x useRef and 1x useSyncExternalStore.
Related Issue(s)
closes #{issue number}
Verification/QA
Manual functionality testing
[x] I have tested these changes manually
[ ] Creator of the original issue (or service owner) has been contacted for manual testing (or will be contacted when released in alpha)
[ ] No testing done/necessary
Automated tests
[x] Unit test(s) have been added/updated
[x] Cypress E2E test(s) have been added/updated
[ ] No automatic tests are needed here (no functional changes/additions)
Description
When profiling loading a form with many repeating group rows, as well as when filling out data in a large repeating group, one of the largest contributors to timing and memory allocation is
useExpressionDataSources
anduseValidationDataSources
. Both of these hooks use a lot of simple hooks within them as well as a lot of delayed selectors. When having a thousands of nodes these hooks are called A LOT, and so optimizing these makes a big difference even though you may not think these are expensive in regular use. There are primarily two optimizations introduced here, one for simple hooks, and one for delayed selectors.createHookContext
method that allows you to run the hook once for the entire node generator and then reuse the value with a useContext instead. It turns out that even a direct subscription to a zustand store via a useSelector is much more expensive than using a simple context value. Some other hooks likeuseExternalAPI
(useQuery) is also expensive when used at this scale.useRef
, 1xuseState
, 1xuseEffect
, and 2xuseCallback
. For expression data sources we probably had something like 10 delayed selectors (there were some overlap as some other hooks used them internally), we use something like up to 4 expressiondatasources for each node (markhidden, runexpressions, options, sourceoptions). With 10 000 nodes, this becomes a cool couple million hooks 🤯 (of just delayed selectors). Now its more like 80 000 hooks for 10 000 nodes of useExpressionDataSources (of just delayed selectors). Delayed selectors are now implemented using only 1xuseRef
, and 1xuseSyncExternalStore
. Additionally, you can create an arbitrary number of delayed selectors withuseMultipleDelayedSelectors
that also just has a fixed 1xuseRef
and 1xuseSyncExternalStore
.Related Issue(s)
Verification/QA
kind/*
label to this PR for proper release notes grouping