Currently, I am facing a significant performance issue when syncing a large dataset that contains over 4000 records. The operation takes approximately 30 seconds to complete and results in blocking and causing UI freezes.
The following is the code snippet responsible for updating the store based on document changes:
`
export async function syncStoreFromDocAction(
storeName: string,
changes: DocumentChange[],
idKey = "id",
removeAndAdd: boolean,
mergeRef: boolean,
formatFromFirestore: (entity: unknown) => EntityType,
): Promise {
setLoading(storeName, false);
if (changes.length === 0) {
return;
}
for (const change of changes) {
const id = change.doc.id;
const entity = change.doc.data();
if (mergeRef) {
await mergeReference(entity as object);
}
const formattedEntity = formatFromFirestore(entity);
switch (change.type) {
case "added": {
upsertStoreEntity(
storeName,
{ [idKey]: id, ...(formattedEntity as object) },
id,
);
break;
}
case "removed": {
removeStoreEntity(storeName, id);
break;
}
case "modified": {
updateStoreEntity(removeAndAdd, storeName, id, formattedEntity);
break;
}
}
}
}
`
The current approach updates the store iteratively for each change, which is not scalable for large datasets. To improve this, I propose the implementation of a more efficient bulk synchronization mechanism that can handle large batches of changes without blocking the UI.
I believe that enhancing the synchronization process to efficiently manage large datasets would be greatly beneficial to many developers using Akita-ng-fire. It would enable smoother user experiences and more responsive applications.
I am looking forward to anyone's feedback on this matter.
Currently, I am facing a significant performance issue when syncing a large dataset that contains over 4000 records. The operation takes approximately 30 seconds to complete and results in blocking and causing UI freezes.
The following is the code snippet responsible for updating the store based on document changes: ` export async function syncStoreFromDocAction(
storeName: string,
changes: DocumentChange[],
idKey = "id",
removeAndAdd: boolean,
mergeRef: boolean,
formatFromFirestore: (entity: unknown) => EntityType,
): Promise {
setLoading(storeName, false);
if (changes.length === 0) {
return;
}
for (const change of changes) {
const id = change.doc.id;
const entity = change.doc.data();
} } ` The current approach updates the store iteratively for each change, which is not scalable for large datasets. To improve this, I propose the implementation of a more efficient bulk synchronization mechanism that can handle large batches of changes without blocking the UI. I believe that enhancing the synchronization process to efficiently manage large datasets would be greatly beneficial to many developers using Akita-ng-fire. It would enable smoother user experiences and more responsive applications.
I am looking forward to anyone's feedback on this matter.