search

flitbit / diff

Javascript utility for calculating deep difference, capturing changes, and applying changes across objects; for nodejs and the browser.
MIT License
2.99k stars 213 forks source link

Massive spike in CPU usage when running diff on a large object #183

Open User9684 opened 2 years ago

User9684 commented 2 years ago

Hi, I run a Discord bot that constantly checks differences in a relatively large object and sends messages accordingly, and I've noticed that every time diff is ran on the object, CPU usage spikes a ton. (See image) image

In my case, the object has roughly 8,100 elements.

User9684 commented 2 years ago

Also, I have no idea how CPU spiked to 160% in that image.

cerebralkungfu commented 7 months ago

I'd love to see a "relatively large object" similar to your case, that I can use for testing. I am working on a significant update (v2.x) which uses a different (more efficient) algorithm and would like to test.

Without the data to test your case, I would guess it is due to the stack. Not only is deep-diff recursive, I track cycles in the graph (recursion in your data), which can cause lots of tracking data on deeply nested objects. But of course, this is speculation.

The new version will give you the option to turn off checking for cycles, which should save some memory in your use case... but I'd only know for sure if I can test with similar data.

User9684 commented 7 months ago

I'd love to see a "relatively large object" similar to your case, that I can use for testing. I am working on a significant update (v2.x) which uses a different (more efficient) algorithm and would like to test.

Without the data to test your case, I would guess it is due to the stack. Not only is deep-diff recursive, I track cycles in the graph (recursion in your data), which can cause lots of tracking data on deeply nested objects. But of course, this is speculation.

The new version will give you the option to turn off checking for cycles, which should save some memory in your use case... but I'd only know for sure if I can test with similar data.

I was particularly monitoring changes to the Roblox Fast Flags list, which changes relatively often.