jmacd / xdelta

open-source binary diff, delta/differential compression tools, VCDIFF/RFC 3284 delta compression
http://xdelta.org
1.09k stars 181 forks source link

system crash due to xdelta3 is out of memory on v3.0.11 #251

Open VikasVishwanath opened 5 years ago

VikasVishwanath commented 5 years ago

We were using the xdelta version 3.0.0 and recently moved to V3.0.11 and we see that the xdelta3 takes lot of memory and the demon gets killed or the whole system reboots. We are using the command as below generate the differential image on the PC xdelta3-pc -e -f -s sys_v1.ubifs sys_v2.ubifs sys.diff

And the below command is run on the ARM cortex 7 target to regenerate the full image sys_v2 using the sys_v1 & the delta sys.diff xdelta3 -f -d -s sys_v1.ubifs sys.diff sys_v2

The memory issue can be seen as below Out of memory: Kill process 1105 (xdelta3) score 302 or sacrifice child Killed process 1105 (xdelta3) total-vm:78844kB, anon-rss:51100kB, file-rss:88kB

This is a sporadic issue and is happening for few of the images.

We tried to use the option as below

xdelta3 -I 2 -f -d -s sys_v1.ubifs sys.diff sys_v2 and it seems to be working but not sure if its a proper way of doing it. Can you please explain more on the memory options and how does this work, what is the best command to use here?

memory options: -B bytes source window size -W bytes input window size -P size compression duplicates window -I size instruction buffer size (0 = unlimited)

Any suggestions?