search

containerd / continuity

A transport-agnostic, filesystem metadata manifest system
https://containerd.io
Apache License 2.0
137 stars 66 forks source link

nerdctl commit taking long time possibly because of fs.Changes walking through all files #236

Open pankajkumar229 opened 5 months ago

pankajkumar229 commented 5 months ago

We have an 8GB image we are trying to commit with nerdctl. It takes a long time for this to happen. It seems to be because fs.Changes implemented in continuity is not using the diff functionality from overlayfs but just walking every file.

https://github.com/containerd/continuity/blob/206f576332bc85557835d77bc18166adbe761f4e/fs/diff.go#L101

It seems like this needs to be implemented here:

https://github.com/containerd/continuity/blob/main/fs/diff_unix.go#L34

Could someone please take a look to see if our understanding is correct. If it is, what would it take to fix this code so it can work more effectively for overlayfs?

AkihiroSuda commented 5 months ago

Link:

pankajkumar229 commented 5 months ago

Thanks for pointing this out. The changes look like they are almost complete. Wondering why this was never merged. if it would be possible to make this fix work by people unfamiliar with this codebase in a reasonable amount of time.