Closed onsi closed 8 years ago
With this shared cache, you could end up with more items than you expected? You say I want cache foo and I want this 1 thing in there, but someone else says 2 things for the same cache.
Do we care that the cache may change as you are using it? User A is using cache foo, user B says add/update this thing in the cache.
How do we remove things from the cache? Unless we are replacing a consistent name, the cache will grow and grow. What if a buildpack is removed? How do we know it should be removed from the cache?
How are we managing disk space? Does the cache count against the total disk allowed?
This is exactly how we are managing buildpacks with DEAs. However, I would add one additional caveat. We only "prune" a shared cache if all the users agree on its content. If we have multiple stagers and they have different content, we punt. We don't really know who has the correct list of items when there are changes afoot.
@fraenkel - I thought of a bunch of these issues as I was writing the proposal up and wanted to see if we could make headway as a group.
but maybe there's a simpler way that resolves most of these. I'll sketch it out briefly here and can update the doc if it has merit. I propose:
DownloadAction
from CachedDownloadAction
CachedDownloadAction
looks like {From: "url", To: "/path", CacheKey: "cache-key"}
/cached-downloads
CachedDownload
assets go but then symlinks just the requested files into the container's cached-downloads
folder. The /path
in the CachedDownloadAction
will always be relative to /cached-downloads
.So how does this single global cache work? I think there's a path forward and we can hash out the details as a team but I was thinking something like this might work (apologies for the detail, I'm working through it as I go. I imagine I'm overcomplicating things and there's a much simpler way forward):
/global-cache/{cache-key}-0
{CacheKey: cacheKey, Index: 0, Date: time.Now(), Etag: etag}
/global-cache/{cache-key}-(m+1)
where m
is the greatest index for this cache-key/global-cache/{cache-key}-m
where m
is the greatest index for this cache-keyit's complicated. Though, honestly, the existing cache is already doing some of this stuff. We just need to be careful about not overwriting things that might be in use and then just symlink things into the directory that is bind-mounted into the container.
We should probably start by making sure that this whole symlink+bindmount trick will work. If it does then I think this approach is feasible and I don't think it's a terrible API to say that Diego can do cached downloads and its up to the consumer to know that they live in /cached-downloads
.
For the concrete case of buildpacks we'd have to teach the builder to look in /cached-downloads
for the buildpacks. That shouldn't be terrible and it could even perform a subsequent symlink if it needs to. We should probably spike and make sure the buildpacks actually work if symlinked in like this (I imagine there may be a whole mess of issues around users and permissions, etc...)
Amicrazy?
What you are proposing does make sense. You have made assumptions about the content a url delivers, and does align with the lifecycle of the content we tend to download. There are the corner cases which we should just handle as non-cached items like no etag support, etc... It might be interesting to add stats around the cache behavior. It would provide useful feedback to operators to know when their caches are too small.
aight cool -- I'll update the doc and add a story.
closing this in favor of #30
I've just added
https://github.com/pivotal-cf-experimental/diego-dev-notes/blob/master/proposals/better-buildpack-caching.md
It's a WIP proposal for improving Diego's staging performance. I'm (very) open to suggestions. We aren't likely to schedule this immediately but wanted to get some thoughts together.