Switch from `memfs` to tmpfs-backed clones

[evankanderson]

Don was poking around and discovered that some Git repos may be 900MB just to shallow clone (think: they check in a copy of the JVM).

This isn't a reasonable amount to store in memory, but it's totally reasonable to put into tmpfs using a Kubernetes EmptyDir volume.

Unfortunately, we don't have a really great way to clean these up afterwards at the moment -- Git.Clone() in https://github.com/stacklok/minder/blob/main/internal/providers/git/git.go#L71 builds the memfs.New() and returns it as part of the *git.Repository, which is called by Git.Ingest() in https://github.com/stacklok/minder/blob/main/internal/engine/ingester/git/git.go#L89, which packages the repository up into an engine/interfaces.Result and possibly stores it in the RuleTypeEngine's r.ingestCache in RuleTypeEngine.Eval() here: https://github.com/stacklok/minder/blob/main/internal/engine/rule_type_engine.go#L160

[evankanderson]

(In fact, I don't see where we ever clean up entries in the ingestCache...)

[evankanderson]

Okay, it looks like the ingestCache is created per-EntityEvent in evalEntityEvent: https://github.com/stacklok/minder/blob/main/internal/engine/executor.go#L193, and presumably only lasts for the duration of handling the event.

Unfortunately, there isn't a good hook in that method or in ingestCache for calling "destructors" on the cached items, so we'd need to plumb that in as well.

[Vyom-Yadav]

I also thought sometime back what'll happen if someone tries to ddos the server by registering a lot of repos that are large in size (e.g. k8s clones). Most likely it will timeout as we have max limit for processing each request, but still multiple timeouts one after the other 🤷🏼‍♂️