Open brandonwillard opened 1 month ago
I noticed from the helpful test case in #1129—and the relevant vllm
code—that the cache decorator is being directly applied to the function object, so we may not even be able to tell when a function is a method in order to give a warning. We might be able to handle the case of a method descriptor object, but that wouldn't have helped in the examples above.
In other words, there's probably not a general way to warn people that they could be doing something undesirable for caching. Because of that, we might want to make the caching utilities private and leave it up to the user to determine how caching should work, since all we can really guarantee is that underlying objects are serializable in a way that can work for caching.
Currently, the
cache
function doesn't truly support methods (see #1129). Let's considercache
interface private/internal,