Support `o_direct` flag for tempfile.TemporaryFile

[Zheaoli]

Feature or enhancement

Proposal:

For now, people use the tempfile.TemporaryFile to generate a tmp file under the /tmp default. In Linux the /tmp would be tmpfs which means the memory would be released when it is deleted.

But in container, /tmp is normal file system like other path. which is means that the memory maybe not be released even if the file has been deleted because of the page cache. like https://github.com/bentoml/BentoML/issues/4760#issuecomment-2145326824

So should we add a flag to support O_DIRECT optional for the high

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

[serhiy-storchaka]

I don't think it's Python's business what the OS does with the page cache. In theory, the cache should be released as fast as memory is required for something more important, especially if it is no longer needed. If this causes a problem, then it is a misconfiguration or an OS bug.

A standard workaround in case you want to create temporary files with non-standard options is -- create a temporary directory, then use os.open() and open() to create temporary files in that directory with whatever options you need. This has other advantages, like hiding the name and the size of the temporary file from other users.

[Zheaoli]

I don't think it's Python's business what the OS does with the page cache

Yes, but I think it would be better to allow people to customize the temple flag based on the TemporaryFile API. The people don't need to create a new feature almost same with the TemporaryFile API

create a temporary directory, then use os.open() and open() to create temporary files in that directory with whatever options you need

Actually, the temporary directory still means a lot of page cache, So the problem still exists.