Closed akostadinov closed 3 years ago
If you point TMPDIR environment variable at that location, does everything work ok?
One issue with pointing TMPDIR at /var/tmp is we get guaranteed cleanup from systemd at some point. Pointing at our own tmp means we need to pay attention to it. (Hint we don't).
TMPDIR works. But then you have to have storage space on root filesystem as well on specially mounted container filesystem. For example oracle image is around 4GB and it can't fit standard fedora qcow2 image. Also if TMPDIR
is used then still user has to deal with clean-up.
@mtrmac @mheon @vrothberg @nalind WDYT?
TMPDIR
, as well as its default value, is now documented in man pages; so that’s a fairly good reason why users should expect it to be used.
I don’t care much at all what the default should be, but I’m not much of a fan of changing the default now that it exists; this can easily break someone’s carefully-tuned quotas / carefully-tested resiliency of existing workloads vs. pulling unexpectedly-large images.
It might be worth considering if there were other benefits (e.g. maybe integrating this with c/storage’s cleanup of orphaned partial layers — OTOH I’m not sure that would be even possible), or if operational experience strongly suggested that sharing a quota is preferable, but we would have to spend quite some effort to communicate the change (or perhaps even wait for a major version break).
One thing we could to is add it to containers.conf, so that we could setup something like
# Default location for storing temporary container image content, Can be overriden with the TMPDIR environment
# variable. If you specify "storage", then the location of the container/storage tmp directory will be used.
# storage_tmp_dir="storage"
storage_tmp_dir="/var/tmp"
And then allow users to override this field. That way we ensure backwords compatibility.
It will be a good start. And also point at this setting from wherever graphRoot
is documented. Honestly I can't find where graphRoot
is officially documented.
Funnily Duckduckgo only shows me oracle docs which do not mention TMPDIR.
And man podman pull
does not show anything about graphRoot
.
btw I find it a very remote possibility that somebody relies on TMPDIR
default being outside graphRoot
. If anybody changed it, it would most likely be because they didn't want to burden their root filesystem with container data. If that temporary storage used the regular data storage location, and it didn't leave orphan tmp data, then it's very unlikely IMO that somebody would care.
On the other hand one has to lose a lot of time figuring out why some data is here, some data not and where exactly data goes and why (at least it took me a good amount of time as well after 3 months I probably would forget the details and could be bitten by that again). My observation on docker is that it keeps everything under one storage directory unless I'm missing something.
A friendly reminder that this issue had no activity for 30 days.
And
man podman pull
does not show anything aboutgraphRoot
.
That’s what I would expect right now, when it does not use graphRoot
for temporary files.
btw I find it a very remote possibility that somebody relies on
TMPDIR
default being outsidegraphRoot
.
The documentation does talk about TMPDIR
though, see https://github.com/containers/podman/blob/main/docs/source/markdown/podman-pull.1.md . And that came from https://github.com/containers/podman/pull/5412 , which points to actual users’ report of needing that setting. So there’s at least one person that relies on that.
Users would need to use TMPDIR
, no wonder. I'm not suggesting to remove this ability.
I'm suggesting to have a better and expected default. At least make this configurable (as Dan suggested) in configuration file so that this environment variable does not need to be set.
/kind bug
Description
I mounted a new volume at graphRoot but pulling images still eats up storage under
/var/tmp
. I think it makes much more sense to use$graphRoot/tmp
for temporary storage instead. That directory is already automatically created. I see no reason that user would expect storage outsidegraphRoot
to be used.Steps to reproduce the issue:
mount volume /home/fedora/.local/share/containers/storage
Describe the results you received:
While pulling I see space on
/
exhausted and pull fails.Describe the results you expected:
Additional information you deem important (e.g. issue happens only occasionally):
Output of
podman version
:Output of
podman info --debug
:Package info (e.g. output of
rpm -q podman
orapt list podman
):Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)
No
Additional environment details (AWS, VirtualBox, physical, etc.):
KVM, VirtualMachineManager