Use /var/tmp/ or ~/.cache directory instead of /tmp for building

[futureisfoss]

This is a followup to this issue - https://github.com/nim-lang/langserver/issues/46#issuecomment-1694694228

In short, I think its best for nimble to use the ~/.cache directory to store its build files instead of /tmp - this will help avoid permission errors, and more importantly it'd avoid filling up the user's RAM. Currently every time I have to install or update anything using nimble I'm afraid that it might fill up my RAM and crash my system, build files inside /tmp can fill up my RAM much more quickly than I thought.

Slightly related question: Why did nimble had to download and compile the entire Nim language for me to install nimlangserver ? I already had Nim installed using choosenim so I don't understand why nimble had to manually compile everything again, correct me if I'm mistaken.

[futureisfoss]

I notice this issue might be related to #674, but that issue has been closed without any explanation so I don't know.

[arnetheduck]

In short, I think its best for nimble to use the ~/.cache directory to store its build files

.cache has similar problems of becoming an unwelcome ever-growing garbage dump - ideally, nimble (and nim) would put build files it doesn't remove automatically as part of each run in the project folder itself (ie {projectDir}/nimcache where they're visible and easily removable

[futureisfoss]

The main problem with using /tmp directory is that it's stored on the user's RAM, and in my case nimble had put so much files in my /tmp directory that it filled up my RAM and froze my laptop (I don't know the exact size but it was upwards of 2 GB). I'm not saying ~/.cache should become a garbage dump, the files could be cleaned afterwards just like how it does now, but at least it won't fill up the user's RAM like in the case of /tmp

[Araq]

While you're right and Nimble should not use /tmp because nothing should ever use it anyway (it's highly unportable), mapping the /tmp to RAM is equally broken nonsense.

[futureisfoss]

As far as I know, mapping /tmp to RAM (tmpfs) is pretty standard these days, for better or for worse.

https://www.freedesktop.org/software/systemd/man/file-hierarchy.html#/tmp/

/tmp The place for small temporary files. This directory is usually mounted as a "tmpfs" instance, and should hence not be used for larger files. (Use /var/tmp/ for larger files.)

https://systemd.io/TEMPORARY_DIRECTORIES/

/tmp/ and /var/tmp/ are two world-writable directories Linux systems provide for temporary files. The former is typically on tmpfs and thus backed by RAM/swap, and flushed out on each reboot. The latter is typically a proper, persistent file system, and thus backed by physical storage. This means:

1. /tmp/ should be used for smaller, size-bounded files only; /var/tmp/ should be used for everything else.

2. Data that shall survive a boot cycle shouldn’t be placed in /tmp/.

In my case I had to manually mount my /tmp directory to tmpfs (RAM) because I'm on Artix linux which didn't do that by default which meant my files inside /tmp weren't being cleaned up on reboot. But in most other linux distros (using systemd) this is being done automatically by default, which is probably why https://www.freedesktop.org is recommending us to avoid putting large files in it.

[Araq]

Never heard of /var/tmp/ before, interesting.

[futureisfoss]

Using /var/tmp instead of /tmp would also solve the problem, I initially didn't think about that directory before reading from these other sources but now I think that's also a good alternative apart from ~/.cache

[arnetheduck]

the broader issue remains: there is no functionality in place for limiting the size of the junk generated here - as such, keeping it local in the project folder makes a lot of sense, instead of creating an ever-growing pile of junk in a folder that the user rarely looks in and might not be aware of until the disk is full and slow.

[futureisfoss]

I'm not sure what you mean by the project folder here, this is about installing a package using nimble not about compiling my own project. Also, from what I've noticed so far, nimble automatically cleans up the /tmp directory after its done building the package, so it won't turn into an ever-growing pile of junk in a folder. My only issue here is the RAM filling up, which can be solved by using the /var/tmp directory instead of /tmp

[futureisfoss]

@arnetheduck I feel like my title might've been a little misleading, my RAM got filled up by the files nimble was putting inside my /tmp directory while trying to install the nimlangserver package, I wasn't trying to build/compile my own projects. Apart from just RAM issues, at first I also got some permission denied errors which was related to the way I mounted my /tmp directory, you can read about the whole thing here (I had mentioned this link above too). In the end after all of the troubleshooting I realized that using /tmp directory to put such big files in was not a good idea, which is why I reported it here.

[arnetheduck]

Ah, fair - if it's being cleaned up, it's fine - nimble and nim both have several cache directories that keep growing (~/.nimble and ~/.cache/nim) and I assumed tmp might be the same leading to these issues.