do not breed ~/.singularity but adhere to standard hierarchy

[yarikoptic]

Version of Singularity:

2.3

Expected behavior

not create ~/.singularity

should use appropriate and canonically located directory under ~/.cache

$> python -c 'from appdirs import AppDirs; dirs = AppDirs("singularity", "singularityware.org"); print(dirs.user_cache_dir)'   
/home/yoh/.cache/singularity

Actual behavior

creates (tries at least ;-) ) to create ~/.singularity even for root

Steps to reproduce behavior

See #699

[bbockelm]

Dumb question - what makes ~/.cache canonical? Not necessarily opposed, I just see pure chaos in my personal home directory and would like to have some, well, canonical written document to point folks at when questions are asked.

[vsoch]

If you don't want the cache to be created (and tmp will be used), set SINGULARITY_DISABLE_CACHE to True/yes, or if you want to move it elsewhere, set SINGULARITY_CACHEDIR. The hierarchy is always the following, by default:

$HOME/.singularity
$HOME/.singularity/docker
$HOME/.singularity/shub
$HOME/.singularity/metadata

[vsoch]

what do you mean canonical?

[yarikoptic]

;-) not that dumb one, since I just took it for granted and forgot details (I also hate having a blooming number of hidden directories and files in my home)... And well, could be argued that singularity has nothing to do with the desktop apps, so why should it care about XDG ;) https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html More info on Debian point of view on it: https://wiki.debian.org/XDGBaseDirectorySpecification which encourages upstreams to conform it but actually suggests maintainers to not patch... oh well -- I have just committed the "crime" -- I did patch singularity debian package to use ~/.cache/singularity :-/ ;-)

[yarikoptic]

according to the code SINGULARITY_DISABLE_CACHE doesn't really "disable it" -- it still creates/uses? some temporary directory as far as I see from the code:

if DISABLE_CACHE == True:
    SINGULARITY_CACHE = tempfile.mkdtemp()
else:
    userhome = pwd.getpwuid(os.getuid())[5]
    _cache = os.path.join(userhome,".singularity") 
    SINGULARITY_CACHE = getenv("SINGULARITY_CACHEDIR", default=_cache)

if not os.path.exists(SINGULARITY_CACHE):
    os.mkdir(SINGULARITY_CACHE)

[yarikoptic]

BTW -- one of the rationales behind having all caches under ~/.cache (besides minimizing pollution of $HOME) is that if I am short of space I could just rm -rf ~/.cache/* in a moment of need. Hunting around $HOME for caches for app X and then Y and then... could be a bit tedious even using cool ncdu ;)

[bbockelm]

Seems like a solid idea to follow the XDG guidelines, but also sounds like it's perhaps a feature improvement for the next release?

[vsoch]

I think we could, but I also think that there are plenty of applications that use a .[name] approach, and don't follow the guidelines. I'll leave it up to @gmkurtzer to decide if this is big enough to warrant changing the entire storage scheme. I sort of like that you can intuitively find .singularity and .singularity.d is similar in the image. What we can do better is tell users how it works, so they can find the cache, and for that your wish is granted :)

[yarikoptic]

someone's future is often our (developers) past! ;)

[yarikoptic]

ok -- I will wait for @gmkurtzer feedback as well before I upload or unpatch debian package

[yarikoptic]

re there are plenty of applications that use a .[name] approach, and don't follow the guidelines

as far as I see -- there is more that follow

$> /bin/ls -l .cache  | nl | tail -n -1
    97  -rw-r--r-- 1 yoh yoh       11 May 27 19:07 zeitgeist-vacuum.stamp

and those which don't are most of the time either small projects not aware of XDG or elderlies which did it for decades already and find it hard to switch ;)

[jmstover]

And well, could be argued that singularity has nothing to do with the desktop apps, so why should it care about XDG ;)

Honestly... this is my take on it.

Singularity itself isn't an X app, so... why worry about following the "X Desktop Group" (freedesktop.org) standard for a non-X app? If some GUI Singularity image manager comes along, then sure, follow the standard it in that application at that point, it's a GUI app then.

But as @yarikoptic did mention there. Singularity itself doesn't have anything to do with desktop apps at this point, so... =)

[bbockelm]

Oh certainly - there is no hard requirements.

But if this helps singularity be a "good citizen" for the larger community - doesn't seem like it would hurt!

(Note - on my personal home directory, nothing in ~/.cache is actually a graphical application. It's things like abrtd and pip)

[jmstover]

Yeah... pip does: path = os.getenv("XDG_CACHE_HOME", expanduser("~/.cache"))

It honestly doesn't surprise me that pip uses it... Firefox will still use ~/.mozilla, and I have a bunch of ${HOME}/.[appname] dirs ... but I'm Slackware, and it's strict however upstream sends it in 90%+ of the cases.

[yarikoptic]

And IMHO even if it were to stay ~/singularity, cashe should reside under ~/singularity/cache to avoid conflicts with whatever might come about and help poor souls in their cleansing endeavours ;-)

[yarikoptic]

@bbockelm just placed a grave stone on top of not yet conceived awesome singularity X app!!! :-)

[gmkurtzer]

Orrrrr, we can use the OSX standard and put it in ~/Librarry/Application\ Support/Singularity/.

In all seriousness, I don't have a strong opinion either way on this, and I think it is just as likely that ~/.cache will also tend to get garbaged up as well, but the benefit is at least the garbage is in a common cesspool of both trash and non-trash.

Since I'm not strongly leaning in either particular direction, and can find benefits and cons to each, I am happy to take consensus.

But I do feel strongly is that the packaged versions shouldn't do anything that would diverge from standard documented behavior.

[yarikoptic]

ok, for consistency and to maintain status quo I will sacrifice a new ~/. directory on each user's box. reverting my change and will upload to (Neuro)Debian(s). Cheers

[vsoch]

hey @yarikoptic where was your build located again? If you are going to reliably be building for each new release, I'd like to put instructions for using your build in the singularity docs proper.

[yarikoptic]

Filled up on #87 . It is now a part of standard Debian and Ubuntu releases

[vsoch]

wicked! Let me add to the docs and you can yell at me when (I mean if) I get something wrong :)

[vsoch]

cool! I don't have that version of Ubuntu but tested on a Docker image. Is there going to be an update for this for the 2.3 release? eg:

Preparing to unpack .../singularity-container_2.1.2-1_amd64.deb ...
Unpacking singularity-container (2.1.2-1) ...
Setting up singularity-container (2.1.2-1) ...
root@8b1fd08d182d:/# which singularity
/usr/bin/singularity
root@8b1fd08d182d:/# singularity --version
2.1

It looks like it installs 2.1. Should I install differently?

[vsoch]

ok I added to the docs, but mentioned that the version is 2.2.1. Let me know when 2.3 is ready and I'll update! http://singularity.lbl.gov/install-linux#debianubuntu-flavor-install

[yarikoptic]

"released" debian and ubuntus would have the version which came out with the release. backport builds would need to be used from neurodebian. See http://neuro.debian.net/pkgs/singularity-container.html about which version available from which offical release of debian/ubuntu or neurodebian

[yarikoptic]

i.e. if you use neurodebian docker image, should come with most recent:

$> docker run -it --rm neurodebian
root@bbf0f5c4da38:/# apt-get update -qqq
root@bbf0f5c4da38:/# apt-cache policy singularity-container
singularity-container:
  Installed: (none)
  Candidate: 2.3-1~nd80+1
  Version table:
     2.3-1~nd80+1 0
        500 http://neuro.debian.net/debian/ jessie/main amd64 Packages

[vsoch]

How did I do?

http://singularity.lbl.gov/install-linux#debianubuntu-package

[yarikoptic]

you did good!!! I would only add in the leading paragraph something about NeuroDebian, since otherwise it comes out out of the blue later on (like a winner though ;-)). Smth along "For recent releases of singularity and backports for older Debian and Ubuntu releases, use NeuroDebian repository."

[jscook2345]

@yarikoptic Things got a little off topic here, but it appears we won't be addressing this issue at the moment. I'd be happy to revisit or discuss the topic of dotfile standards in another issue if you wish. Thanks!

[yarikoptic]

as you wish. I will come back whining again in the next moment of "out of cheese" situation ;-)