'~' not expanded when calculating local reg dirs during install

[clbarnes]

When installing the package with devtools::install_github("natverse/nat.templatebrains"), on my system whatever functions get the directories starting with ~/.local/usr/ etc. seem to differ in whether they expand ~ to $HOME. So they create a directory literally called \~/.local ..., and then later try to read from $HOME/.local... and can't find it.

The workaround in the short term was ln -s $HOME $(pwd)/\~, install, then rm \~

[jefferis]

Hi Chris, Thanks for raising the issue and sorry that you ran into trouble. Can you provide any more information about exactly when this issue occurs. Is it during package install or afterwards when registrations are downloaded? What is inside these directories, what do the paths look like?

The install process for nat.templatebrains should not actually create any local registration directories. That is a separate post-install step e.g. when nat.flybrains or nat.jrcbrains download registrations. Indeed it is a condition of the nat.templatebrains being on the central CRAN repository that it does not make files or folders anywhere other than R's temp folder without receiving end user instructions.

[clbarnes]

Ah, got it - it probably is something downstream then (I was setting up a catmaid instance which depends on a solid chunk of natverse), I just noticed the error messages in the context of templatebrains. I'll have a dig tomorrow!

[jefferis]

Incidentally at some point you could check what these report.

nat.templatebrains::local_reg_dir_for_url()
nat.templatebrains::extra_reg_folders()
nat.templatebrains::allreg_dataframe()
system.file("extdata", "bridgingregistrations", package = "nat.flybrains")

[clbarnes]

You're right; after installing templatebrains and flybrains, the problem arises when calling nat.flybrains::download_jefferislab_registrations() (didn't spot the distinction as CATMAID wraps it all into one function).

> nat.templatebrains::local_reg_dir_for_url()

[1] "~/.local/share/rpkg-nat.templatebrains/regfolders"

> nat.templatebrains::extra_reg_folders()

character(0)

> nat.templatebrains::allreg_dataframe()
                                                                                              path
1          /home/cbarnes/rlibtest/rlibs1/nat.flybrains/extdata/bridgingregistrations/FCWB_IS2.list
2        /home/cbarnes/rlibtest/rlibs1/nat.flybrains/extdata/bridgingregistrations/FCWB_JFRC2.list
3         /home/cbarnes/rlibtest/rlibs1/nat.flybrains/extdata/bridgingregistrations/IBNWB_IBN.list
4       /home/cbarnes/rlibtest/rlibs1/nat.flybrains/extdata/bridgingregistrations/JFRC2_IBNWB.list
5         /home/cbarnes/rlibtest/rlibs1/nat.flybrains/extdata/bridgingregistrations/JFRC2_IS2.list
6      /home/cbarnes/rlibtest/rlibs1/nat.flybrains/extdata/mirroringregistrations/FCWB_mirror.list
7       /home/cbarnes/rlibtest/rlibs1/nat.flybrains/extdata/mirroringregistrations/IS2_mirror.list
8     /home/cbarnes/rlibtest/rlibs1/nat.flybrains/extdata/mirroringregistrations/JFRC2_mirror.list
9  /home/cbarnes/rlibtest/rlibs1/nat.flybrains/extdata/mirroringregistrations/JFRC2013_mirror.list
10       /home/cbarnes/rlibtest/rlibs1/nat.flybrains/extdata/mirroringregistrations/T1_mirror.list
11                             /tmp/RtmpDBtyZu/nat.templatebrains/tempreglists/FAFB14_FAFB14um.rds
12                           /tmp/RtmpDBtyZu/nat.templatebrains/tempreglists/FAFB14um_JRC2018F.rds
13                               /tmp/RtmpDBtyZu/nat.templatebrains/tempreglists/FCWB_JRC2018F.rds
14                              /tmp/RtmpDBtyZu/nat.templatebrains/tempreglists/JFRC2010_JFRC2.rds
15                           /tmp/RtmpDBtyZu/nat.templatebrains/tempreglists/JFRC2010_JRC2018F.rds
16                           /tmp/RtmpDBtyZu/nat.templatebrains/tempreglists/JFRC2013_JRC2018F.rds
17                           /tmp/RtmpDBtyZu/nat.templatebrains/tempreglists/JRC2018F_FAFB14um.rds
18                               /tmp/RtmpDBtyZu/nat.templatebrains/tempreglists/JRC2018F_FCWB.rds
19                           /tmp/RtmpDBtyZu/nat.templatebrains/tempreglists/JRC2018F_JFRC2010.rds
20                           /tmp/RtmpDBtyZu/nat.templatebrains/tempreglists/JRC2018F_JFRC2013.rds
21                           /tmp/RtmpDBtyZu/nat.templatebrains/tempreglists/JRC2018F_JRC2018U.rds
22                        /tmp/RtmpDBtyZu/nat.templatebrains/tempreglists/JRC2018F_JRCFIB2018F.rds
23                           /tmp/RtmpDBtyZu/nat.templatebrains/tempreglists/JRC2018U_JRC2018F.rds
24                        /tmp/RtmpDBtyZu/nat.templatebrains/tempreglists/JRCFIB2018F_JRC2018F.rds
                       name   dup bridge   reference      sample
1             FCWB_IS2.list FALSE   TRUE        FCWB         IS2
2           FCWB_JFRC2.list FALSE   TRUE        FCWB       JFRC2
3            IBNWB_IBN.list FALSE   TRUE       IBNWB         IBN
4          JFRC2_IBNWB.list FALSE   TRUE       JFRC2       IBNWB
5            JFRC2_IS2.list FALSE   TRUE       JFRC2         IS2
6          FCWB_mirror.list FALSE  FALSE        FCWB        <NA>
7           IS2_mirror.list FALSE  FALSE         IS2        <NA>
8         JFRC2_mirror.list FALSE  FALSE       JFRC2        <NA>
9      JFRC2013_mirror.list FALSE  FALSE    JFRC2013        <NA>
10           T1_mirror.list FALSE  FALSE          T1        <NA>
11      FAFB14_FAFB14um.rds FALSE   TRUE      FAFB14    FAFB14um
12    FAFB14um_JRC2018F.rds FALSE   TRUE    FAFB14um    JRC2018F
13        FCWB_JRC2018F.rds FALSE   TRUE        FCWB    JRC2018F
14       JFRC2010_JFRC2.rds FALSE   TRUE    JFRC2010       JFRC2
15    JFRC2010_JRC2018F.rds FALSE   TRUE    JFRC2010    JRC2018F
16    JFRC2013_JRC2018F.rds FALSE   TRUE    JFRC2013    JRC2018F
17    JRC2018F_FAFB14um.rds FALSE   TRUE    JRC2018F    FAFB14um
18        JRC2018F_FCWB.rds FALSE   TRUE    JRC2018F        FCWB
19    JRC2018F_JFRC2010.rds FALSE   TRUE    JRC2018F    JFRC2010
20    JRC2018F_JFRC2013.rds FALSE   TRUE    JRC2018F    JFRC2013
21    JRC2018F_JRC2018U.rds FALSE   TRUE    JRC2018F    JRC2018U
22 JRC2018F_JRCFIB2018F.rds FALSE   TRUE    JRC2018F JRCFIB2018F
23    JRC2018U_JRC2018F.rds FALSE   TRUE    JRC2018U    JRC2018F
24 JRCFIB2018F_JRC2018F.rds FALSE   TRUE JRCFIB2018F    JRC2018F

> system.file("extdata", "bridgingregistrations", package = "nat.flybrains")
[1] "/home/cbarnes/rlibtest/rlibs1/nat.flybrains/extdata/bridgingregistrations"

I suspect that the path generated by local_reg_dir_for_url gets passed to some different functions, some of which expand the ~, and some of which treat it as a literal.

cbarnes@neurophyla:~/rlibtest$ ls \~/.local/share/rpkg-nat.templatebrains/regfolders
09ea2745e48f45dd70e311a577c8d5414aea2ce5

cbarnes@neurophyla:~/rlibtest$ ls ~/.local/share/rpkg-nat.templatebrains/regfolders

It seems that the downloads are made into the literal \~, but the regfolders directory is at least created in $HOME. Then some later call tries to look into $HOME, but doesn't find the download (for obvious reasons).

I didn't have any problems with nat.jrcbrains::download_saalfeldlab_registrations(), so this issue may be in the wrong place.

[jefferis]

In theory this should have been fixed by https://github.com/natverse/nat.templatebrains/commit/9970b5d14e40bc0165e366817ab6ebcfa129778e. @clbarnes did you look at this again?

[jefferis]

@dokato fyi.

[clbarnes]

Seems to have done the trick, thanks all!

A related feature request (which I can open a new issue for): might it be possible to set an environment variable to point to the registration folders? On the server it would be useful to have them in a central location rather than bound to the calling user.

[jefferis]

A related feature request (which I can open a new issue for): might it be possible to set an environment variable to point to the registration folders? On the server it would be useful to have them in a central location rather than bound to the calling user.

@clbarnes if you still need this I have added a documentation note about how it can be achieved.