search

sidora-tools / sidora.cli

A CLI for terminal based data extraction and summary for the MPI-SHH Department of Archaeogenetics PANDORA Database
Other
0 stars 0 forks source link

Cluster setup #16

Open nevrome opened 4 years ago

nevrome commented 4 years ago

What's the best way to install this tool on the cluster, make it easily available and and keep it updated?

In theory it should be possible to install it in the system wide R package library with remotes::install_github() and then create a system wide alias from sidora to path/sidora.R. Updating would then boil down to running remotes::install_github(). But that has to be done by an admin. Inconvenient - especially in the beginning when we still make a lot of changes.

A completely different solution would be to ask the users to install it into their own package libraries and create an own alias. We could provide an install script to do this automatically.

jfy133 commented 4 years ago

Honestly our cluster is a little bit of a mess. There was historically a common version of R and libraries (and I think there still is), however, because of a variety of reasons people started doing their own think at varying extents (own R package library in home, to full blown R environment in conda).

I guess we can do both once everything is stable, but until then I think you second solution is probably best.

nevrome commented 4 years ago

We just tested the tutorial with @TCLamnidis. Apart from pretty much exactly the same observations you, @jfy133, already mentioned in #17, we also realized that the sidora.cli is highly dependent on specific (up-to-date!) R package versions.

For the cluster setup we should probably go for a singularity container with all the necessary software.

jfy133 commented 4 years ago

Or conda? At least that's much easier to build...

nevrome commented 4 years ago

I'm more comfortable with singularity, but I'm open to every solution.

Doesn't conda require the user to manually switch to a certain environment? Could be more inconvenient, but I have no idea.

jfy133 commented 4 years ago

Not necessarily, as long as you configure the 'executable' to point to the correct paths that are installed in the conda environment.

It's much more involved to install singularity on a persons laptop, is my thinking. We can just offer both of course! I could make a conda environment this evening, no problem.

nevrome commented 4 years ago

We came up with a cluster release workflow based on singularity. It only lacks documentation.

jfy133 commented 4 years ago

Sadly it appears we won't be able to get the conda environment to work easily to loads of bizarre conflicts. I will think if there is another solution. Maybe just making a base R environment then a small script a user can run to install the dependencies packages.