Open hpages opened 7 months ago
This is the setup I came up with:
Change the primary group of the pkgbuild
user to make it the same as the primary group of the biocbuild
user.
Check their primary groups (gid) with id biocbuild
and id pkgbuild
.
Assuming that biocbuild
's primary group is biocbuild
, make this the primary group of the pkgbuild
user with:
sudo usermod -g biocbuild pkgbuild
Then check their primary groups again.
An additional check is to check their effective group ids with id -g biocbuild
and id -g pkgbuild
. They should be the same.
Create basilisk's cache folder in /var/cache/
:
cd /var/cache
sudo mkdir basilisk
sudo chown biocbuild:biocbuild basilisk
sudo chmod 775 basilisk
Basic testing. Log as biocbuild
and do:
cd /var/cache/basilisk
mkdir titi
The folder should be writable by any member of the group. Check this with:
ls -ld titi
This should display something like:
drwxrwxr-x 2 biocbuild biocbuild 4096 Feb 8 01:04 titi
In particular the flags must be drwxrwxr-x
. If not, then set umask
to 002
in ~/.profile
, logout and login again for the change to take effect, and try to create another folder in /var/cache/basilisk/
.
Then log as pkgbuild
and do:
cd /var/cache/basilisk
mkdir toto
The folder should be writable by any member of the group:
ls -ld toto
This should display something like:
drwxrwxr-x 2 pkgbuild biocbuild 4096 Feb 8 01:03 toto
If not, then set umask
to 002
in ~/.profile
, logout and login again for the change to take effect, and try to create another folder in /var/cache/basilisk/
.
Finally clean /var/cache/basilisk/
with:
cd /var/cache/basilisk
rm -rf *
Set BASILISK_EXTERNAL_DIR
to /var/cache/basilisk
in ~/.profile
for biocbuild
and pkgbuild
by putting the following line at the end of each file:
export BASILISK_EXTERNAL_DIR="/var/cache/basilisk"
Do NOT put this in /etc/profile
!
IMPORTANT: This setting won't be effective on the SPB until the SPB server gets restarted (even if, in an interactive R session, basilisk.utils::getExternalDir()
points to the new location).
Next build run will repopulate /var/cache/basilisk/
.
Let's see how things go in the next couple of days. If everything goes fine, then we'll be able to remove ~biocbuild/.cache/R/basilisk
and ~pkgbuild/.cache/R/basilisk
.
Checking again a couple of days later that ~biocbuild/.cache/R/basilisk
and ~pkgbuild/.cache/R/basilisk
have not been recreated will be an extra confirmation that the BBS and the SPB are now both using the new cache location. If we are curious, we could even look at the ownership of the stuff under /var/cache/basilisk
, it should tell us who created what.
I'm just noting here that we'll want to try setting BASILISK_EXTERNAL_DIR
on merida1 next, where both biocbuild and pkgbuild are members of group staff
.
It is important that the group is the primary one for both users! Otherwise newly created subfolders won’t be shareable.
On Thu, 8 Feb 2024 at 17:14, jwokaty @.***> wrote:
I'm just noting here that we'll want to try setting BASILISK_EXTERNAL_DIR on merida1 next, where both biocbuild and pkgbuild are members of group staff.
— Reply to this email directly, view it on GitHub https://github.com/Bioconductor/BBS/issues/400#issuecomment-1934329564, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABYUQXTSKJWBKNIXJ2F3OTYSTTTZAVCNFSM6AAAAABC6ZIJPWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZUGMZDSNJWGQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>
It is important that the group is the primary one for both users! Otherwise newly created subfolders won’t be shareable.
@martin-g Absolutely!
I'm just noting here that we'll want to try setting BASILISK_EXTERNAL_DIR on merida1 next
@jwokaty That would be the next step.
I'm running into some gotchas with the new setup on nebbiolo1. The -00LOCK
files and other files (e.g. basilisk/1.15.4/0/pkgs/cache/47929eba.json
) under /var/cache/basilisk
are created with restrictive permissions that do not let the pkgbuild
user touch them. So despite the umask setting, it seems that when these files get created, their perms get explicitly restricted.
I'll investigate more.
Temporary hack (from a sudo account):
cd /var/cache/basilisk
sudo chmod --quiet -Rc g+rw .
sudo find . -type d -exec chmod --quiet -c go+rx {} \;
With this I can finally run R CMD build
and R CMD check
on sketchR as the pkgbuild
user.
Of course far from being satisfying because these commands would need to be re-run every time more stuff gets added to the cache.
Similar setup on merida1 (but using .bash_profile
instead of ~/.profile
, and replacing --quiet
with -f
in chmod
).
With this new setup sketchR passes R CMD build
and R CMD check
in 9m32s and 4m20s, respectively. The daily builds were not running though so it might take a little bit longer when they are.
I appreciate pointer; I did just get lucky. On Macs, you don't automatically get a group that is the same as your username, like Linux. If https://unix.stackexchange.com/questions/410367/how-to-get-the-primary-group-of-a-user#answer-410370 is correct, the primary group is staff
.
merida1:~ biocbuild$ id -gn biocbuild
staff
merida1:~ biocbuild$ id -gn pkgbuild
staff
Yes, we are lucky that on merida1, unlike on nebbiolo1, biocbuild
and pkgbuild
already had the same primary group. That was one less thing to change :four_leaf_clover: I don't think it would be too hard to change that if we had too though...
So I'm trying to find a satisfying solution for sharing basilisk's cache between the biocbuild (BBS) and pkgbuild (SPB) users on nebbiolo1.
For this to work, basilisk's cache and any subfolder would need to be writable by both users. A good place for such shared cache is
/var/cache/basilisk
(/var/cache
is kind of the standard system-level cache location on Linux). This can be controlled with theBASILISK_EXTERNAL_DIR
environment variable.Now it's easy to manually set the permissions on this folder so that both biocbuild and pkgbuild can write to it, this is a one time thing to do. But we also need to make sure that any subfolder that gets automatically created after that will be writable by both users. One way to achieve this maybe is by having biocbuild and pkgbuild belong to the same group (right now they don't), and by setting the umask value appropriately. I'm going to explore that approach a little.