search

ai4d-iasc / trixie

Scripts and documentation about trixie hpc
17 stars 4 forks source link

trixie cvmfs issue: failed to fetch file catalog #94

Closed nrcfieldsa closed 1 year ago

nrcfieldsa commented 1 year ago

There is presently errors being seen on trixie for new logins and existing logged in sessions related to environment-modules and CVMFS. The server soft.computecanada.ca is unreachable with local I/O failure error.

Investigating to attempt to determine the root cause.

nrcfieldsa commented 1 year ago

Workaround: For login shells press CTRL-C when starting a new shell. Regular commands may be impacted.

nrcfieldsa commented 1 year ago

This issue was due to an over-sized cache filling the root file system on trixie-hn1 only. I’ve cleared the space and resized the cache file system. cvmfs mount point was restarted.

There is now sufficient disk space free.

[root@trixie-hn1 /]# df -h / /mnt/cvmfs
Filesystem                                  Size  Used Avail Use% Mounted on

/dev/mapper/centos_trixie--hn1-root            50G  7.8G   39G  17% /
/dev/mapper/centos_trixie--hn1-cvmfs_cache  100G   40G   61G  40% /mnt/cvmfs

Please log-in and retry your processes. You may need to start tmux/screen session again.

SamuelLarkin commented 1 year ago

I'm now able to properly login Trixie. Thanks