Users are unable to download the EC2 credential from the NERC OpenStack Web Console.

[Milstein]

Steps:

Project >> API Access >> Download OpenStack RC File >> EC2 Credentials

Error Output:

[aabaris]

root disk on the active controller ctl-2 (10.255.0.8) is half the size of disk on the other 2. It has 4x900GB physical disks, unlike the other controllers that have 2x1.8TB disks. Virtual volumes are hw mirrors in all cases. ctl-2 has 2 virtual disks: 1 rootfs and the other "var_lib_ceph" each at 900GB.

[aabaris]

I restarted one of the containers:

[root@nerc-ctl-2 log]# df -h /
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda3       894G  894G   20K 100% /
[root@nerc-ctl-2 log]# systemctl |grep fileb
  container-nerc_filebeat.service                                                                                                      loaded active running   Podman container-nerc_filebeat.service                                                                                      
[root@nerc-ctl-2 log]# systemctl restart container-nerc_filebeat
[root@nerc-ctl-2 log]# df -h /
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda3       894G  470G  424G  53% /

System should now be working again, but both missing disk and ballooning container disk usage are concerning.

[jtriley]

@aabaris can we consider this issue closed? Do we want to open a new one for:

both missing disk and ballooning container disk usage are concerning.