This issue documents some recent issues with running Sparrow on RHEL systems that are common at some universities. Many of these issues should be mentioned on a troubleshooting help page.
We successfully solved a compilation issue for the CLI (caused by Github actions) with #246 and #254, allowing us to install the Sparrow executable again.
On trying to run Sparrow, we get a mostly successful build, with a no space left on device error near the end:
------
> [frontend 2/5] COPY packages/sheet/package.json /app/packages/sheet/:
------
failed to solve with frontend dockerfile.v0: failed to build LLB: failed to load cache: failed to register layer: open /var/lib/docker/image/devicemapper/layerdb/tmp/write-set-324648261/diff: no space left on device
ERROR: Service 'frontend' failed to build : Build failed
ERROR: Service 'frontend' needs to be built, but --no-build was passed.
One or more containers did not build successfully, aborting.
The database/Python server application are running but the frontend is not.
Digging into this (looking at disk space with df -h) we learn that the /var folder (where Docker stores its files) is using almost all of its allotted space. This is a common problem with "hardened" servers that parcel the disk into space-limited volumes: /var is give relatively little space on the server relative to other functional groups, although Docker stores all internal files there. On this system, it gets 10GB (fairly standard I think).
docker system prune and docker system prune --volumes (after backing up the database out of caution!) reclaim relatively little space. This is sensible since Sparrow is basically the only application that has been run on this server; there aren't unused resources to reclaim.
One solution would be to just expand /var but this requires dealing with LVM which is really obnoxious.
Sparrow takes up a fair bit of space, but not 10 GB! Looking at the /var/lib/docker directory in more detail, it appears that most space is used in the devicemapper directory.
docker info reports that the devicemapper driver is deprecated, and asks that a --storage-opt be configured:
WARNING: the devicemapper storage-driver is deprecated, and will be removed in a future release.
WARNING: devicemapper: usage of loopback devices is strongly discouraged for production use.
Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.
I think this may be a RHEL-specific storage bug that needs to be solved at the Docker level. Ubuntu systems use overlay2 by default. It feels above my pay grade, especially since I basically never use RHEL systems. Any insight from RHEL users would be appreciated.
This issue documents some recent issues with running Sparrow on RHEL systems that are common at some universities. Many of these issues should be mentioned on a troubleshooting help page.
We successfully solved a compilation issue for the CLI (caused by Github actions) with #246 and #254, allowing us to install the Sparrow executable again.
On trying to run Sparrow, we get a mostly successful build, with a
no space left on deviceerror near the end:The database/Python server application are running but the frontend is not.
df -h) we learn that the/varfolder (where Docker stores its files) is using almost all of its allotted space. This is a common problem with "hardened" servers that parcel the disk into space-limited volumes:/varis give relatively little space on the server relative to other functional groups, although Docker stores all internal files there. On this system, it gets10GB(fairly standard I think).docker system pruneanddocker system prune --volumes(after backing up the database out of caution!) reclaim relatively little space. This is sensible since Sparrow is basically the only application that has been run on this server; there aren't unused resources to reclaim./varbut this requires dealing with LVM which is really obnoxious./var/lib/dockerdirectory in more detail, it appears that most space is used in thedevicemapperdirectory.docker inforeports that thedevicemapperdriver is deprecated, and asks that a--storage-optbe configured:I think this may be a RHEL-specific storage bug that needs to be solved at the Docker level. Ubuntu systems use
overlay2by default. It feels above my pay grade, especially since I basically never use RHEL systems. Any insight from RHEL users would be appreciated.