devinrsmith
commented
1 month ago From Colin:
https://github.com/jlumbroso/free-disk-space
as potential ways to clean up at a runner environment level if necessary.
Open devinrsmith opened 1 month ago
devinrsmith
commented
1 month ago From Colin:
https://github.com/jlumbroso/free-disk-space
as potential ways to clean up at a runner environment level if necessary.
devinrsmith
commented
1 month ago The actual test image being created (deephaven/test-c-python-client-ticking-3.9-ubi-minimal:local-build) isn't too big:
The entrypoint code doesn't seem to be that big either:
$ docker run -it --name test --entrypoint bash deephaven/test-c-python-client-ticking-3.9-ubi-minimal:local-build
[root@66992a12062f project]# source /project/
3.9/ dep-wheels/ pyt-wheels/ tests/
[root@66992a12062f project]# source /project/3.9/bin/activate
(3.9) [root@66992a12062f project]# df -h .
Filesystem Size Used Avail Use% Mounted on
/dev/sdb3 930G 683G 246G 74% /
(3.9) [root@66992a12062f project]# df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/sdb3 930G 683G 246G 74% /
(3.9) [root@66992a12062f project]# pip install \
> --only-binary=":all:" \
> --find-links=/project/dep-wheels \
> --find-links=/project/pyt-wheels \
> --no-index \
> --no-deps \
> pydeephaven pydeephaven-ticking
Looking in links: /project/dep-wheels, /project/pyt-wheels
Processing ./dep-wheels/pydeephaven-0.36.0+snapshot-py3-none-any.whl
Processing ./pyt-wheels/pydeephaven_ticking-0.36.0+snapshot-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Installing collected packages: pydeephaven-ticking, pydeephaven
Successfully installed pydeephaven-0.36.0+snapshot pydeephaven-ticking-0.36.0+snapshot
(3.9) [root@66992a12062f project]# pip install \
> --only-binary=":all:" \
> --find-links=/project/dep-wheels \
> --find-links=/project/pyt-wheels \
> pydeephaven pydeephaven-ticking
Looking in links: /project/dep-wheels, /project/pyt-wheels
Requirement already satisfied: pydeephaven in ./3.9/lib/python3.9/site-packages (0.36.0+snapshot)
Requirement already satisfied: pydeephaven-ticking in ./3.9/lib/python3.9/site-packages (0.36.0+snapshot)
Collecting pyarrow (from pydeephaven)
Downloading pyarrow-17.0.0-cp39-cp39-manylinux_2_28_x86_64.whl.metadata (3.3 kB)
Collecting bitstring (from pydeephaven)
Downloading bitstring-4.2.3-py3-none-any.whl.metadata (5.0 kB)
Collecting grpcio (from pydeephaven)
Downloading grpcio-1.65.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.3 kB)
Collecting protobuf (from pydeephaven)
Downloading protobuf-5.27.3-cp38-abi3-manylinux2014_x86_64.whl.metadata (592 bytes)
Collecting bitarray<3.0.0,>=2.9.0 (from bitstring->pydeephaven)
Downloading bitarray-2.9.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (34 kB)
Collecting numpy>=1.16.6 (from pyarrow->pydeephaven)
Downloading numpy-2.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
Downloading bitstring-4.2.3-py3-none-any.whl (71 kB)
Downloading grpcio-1.65.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.7/5.7 MB 2.9 MB/s eta 0:00:00
Downloading protobuf-5.27.3-cp38-abi3-manylinux2014_x86_64.whl (309 kB)
Downloading pyarrow-17.0.0-cp39-cp39-manylinux_2_28_x86_64.whl (39.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 39.9/39.9 MB 1.6 MB/s eta 0:00:00
Downloading bitarray-2.9.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (286 kB)
Downloading numpy-2.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (19.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.5/19.5 MB 1.9 MB/s eta 0:00:00
Installing collected packages: bitarray, protobuf, numpy, grpcio, bitstring, pyarrow
Successfully installed bitarray-2.9.2 bitstring-4.2.3 grpcio-1.65.4 numpy-2.0.1 protobuf-5.27.3 pyarrow-17.0.0The bigger culprit may be the image builder image:
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
deephaven/test-c-python-client-ticking-3.9-ubi-minimal local-build af68b8e823ac 44 minutes ago 395MB
deephaven/py-client-ticking-wheel-cp39-cp39 local-build 0bb83c4024f6 46 minutes ago 1.5GB It's possible that we've relied on >14GB of disk space, and just more recently, GH CI has reduced the amount of disk space:
devinrsmith
commented
1 month ago
py-client-ticking tests were disabled in #5893 due to CI disk space related failures; it's conceivable that py-client-ticking isn't to blame, and it's just the straw that breaks CIs disk. We should do further investigation into the amount of disk space used in CI in general and proceed as appropriate; to either reduce disk space across a wider range of tasks, or reduce the disk space of py-client-ticking tests if it is to blame.