search

solo-io / gloo

The Feature-rich, Kubernetes-native, Next-Generation API Gateway Built on Envoy
https://docs.solo.io/
Apache License 2.0
4.1k stars 444 forks source link

performance_tests: No space left on device #9912

Open npolshakova opened 2 months ago

npolshakova commented 2 months ago

Which tests failed?

Main performance tests failed on nightlys: https://github.com/solo-io/gloo/actions/runs/10429716395/job/28887475657

Initial Investigation

Looks like we need to bump the runner or investigate some more into what can be slimmed down: 

2024-08-17T05:23:55.8186480Z Failed to compile helm:
2024-08-17T05:23:55.8191875Z 
2024-08-17T05:23:55.8195622Z # github.com/solo-io/gloo/test/kube2e/helm.test
2024-08-17T05:23:55.8204735Z /opt/hostedtoolcache/go/1.22.4/x64/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
2024-08-17T05:23:55.8216855Z /usr/bin/ld: final link failed: No space left on device
2024-08-17T05:23:55.8225829Z collect2: error: ld returned 1 exit status
2024-08-17T05:23:55.8228812Z 
2024-08-17T05:23:55.8228821Z 
2024-08-17T05:23:55.8232910Z Failed to compile helper:
2024-08-17T05:23:55.8235805Z 
2024-08-17T05:23:55.8243133Z # github.com/solo-io/gloo/test/kube2e/helper.test
2024-08-17T05:23:55.8246973Z /opt/hostedtoolcache/go/1.22.4/x64/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
2024-08-17T05:23:55.8257961Z /usr/bin/ld: $WORK/b001/helper.test: final close failed: No space left on device
2024-08-17T05:23:55.8263766Z collect2: error: ld returned 1 exit status
2024-08-17T05:23:55.8267869Z 
2024-08-17T05:23:55.8267877Z 
2024-08-17T05:23:55.8272272Z Failed to compile ingress:
2024-08-17T05:23:55.8284294Z 
2024-08-17T05:23:55.8287734Z # github.com/solo-io/gloo/test/kube2e/ingress.test
2024-08-17T05:23:55.8302033Z /opt/hostedtoolcache/go/1.22.4/x64/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
2024-08-17T05:23:55.8316190Z /usr/bin/ld: final link failed: No space left on device
2024-08-17T05:23:55.8321888Z collect2: error: ld returned 1 exit status
2024-08-17T05:23:55.8324112Z 
2024-08-17T05:23:55.8324119Z 
2024-08-17T05:23:58.0320458Z Could not open /home/runner/work/gloo/gloo/_output/projects_gloo_pkg_plugins_registry_test-report.json:
2024-08-17T05:23:58.0325676Z open /home/runner/work/gloo/gloo/_output/projects_gloo_pkg_plugins_registry_test-report.json: no such file or directory
2024-08-17T05:23:58.0329874Z Could not open /home/runner/work/gloo/gloo/_output/projects_gloo_pkg_plugins_registry_junit.xml:
2024-08-17T05:23:58.0337786Z open /home/runner/work/gloo/gloo/_output/projects_gloo_pkg_plugins_registry_junit.xml: no such file or directory
2024-08-17T05:23:58.0338439Z 
2024-08-17T05:23:58.0343128Z Ginkgo ran 165 suites in 19m3.319029722s
2024-08-17T05:23:58.0349480Z 
2024-08-17T05:23:58.0354304Z There were failures detected in the following suites:
2024-08-17T05:23:58.0366703Z      helm ./test/kube2e/helm [Compilation failure]
2024-08-17T05:23:58.0374953Z    helper ./test/kube2e/helper [Compilation failure]
2024-08-17T05:23:58.0385288Z   ingress ./test/kube2e/ingress [Compilation failure]

Additional Information

No response

npolshakova commented 2 months ago

Another nightly failure: https://github.com/solo-io/gloo/actions/runs/10447462579

npolshakova commented 2 months ago

Another nightly failure on 1.17: https://github.com/solo-io/gloo/actions/runs/10448006334/job/28927780690

npolshakova commented 2 months ago

A failure on main: https://github.com/solo-io/gloo/actions/runs/10483820806/job/29037161661

bewebi commented 3 days ago

This is occurring once more, for example here