Conformance report not generated for Contour or Envoy-Gateway

[heyste]

Even though the option is set to save a report, none is written to disk.

https://github.com/apisnoop/gateway-api-test-framework/blob/545d4573ab15aa91407d8b748ff89848f9ba488e/implementation/contour.sh#L61-L67

[heyste]

After running the conformance tests for envoy-gateway it looks like there is a missing setting for generating the conformance reports.

PASS
ok      sigs.k8s.io/gateway-api/conformance     58.471s
/tmp/gtwfrwk-envoy-gateway

Conformance Suite completed.
envoy-gateway report saved: /tmp/conformance-suite-report-20240607-1533-envoy-gateway.yaml.

cat: /tmp/conformance-suite-report-20240607-1533-envoy-gateway.yaml: No such file or directory

Ref: https://github.com/kubernetes-sigs/gateway-api/blob/main/conformance/conformance.go

[BobyMCbobs]

@heyste, I remember running some tests and finding a file like /tmp/conformance.yaml. Do you see a file there? If so, the parameter of output might not be being respected.

[BobyMCbobs]

Hmm, it could be possible that the report output path isn't linked up correctly

• https://github.com/search?q=repo%3Akubernetes-sigs%2Fgateway-api+ReportOutputPath&type=code
• https://github.com/kubernetes-sigs/gateway-api/blob/main/conformance/utils/suite/suite.go#L175

[BobyMCbobs]

Istio is configuring to write to basedir (/tmp) + conformance.yaml https://github.com/istio/istio/blob/886b4d586cc6d514fbbb5646a082719803c6587b/tests/integration/pilot/gateway_conformance_test.go#L162

[BobyMCbobs]

v1.0 does not contain the code to write the report https://github.com/kubernetes-sigs/gateway-api/blob/release-1.0/conformance/conformance.go but v1.1 does https://github.com/kubernetes-sigs/gateway-api/blob/release-1.1/conformance/conformance.go#L130

this is likely why Istio's suite writes it manually https://github.com/istio/istio/blob/886b4d586cc6d514fbbb5646a082719803c6587b/tests/integration/pilot/gateway_conformance_test.go#L162