search

Skarlso / crd-to-sample-yaml

Generate a sample YAML file from a CRD and view it rendered on a nice website
https://crdtoyaml.com/
Apache License 2.0
131 stars 17 forks source link

tests on main FAIL #104

Closed haarchri closed 2 months ago

haarchri commented 2 months ago

for this PR i tried to execute the test suite:

crd-to-sample-yaml git:(main) ./cty test sample-tests
+--------+-----------------------------------+---------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------+
| STATUS | IT                                | MATCHER       | ERROR                                                                            | TEMPLATE                                                           |
+--------+-----------------------------------+---------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------+
| PASS   | matches AWSCluster crds correctly | matchSnapshot |                                                                                  | sample-tests/crds/infrastructure.cluster.x-k8s.io_awsclusters.yaml |
| PASS   | matches AWSCluster crds correctly | matchSnapshot |                                                                                  | sample-tests/crds/infrastructure.cluster.x-k8s.io_awsclusters.yaml |
| PASS   | matches AWSCluster crds correctly | matchString   |                                                                                  | sample-tests/crds/infrastructure.cluster.x-k8s.io_awsclusters.yaml |
| FAIL   | matches AWSCluster crds correctly | matchString   | matcher returned failure: failed to validate kind AWSCluster and version v1beta2 | sample-tests/crds/infrastructure.cluster.x-k8s.io_awsclusters.yaml |
|        |                                   |               | : spec.controlPlaneEndpoint.host in body must be of type string: "number"        |                                                                    |
| PASS   | matches bootstrap crds correctly  | matchSnapshot |                                                                                  | sample-tests/crds/bootstrap_crd.yaml                               |
| PASS   | matches bootstrap crds correctly  | matchSnapshot |                                                                                  | sample-tests/crds/bootstrap_crd.yaml                               |
| PASS   | matches some custom stuff         | matchString   |                                                                                  | sample-tests/crds/bootstrap_crd.yaml                               |
+--------+-----------------------------------+---------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------+

Tests total: 7, failed: 1, passed: 6
crd-to-sample-yaml git:(main) ./cty test sample-tests
+--------+-----------------------------------+---------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------+
| STATUS | IT                                | MATCHER       | ERROR                                                                            | TEMPLATE                                                           |
+--------+-----------------------------------+---------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------+
| PASS   | matches AWSCluster crds correctly | matchSnapshot |                                                                                  | sample-tests/crds/infrastructure.cluster.x-k8s.io_awsclusters.yaml |
| FAIL   | matches AWSCluster crds correctly | matchSnapshot | matcher returned failure: failed to validate kind AWSCluster and version v1beta1 | sample-tests/crds/infrastructure.cluster.x-k8s.io_awsclusters.yaml |
|        |                                   |               | : spec in body must be of type object: "null"                                    |                                                                    |
|        |                                   |               | failed to validate kind AWSCluster and version v1beta2: spec in body must be of  |                                                                    |
|        |                                   |               | type object: "null"                                                              |                                                                    |
| PASS   | matches AWSCluster crds correctly | matchString   |                                                                                  | sample-tests/crds/infrastructure.cluster.x-k8s.io_awsclusters.yaml |
| FAIL   | matches AWSCluster crds correctly | matchString   | matcher returned failure: failed to validate kind AWSCluster and version v1beta2 | sample-tests/crds/infrastructure.cluster.x-k8s.io_awsclusters.yaml |
|        |                                   |               | : spec.controlPlaneEndpoint.host in body must be of type string: "number"        |                                                                    |
| PASS   | matches bootstrap crds correctly  | matchSnapshot |                                                                                  | sample-tests/crds/bootstrap_crd.yaml                               |
| FAIL   | matches bootstrap crds correctly  | matchSnapshot | matcher returned failure: failed to validate kind Bootstrap and version v1alpha1 | sample-tests/crds/bootstrap_crd.yaml                               |
|        |                                   |               | : spec.source in body must be of type object: "null"                             |                                                                    |
| PASS   | matches some custom stuff         | matchString   |                                                                                  | sample-tests/crds/bootstrap_crd.yaml                               |
+--------+-----------------------------------+---------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------+

Tests total: 7, failed: 3, passed: 4
Skarlso commented 2 months ago

Yes, those helm test failures are on purpose to showcase that they can fail as well. :D

But also, yeah, rendering objects for the minimal case is a bit flaky right now, because of how we parse things.

For example, if you require a minimal case, and an object is prepared to be rendered, but there are no properties for it, we can't close it with {} since we don't know beforehand that it will be an object with no values.

Either some kind of pre-lookup is required, or post tracking 🤔 I haven't figured it out yet. :/

Skarlso commented 2 months ago

Actually, I think I have a solution for this.