search

pulumi / examples

Infrastructure, containers, and serverless apps to AWS, Azure, GCP, and Kubernetes... all deployed with Pulumi
https://www.pulumi.com
Apache License 2.0
2.36k stars 877 forks source link

Do not run benchmark prewarm in parallel with the benchmark #1584

Closed t0yv0 closed 7 months ago

t0yv0 commented 7 months ago

Looks like ProgramTest defaults to enabling t.Parallel(), and this defeats the purpose of prewarm phase in provider benchmarks. The intent of prewarm was to do a no-op to make sure dependencies are downloaded and avoid measuring that overhead as part of the benchmark. This was not the case though because prewarm was running in parallel with the actual benchmark.

After the change, each individual benchmark first does unmeasured prewarm, and then does the measurement. In the future we may want to measure prewarm as it is interesting for user experience as well.

pulumi-staging[bot] commented 7 months ago

:tropical_drink: The Update for pulumi/k8s-ci-cluster/dcb98b0ba0ac5801fe1b47555bcddeadccc49832-1687 was successful.

Resource Changes

    Name                                                          Type                                        Operation
+   multicloud                                                    pulumi-kubernetes:ci:GkeCluster             create
+   password                                                      random:index/randomPassword:RandomPassword  create
+   ephemeral-ci-cluster                                          gcp:container/cluster:Cluster               create
+   primary-node-pool                                             gcp:container/nodePool:NodePool             create
+   gke                                                           pulumi:providers:kubernetes                 create
+   k8s-ci-cluster-dcb98b0ba0ac5801fe1b47555bcddeadccc49832-1687  pulumi:pulumi:Stack                         create
iwahbe commented 7 months ago

Looks like the test failure is an instance of https://github.com/pulumi/pulumi-gcp/issues/1387. I closed the original issue due to a missing repro, but I am re-opening.

pulumi-staging[bot] commented 7 months ago

:tropical_drink: The Destroy for pulumi/k8s-ci-cluster/dcb98b0ba0ac5801fe1b47555bcddeadccc49832-1687 was successful.

Resource Changes

    Name                                                          Type                                        Operation
-   gke                                                           pulumi:providers:kubernetes                 delete
-   primary-node-pool                                             gcp:container/nodePool:NodePool             delete
-   ephemeral-ci-cluster                                          gcp:container/cluster:Cluster               delete
-   multicloud                                                    pulumi-kubernetes:ci:GkeCluster             delete
-   password                                                      random:index/randomPassword:RandomPassword  delete
-   k8s-ci-cluster-dcb98b0ba0ac5801fe1b47555bcddeadccc49832-1687  pulumi:pulumi:Stack                         delete