search

redhat-developer / odo

odo - Developer-focused CLI for fast & iterative container-based application development on Podman and Kubernetes. Implementation of the open Devfile standard.
https://odo.dev
Apache License 2.0
794 stars 243 forks source link

Improvements to tests running on IBM Cloud #5274

Closed kadel closed 1 year ago

kadel commented 2 years ago

low priority optimizations

dharmit commented 2 years ago

allow running tests in parallel for multiple PRs

Counter-productive to focus on this one. Tests are fast at the moment because of the sequential nature. Making them parallel will slow things down.

Running parallel would also mean reserving nodes for certain tests.

Deploying a beefier cluster could be the way to achieve this. Even then, just 2 tests in parallel.

anandrkskd commented 2 years ago

As per the cabal discussion, we will be creating new cluster to run windwos and odo-init-image we will not Run tests in parallel and will focus on test cleanup and test enhancement

dharmit commented 2 years ago

focus on test cleanup

Adding some more context on this one. We decided in the Cabal discussion to prioritize a low priority item - auto-cancel running tests when new commits are pushed to PR. This is because if someone pushes a couple of commits within a short time (very common when we are nearing the end of the Sprint), it would be futile to keep running its existing test job. It would also lead to long wait time. Currently, in sequential approach, it takes ~30 minutes to test a PR. Parallelizing them would cause a single PR to take over an hour (correct me if I'm wrong @feloy)

rnapoles-rh commented 2 years ago

I found the following docs about communicating with Pipelines, although this relates to IBM toolcahins communicating with Jenkins, it might be a starting point https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-communicating-toolchains-jenkins

Docs about IBM Cloud Cli to interact with pipelines: https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-faq_delivery_pipeline

Still looking for options for re-running failed runs

rnapoles-rh commented 2 years ago

For auto-cancel running tests when new commits are pushed to PR: I found that the ibmcloud dev command has beta commands for DevOps but does not include a pipeline cancel subcommand. I contacted IBM Cloud support and they confirmed that there is no pipeline -cancel command available. So I opened a ticket with the feature request. The workaround is to manually cancel the running pipeline form the IBM Cloud Web UI.

Output of ibmcloud dev command ussage:

ibmcloud dev
NAME:
   ibmcloud dev - Create, develop, deploy, and monitor applications

USAGE:
   ibmcloud dev command [arguments...] [command options]

VERSION:
   2.10.1

COMMANDS:

App Commands
   build             Build the application in a local container
   code              Download the code from an application
   console           Opens the IBM Cloud console for an application
   create            Creates a new application and gives you the option to add services
   debug             Debug your application in a local container
   delete            Deletes an application from your resource group
   deploy            Deploy an application to IBM Cloud
   diag              This command displays version information about installed dependencies
   edit              Add or remove services for your application
   enable            [Beta] Add IBM Cloud files to an existing application
   get-credentials   Gets credentials required by the application to enable use of connected services
   list              List all IBM Cloud applications in a resource group
   run               Run your application in a local container
   shell             Open a shell into a local container
   status            Check the status of the containers used by the CLI
   stop              Stop a container
   test              Test your application in a local container
   view              View the URL of your application

DevOps Commands
   pipeline-get          [Beta] Get details for a pipeline
   pipeline-log          [Beta] Get logs for a pipeline, stage, job, or job execution
   pipeline-open         [Beta] View a pipeline in the browser
   pipeline-run          [Beta] Run a pipeline
   tekton-info           [Beta] Get details for a tekton pipeline
   tekton-logs           [Beta] Get logs for a tekton pipelinerun
   tekton-pipelinerun    [Beta] Get details for a tekton pipelinerun
   tekton-pipelineruns   [Beta] List the pipelineruns for a tekton pipeline
   tekton-trigger        [Beta] Trigger a tekton pipeline
   toolchain-delete      [Beta] Delete a toolchain
   toolchain-get         [Beta] Get details for a toolchain in a resource group
   toolchain-open        [Beta] View a toolchain in the browser
   toolchains            [Beta] List all toolchains in a resource group
rnapoles-rh commented 2 years ago

Blocked by #5487

kadel commented 1 year ago

there is still no solution for this, and it looks like we learned to cope without it /close

openshift-ci[bot] commented 1 year ago

@kadel: Closing this issue.

In response to [this](https://github.com/redhat-developer/odo/issues/5274#issuecomment-1422888028): >there is still no solution for this, and it looks like we learned to cope without it >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.