tektoncd / website

Tekton Website
https://tekton.dev
Apache License 2.0
62 stars 149 forks source link

How the same pipeline works in order? #416

Closed r67yfyt closed 1 year ago

r67yfyt commented 1 year ago

Expected Behavior

if I have a pipeline named "A", when I excute pipelineruns named "A-1" and "A-2", i want A-2 wait util A-1 end. how can i do that? By the way, Alibaba Flow can do that, here's the doc: https://help.aliyun.com/document_detail/153777.html

Actual Behavior

i can't figure out how to deal with the problem, hardly find a solution in official document.

Steps to Reproduce the Problem

  1. install pipeline, dashboard in k8s
  2. install buildpack pipeline and his little brothers
  3. create multiple pipelinerun with the same git repo, same params, same local cache dir, just all same
  4. high concurrency of the pipelinerun affect the cache consistency, assume that the cache process do two things:
    1. write new cache metadata or override old metadata, simply, i call this "METADATA"
    2. cache data, simply, i call this "CACHE" now, the probelm occurs: first, we assume the name of two cache processes is "A-1" and "A-2" second, i'll show the timeline each process do A-1: start "METADATA" A-2: start "METADATA" A-1: end "METADATA", start "CACHE" A-2: end "METADATA", start "CACHE"
      A-2: end "CACHE" A-1: end "CACHE" explicitly, we have the METADATA with A-2 AND CACHE with A-1, and we will not restore the cache next run
      just like the same problem about the consistency with cache and db when we deal with web applications.
AlanGreene commented 1 year ago

Thanks for raising this issue @r67yfyt

This is not supported out of the box in Tekton Pipelines today however you may be interested in https://github.com/tektoncd/community/pull/716 and https://hackmd.io/GK_1_6DWTvSiVHBL6umqDA which discuss possible approaches to managing concurrency and this kind of behaviour.

Closing this issue to avoid forking the discussion but feel free to add your comments / use cases at those links and continue the discussion there.