harness / gitness

Gitness is an Open Source developer platform with Source Control management, Continuous Integration and Continuous Delivery.
https://gitness.com
Apache License 2.0
31.96k stars 2.79k forks source link

Drone pipeline converter cache/memorizer? #3475

Closed nmapx closed 5 months ago

nmapx commented 5 months ago

I can't find anything related to pipeline cache in Drone docs but it seems there is such thing. I'm using a conversion extension (starlark - external one). When I update it's components Drone does not apply it in the pipelines. It's using the older version that was built by the extension before the change. It works if i trigger a different pipeline or just restart the drone server. Is it possible to force Drone to parse pipeline every time even if there is no change in the file (I mean the main .drone.yml/.drone.star)?

nmapx commented 5 months ago

Correct me if I'm wrong @bradrydzewski but I suspect this piece of code is the reason I'm looking for

package config

import (
    "context"
    "fmt"

    "github.com/drone/drone/core"

    lru "github.com/hashicorp/golang-lru"
    "github.com/sirupsen/logrus"
)

// cache key pattern used in the cache, comprised of the
// repository slug and commit sha.
const keyf = "%d|%d|%s|%s|%s|%s|%s"

// Memoize caches the conversion results for subsequent calls.
// This micro-optimization is intended for multi-pipeline
// projects that would otherwise covert the file for each
// pipeline execution.
func Memoize(base core.ConfigService) core.ConfigService {
    // simple cache prevents the same yaml file from being
    // requested multiple times in a short period.
    cache, _ := lru.New(10)
    return &memoize{base: base, cache: cache}
}

type memoize struct {
    base  core.ConfigService
    cache *lru.Cache
}
...

Any way to make it customized through env var? I would agree some degree of cache is necessary here but can't see any TTL set on this which creates more issues than it solves at least in my case.

bradrydzewski commented 5 months ago

You can set DRONE_CONVERT_PLUGIN_CACHE_SIZE=0 to disable caching. See relevant issue: https://github.com/harness/gitness/pull/3292#issuecomment-1332623285

nmapx commented 5 months ago

Thanks! Couldn't find this env var in the docs but it does the job! 🎉