Generate a unique index for every job to link as the public URL - Githubissues

instructlab / instructlab-bot

GitHub bot to assist with the taxonomy contribution workflow

Apache License 2.0

14 stars 16 forks source link

Generate a unique index for every job to link as the public URL #236

Closed nerdalert closed 4 months ago

nerdalert commented 4 months ago

Since multiple workers will be submitted an index for a PR, there would eventually be a scenario where one worker would overwrite another workers index. This is fine when it is one worker submitting all jobs in the index, but when it's one link per job it will overwrite itself as well as another worker would overwrite each other. It would also lose history of older jobs.

Something like:

publicURL := fmt.Sprintf("https://%s.s3.%s.amazonaws.com/%s", S3Bucket, AWSRegion, fmt.Sprintf("%s/%s", outDirName, filename))
// replaced with:
publicURL := fmt.Sprintf("https://%s.s3.%s.amazonaws.com/%s", S3Bucket, AWSRegion, fmt.Sprintf("%s/%s/%s", outDirName, jobID, filename))

Depends on #235