concourse / s3-resource

Concourse resource for interacting with AWS S3
Apache License 2.0
62 stars 106 forks source link

version number was not valid: Expected version '0.0.79+gdead53c+1525773154' to match version format #101

Open kalfa opened 6 years ago

kalfa commented 6 years ago

I use some S3 buckets, two examples in the image.

In the task creating the Pipfile I added a further information in one of the s3 stored data's version, which is the timestamp. So it changed from Pipfile-0.0.79+gdead53c.lock to Pipfile-0.0.79+gdead53c+1525773154.lock

The first filename matchs the regexp and everything worked nice. The second raises the error attached. I would think that Pipfile-(.*).lock would match both, but it doesn't. The build-package artefact which does not contain the timestamp still works.

pipevn-lock: 
  regexp: cicd/pipelines/((REPO))/artefacts/pipfiles/Pipfile-(.*)\.lock
  put file: pipenv-lock/Pipfile*.lock

built-package:
  regexp: cicd/pipelines/((REPO))/artefacts/packages/((PYPI_PACKAGE_BASE_FILE_NAME))-(.*)-.*\.whl
  put file: built-package/((PYPI_PACKAGE_BASE_FILE_NAME))-*.whl
screen shot 2018-05-08 at 11 01 31
HartS commented 6 years ago

@kalfa :

I just noticed this same issue with a path where we have two versioned resources. One of them was throwing the same error, and the other wasn't. My error looked like

resource script '/opt/resource/check []' failed: exit status 2

stderr:
panic: version number was not valid: Expected version '0.0.0-pre.29-develop~1' to match version format

goroutine 1 [running]:
github.com/concourse/s3-resource/versions.Extract(0xc420108ac0, 0x37, 0xc42006e420, 0x27, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /go/src/github.com/concourse/s3-resource/versions/versions.go:57 +0x36d
github.com/concourse/s3-resource/versions.GetBucketFileVersions(0xa66220, 0xc4200d7000, 0xc420074b00, 0x14, 0xc42006e450, 0x28, 0x0, 0x0, 0xc420074b20, 0x14, ...)
    /go/src/github.com/concourse/s3-resource/versions/versions.go:145 +0x213
github.com/concourse/s3-resource/check.(*CheckCommand).checkByRegex(0xc42018ff60, 0xc420074b00, 0x14, 0xc42006e450, 0x28, 0x0, 0x0, 0xc420074b20, 0x14, 0xc42006e420, ...)
    /go/src/github.com/concourse/s3-resource/check/check_command.go:33 +0x7c
github.com/concourse/s3-resource/check.(*CheckCommand).Run(0xc42018ff60, 0xc420074b00, 0x14, 0xc42006e450, 0x28, 0x0, 0x0, 0xc420074b20, 0x14, 0xc42006e420, ...)
    /go/src/github.com/concourse/s3-resource/check/check_command.go:26 +0xec
main.main()
    /go/src/github.com/concourse/s3-resource/cmd/check/main.go:32 +0x179

Since the other resource was working fine, I took a look at the filenames and noticed that the one which is erroring out was the only one which had a filename with a ~ in it ( the full filename was some-thing-0.0.0-pre.29-develop~1.tgz . After renaming it to remove the tilde, (to some-thing-0.0.0-pre.29-develop-1.tgz) the concourse resource started working, and picked up the latest (the file causing the problem wasn't even the latest) So assuming your issue is related, replacing the second + could fix it.

I'm not saying this isn't also a bug; I think it is. But trying the same workaround I used might help the concourse team to understand it better also

kalfa commented 6 years ago

@HartS thanks, good hint. I removed the timestamp completely now, also becuase it failed achieving what I wanted to achieve (idependently from the regex issue), but it's worth trying. Unable to do it now quickly, though.

If true, gives a workaround.

eedwards-sk commented 6 years ago

I haven't checked the source code, but supposedly the version needs to be semver compliant, which neither of the examples y'all posted were.

(except your original version X.Y.Z+1234 where the +SOANDSO is considered build metadata, but +SOANDSO+MORESTUFF would not)

kalfa commented 6 years ago

@eedwards-sk, that's a good reason. So "match version format" means the extracted version doesn't respect the semver syntax. In this case the error message could be more precise and show for example what's the syntax.

Expected $$$ to match SemVer version format

I thought it meant the version format I provided in the resource definition, rather the internal version syntax

TamimiGitHub commented 5 years ago

I had the same issue. Turns out I had a special character in name of the file ! once removed the resource worked fine

HartS commented 5 years ago

This just popped up in my notifications and I found the relevant clauses in the semver spec: https://semver.org/#spec-item-9 and https://semver.org/#spec-item-10

A pre-release version MAY be denoted by appending a hyphen and a series of dot separated identifiers immediately following the patch version. Identifiers MUST comprise only ASCII alphanumerics and hyphen [0-9A-Za-z-]. Identifiers MUST NOT be empty. Numeric identifiers MUST NOT include leading zeroes. Pre-release versions have a lower precedence than the associated normal version. A pre-release version indicates that the version is unstable and might not satisfy the intended compatibility requirements as denoted by its associated normal version. Examples: 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-0.3.7, 1.0.0-x.7.z.92.

Build metadata MAY be denoted by appending a plus sign and a series of dot separated identifiers immediately following the patch or pre-release version. Identifiers MUST comprise only ASCII alphanumerics and hyphen [0-9A-Za-z-]. Identifiers MUST NOT be empty. Build metadata SHOULD be ignored when determining version precedence. Thus two versions that differ only in the build metadata, have the same precedence. Examples: 1.0.0-alpha+001, 1.0.0+20130313144700, 1.0.0-beta+exp.sha.5114f85.

So basically, all characters after the major.minor.patch numbers should match [a-zA-Z-.]