Handle unignored VERSION file

[pdf]

Unless I'm missing something, trying to use the method described in Adding autoversioning to a chef cookbook results in cookbooks being checked out of git having no version information available (for example, when using the cookbook-as-library model, or generally managing dependent cookbooks via berkshelf, served out of git).

One option might be to manage the VERSION file by updating and/or committing it prior to tagging if it's not ignored, similar to how some of the gem-related utilities handle it.

[capoferro]

It relies on using a continuous integration job to manage cookbook uploads, or relies on the user running at least thor version:current before performing an upload.

Our CI job checks out the cookbook, runs thor version:bump, then uploads the cookbook with the newly created VERSION file.

Our developers typically don't upload cookbooks directly (just like you would not upload a self built binary of software to an artifact server).

[pdf]

Can you share some more details about your workflow? Like, where are you uploading these cookbooks to, for example?

I think there's still an issue in some development scenarios, in that it's not possible to resolve a cookbook dependency from a git repository, but I'll think about that some more.

[bdangit]

@pdf just guessing here, but the workflow for each developer is to do cookbook unit testing via vagrant and check those patches/bug fixes/etc into git. If they have a cookbook dependency and it is private, they would refer to the cookbook that is stored into their Chef Server (not github). Then at some periodic time they have a CI server that as @bluepojo says "checks out the cookbook, runs thor version:bump, then uploads the cookbook [to their Chef Server] with the newly created VERSION file."

The github dependency is something I struggled with for a long time until this post came about; I'm assuming you kept getting version 0.0.0 showing up because VERSION did not exist.

BTW, here is what I use to do cookbook version bumps and then upload them to my Chef Server.

gist 6116374

[pdf]

@bdangit I suspected that was the workflow. With that said, there's nothing there that prevents having the VERSION under source control - the CI server can still bump as usual, the difference being that the repository can be pulled and is actually usable, rather than requiring post-processing every time it's cloned/pulled. And I still think there's a need for developers and/or users to be able to resolve deps via Berkshelf as expected, when the cookbook's primary source is a git repository, for example, for any public cookbook that's not on Opscode. On top of that, our policy is not to tag something until it's ready for QA, so we use explicit bumping on all our projects - for us, the CI server should never decide when a new version is to be created, that's the job of the developer/project lead/release manager. Tagging a new version for every commit or even merge is insane IMO.

[ivey]

@pdf It sounds like you shouldn't use thor-scmversion then.

When doing continuous delivery, every commit is potentially deployable, hence it needs a unique version.

To prevent potential conflicts and avoid having commit history littered with CI version bumps ever commit, we created thor-scmversion to specifically address versioning outside of a file in SCM.

If you're not doing CD you probably don't need this tool.

[pdf]

It seems like your workflow is a bit odd, even if you're doing CD, the only time you want to bump versions ought to be when you promote the build. Until then, any builds should be treated as snapshots, maybe using the commit hash as a suffix if you need a unique reference in the version number (and aren't explicitly tied to semver).

And I can't really see how polluting the repository with tags is better than commits, particularly when there's actually more pollution generated by the method you're employing than by fixing the workflow to tag at the right point.

Anyway, I'll close this issue since you guys seem pretty keen on keeping the scope focused on your particular workflow, and either fork this codebase or write something that makes more sense to me when I can allocate the time for it.

[ivey]

Feel free to submit a PR that adds the behavior, possibly behind an option or other guard. This just isn't going to be a priority for us, since we wrote the tool to do 1 thing, and it does that.

[capoferro]

@pdf Instead of bumping patch every commit, you could bump build # or a prerelease. In your case, I suppose a prerelease level bump would be most appropriate. The downside is that chef doesn't support version numbers with more precision than patch, so this approach won't work for Chef cookbooks.

I tend to agree that it's preferable not to bump version until you're ready to deploy something, however with Chef cookbooks not supporting complex version numbers we're stuck with bumping patch every testable set of changes. Our CI job only watches the master branch, so we don't bump until we've completed a feature and have merged to master. Merging to master is the same as saying "this is potentially releasable, so bump the version so we can promote it through our deployment pipeline."

[rberger]

If the VERSION file is not checked in to SCM, how does one deal with situations where developers may have a local test environment using something like chef-zero? The local chef-zero server is obviously not getting updated by a CI system. We still want a CI system to manage the master branch of our cookbooks. We also have a situation where we have multiple chef servers fo different reasons. Some of them may not be accessible from the CI server (though we can probably fix the CI->Chef server access one way or another).

So my core question is:

If I have a chef-repo with a berksfile pointing to cookbooks via git with a tag, where there is no VERSION file (i.e. cookbook version managed by thor-scmversion) in that repo, how can various tools know the version these cookbooks and get them into the local test chef server?

Even if I have a Berksfile line like:

cookbook "rm_dummy", git: "git@github.com:RMCookbooks/rm_dummy.git", tag: '0.1.0'

The Berskfile.lock file after berks install shows the version as 0.0.0

Is there a "best practices" use pattern for cases where there are multiple Chef Servers and/or using local test chef or chef-zero server and thor-scmversion that is compatible with using CI?

Or am I just missing something? Thanks, Rob

[ivey]

We run thor version:current to update the VERSION file when we need to do local testing that is version dependent.

You could also update your metadata’s version/rescue line periodically to return the latest major release # if there’s no VERSION file.

[yoshiwaan]

On a pretty similar tone to this, is there any reason that the tag for a version bump is created before the version file is updated?

If the version file was updated and then pushed up with the tag (if it's not excluded in the repo) then you can have a section in your metadata.rb for a chef cookbook saying to read the version from the VERSION file.

As it stands now the version in the version file is one below the tag created in github, which means that other work is needed to bump the version in the metadata.rb file as well.

Changing this in lib/thor-scmversion.rb:

begin
  say "Creating and pushing tags", :yellow
  current_version.tag
  say "Writing files: #{version_files.join(', ')}", :yellow
  write_version
  say "Tagged: #{current_version}", :green
rescue => e

To this:

begin
  say "Writing files: #{version_files.join(', ')}", :yellow
  write_version
  say "Creating and pushing tags", :yellow
  current_version.tag
  say "Tagged: #{current_version}", :green
rescue => e

should do the trick and not break any workflows

[yoshiwaan]

Hmm I thought this job did a commit before the tag for some reason... I'll see what I can come up with.

[yoshiwaan]

I also didn't realise that the tag was based off of git tags instead of the version file.

I created PR https://github.com/RiotGamesMinions/thor-scmversion/pull/41 for this. It adds new functionality without touching anything existing

[yoshiwaan]

Haha, your .gemspec file uses thorscmversion but there's no VERSION file checked in XD