`bosh deploy` unnecessarily updates instance

[ansd]

Description: If two bosh releases in a deployment contain the same vendored package version, a redeploy (i.e. bosh deploy) following a successful first deploy will wrongly detect a package change and therefore unnecessarily update the instance.

Impact: Unnecessary downtime of bosh for no-op deployments (e.g. triggered by a pipeline)

Steps to reproduce:

1. Have a running inner bosh deployed with bosh release v266.6.0 and bosh-aws-cpi release v71.
2. re-upload the bosh and bosh-aws-cpi releases:

   $ bosh ur --fix https://bosh.io:443/d/github.com/cloudfoundry/bosh?v=266.6.0
   $ bosh ur --fix https://bosh.io:443/d/github.com/cloudfoundry-incubator/bosh-aws-cpi-release?v=71

Note that both releases use the same ruby package.

$ bosh inspect-release bosh/266.6.0                                                                                                                                                                                                                  
...
Package                                                      Compiled for  Blobstore ID                          Digest
ruby-2.4-r4/0cdc60ed7fdb326e605479e9275346200af30a25         (source)      9750a25c-9e34-42b8-bbdb-b2f5a72c3e81  3f1809b9d7e326d8a701de516fb51ea4245913c6
...

$ bosh inspect-release bosh-aws-cpi/71
...
Package                                                Compiled for  Blobstore ID                          Digest
ruby-2.4-r4/0cdc60ed7fdb326e605479e9275346200af30a25   (source)      d2cbe19c-b298-4952-80c8-76f6e11bebf4  3f1809b9d7e326d8a701de516fb51ea4245913c6
...

3. Redeploy (i.e. bosh deploy) inner bosh. Packages for bosh and bosh-aws-cpi will get compiled and the instance will get updated.

   
   $ bosh inspect-release bosh/266.6.0                                                                                                                                                                                                                  
   ...
   Package                                                      Compiled for  Blobstore ID                          Digest
   ruby-2.4-r4/0cdc60ed7fdb326e605479e9275346200af30a25         (source)               9750a25c-9e34-42b8-bbdb-b2f5a72c3e81  3f1809b9d7e326d8a701de516fb51ea4245913c6
   ~                                                            ubuntu-trusty/3586.26  d74c6edf-d189-4768-6b20-d99565ce62e9  b2bc10a286887764a7f289dbc51d40d178985b94
   ...

$ bosh inspect-release bosh-aws-cpi/71 ... Package Compiled for Blobstore ID Digest ruby-2.4-r4/0cdc60ed7fdb326e605479e9275346200af30a25 (source) d2cbe19c-b298-4952-80c8-76f6e11bebf4 3f1809b9d7e326d8a701de516fb51ea4245913c6 ~ ubuntu-trusty/3586.26 c3fcd6c9-2c64-4b70-677a-1fa2b93652c9 c00e539f8c8a80faf8ba9e80f429f3622157bcbf ...

4. Redeploy inner bosh *again*.
**desired behavior**: deploy will be a no-op, i.e. no updating instance, since nothing changed
**actual behavior**: deploy will update the instance:

Using environment 'https://10.0.3.12:25555' as client 'admin'

Using deployment 'bosh'

Release 'syslog/11.3.2' already exists.

Release 'ulimit/1' already exists.

Release 'bosh/266.6.0' already exists.

Release 'uaa/60' already exists.

Release 'credhub/1.9.5-sap.1' already exists.

Release 'bpm/0.8.0' already exists.

Release 'os-conf/20' already exists.

Release 'bosh-aws-cpi/71' already exists.

Release 'dynatrace-oneagent/1.0.3' already exists.

Task 856

Task 856 | 09:17:21 | Preparing deployment: Preparing deployment (00:00:01) Task 856 | 09:17:43 | Preparing package compilation: Finding packages to compile (00:00:00) Task 856 | 09:17:43 | Updating instance bosh: bosh/3276dfbe-4b81-4976-ad62-9baf13719d4c (0) (canary) (00:02:20)

Task 856 Started Thu Aug 2 09:17:21 UTC 2018 Task 856 Finished Thu Aug 2 09:20:03 UTC 2018 Task 856 Duration 00:02:42 Task 856 done

Succeeded

The reason why the instance is unnecessarily updated is that bosh detects a change for the ruby package (precisely: a change in the `blobstore_id`) although nothing changed. This can be observed in the deploy task debug output

bosh task 856 --debug ... D, [2018-08-02T09:17:43.222652 #5233] [task:856] DEBUG -- DirectorJobRunner: packages_changed? changed FROM: "ruby-2.4-r4" => {"name"=>"ruby-2.4-r4", "version"=>"0cdc60ed7fdb326e605479e9275346200af30a25.1", "sha1"=>"b2bc10a286887764a7f289dbc51d40d178985b94", "blobstore_id"=>"d74c6edf-d189-4768-6b20-d99565ce62e9"}

...

TO: "ruby-2.4-r4"=> {"name"=>"ruby-2.4-r4", "version"=>"0cdc60ed7fdb326e605479e9275346200af30a25.1", "sha1"=>"c00e539f8c8a80faf8ba9e80f429f3622157bcbf", "blobstore_id"=>"c3fcd6c9-2c64-4b70-677a-1fa2b93652c9"} ... D, [2018-08-02T09:17:43.244533 #5233] [task:856] DEBUG -- DirectorJobRunner: Need to update instance 'bosh/3276dfbe-4b81-4976-ad62-9baf13719d4c (0)', changes: "packages"

Note that the `blobstore_id` of the ruby package changed from the ruby package in the `bosh` release to the ruby package in the `bosh-aws-cpi` release (see output of `bosh inspect-release` above).

Subsequent redeploys of inner bosh (without any changes) result as expected in no-ops.

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/159492735

The labels on this github issue will be updated when the story is started.

[mfine30]

This behavior should be resolved in v268.0.1. I'm going to close this issue, but if it's still an issue with the newer version, please open a new/updated issue.