'create-release' fails when path to release folder contains symlink

[ivandavidov]

Hello,

I believe there is a bug in the BOSH CLI tool version 5.0.1-2432e5e9-2018-07-18T21:41:06Z and in the newer versions as well.

Here is what we do:

1. Navigate to the folder where we have our BOSH release that we are trying to create.

   cd /home/apama/cloudfoundry/repos/cf-apama/cf-Apama

   Note that /home/apama/cloudfoundry is symlink to another folder:

   /home/apama/cloudfoundry => /FS/fslocal/cloudfoundry

2. Execute the following command: bosh2 create-release --name cf-apama --force

The expected output is to successfully generate new release. However, what we get as output is this failure:

Building a release from directory '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama':
  - Constructing packages from directory:
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/Zementis':
          Collecting package files:
            Missing files for pattern 'Zementis/Zementis.tar.gz'
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/apama':
          Collecting package files:
            Missing files for pattern 'apama/apama.tar.gz'
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/apama_broker':
          Collecting package files:
            Missing files for pattern 'apamabroker/apamabroker.tar.gz'
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/apama_common':
          Collecting package files:
            Missing files for pattern 'apamacommon/apamacommon.tar.gz'
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/apama_security_manager':
          Collecting package files:
            Missing files for pattern 'apama-security-manager/apama-security-manager.tar.gz'
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/apama_server':
          Collecting package files:
            Missing files for pattern 'apamaserver/apamaserver.tar.gz'
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/broker':
          Collecting package files:
            Missing files for pattern 'broker/common-servicebroker.tar.gz'
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/common':
          Collecting package files:
            Missing files for pattern 'common/common.tar.gz'
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/common_cfc':
          Collecting package files:
            Missing files for pattern 'common_cfc/common_cfc.tar.gz'
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/google-oauth':
          Collecting package files:
            Missing files for pattern 'oauth/google-oauth.tar.gz'
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/icons':
          Collecting package files:
            Missing files for pattern 'icons/*'
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/java8':
          Collecting package files:
            Missing files for pattern 'java8/jvm-1.8.tar.gz'
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/jetty':
          Collecting package files:
            Missing files for pattern 'jetty/jetty-jersey.tar.gz'
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/jnats':
          Collecting package files:
            Missing files for pattern 'jnats/jnats.tar.gz'
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/passman':
          Collecting package files:
            Missing files for pattern 'passman/common-passman.tar.gz'
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/plans':
          Collecting package files:
            Missing files for pattern 'plans/*'
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/scg':
          Collecting package files:
            Missing files for pattern 'scg/scg.tar.gz'
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/server':
          Collecting package files:
            Missing files for pattern 'server/common-serviceinstance.tar.gz'
      - Reading package from '/home/apama/cloudfoundry/repos/cf-apama/cf-Apama/packages/um':
          Collecting package files:
            Missing files for pattern 'umlib/umlib.tar.gz'
  - Constructing jobs from manifest:
      - Expected to find package 'java8' since it's a dependency of job 'apama_broker'
      - Expected to find package 'java8' since it's a dependency of job 'apama_server'

Exit code 1

The full strace output for this failure can be found here. The above mentioned error message can be found here. Please have in mind that the gist page contains 2 files and the upper one contains the error.

We observed that there is no problem to generate new release when we navigate to the release folder but without using symlinks:

1. Navigate to the folder where we have our BOSH release that we are trying to create.

   cd /FS/fslocal/cloudfoundry/repos/cf-apama/cf-Apama

   This time the folder path contains no symlinks.

2. Execute the same command as before: bosh2 create-release --name cf-apama --force

In this particular use case we see the following output:

Added dev release 'cf-apama/10.3.0.0+dev.8'

Name         cf-apama
Version      10.3.0.0+dev.8
Commit Hash  17e004d+

...jobs and packages are omitted...

19 packages

Succeeded

The full strace output of the successful execution can be found here.

We confirmed that the release generation fails even for very simple releases, e.g. @mariash's learn-bosh-release if there is symlink involved somewhere in the path to the release folder.

The BOSH CLI versions that we tested were 5.0.1 and the latest released 5.1.2. They both fail when we try to generate release with symlink in the path to the release folder. The issue is not observed with BOSH CLI version 2.0.28.

Could you please confirm whether or not this is a bug or expected behavior? In the latter case, is this behavior documented somewhere? Thank you!

Regards, Ivan Davidov

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/159993979

The labels on this github issue will be updated when the story is started.

[belinda-liu]

We believe this has been fixed by https://github.com/cloudfoundry/bosh-cli/commit/5f5f3d55c0c5160517cfa408dddfefefc9239994 as part of this story: https://www.pivotaltracker.com/story/show/159641167. Try using the latest CLI (currently 5.1.2) which contains the fix and re-open this if it's still an issue.

Best, @belinda-liu & @luan, CF BOSH

[ivandavidov]

The latest released version of the BOSH CLI is from 17-Aug-2018 (you can check here) and the commit you refer to seems to be from approximately 20-Aug-2018. I am not that good at math but to me it seems there is no way the fix could have been applied in the 17-Aug version of the BOSH CLI tool. Besides, in our team we already tried the 5.1.2 version and the issue is there.

[dpb587-pivotal]

You seem pretty good math, to me. Sorry for the confusion; the fix has not yet been included in a release. We'll keep this issue open until the release if you want to use it to track. Shouldn't be too long.

[mfine30]

Hey @ivandavidov, we just released another version of bosh-cli yesterday, v5.2.x which should resolve these issues. Please re-open this issue if it continues to be a problem.