search

vmware-archive / pcf-pipelines

PCF Pipelines
Apache License 2.0
158 stars 283 forks source link

Using pcf pipelines v0.16 to deploy pcf #192

Closed filidav closed 6 years ago

filidav commented 6 years ago

Job upload and stage ert fails.

Error pcf-pipelines/tasks/stage-product/task.sh: line 19: ./pivnet-product/metadata.json: No such file or directory

Is this file part of the pivnet resource? I noticed the resource this job is using is: source: repository: czero/cflinuxfs2

I deployed a simple pipeline with

And this file exists.

Is there an error in this version?

cf-gitbot commented 6 years ago

We have created an issue in Pivotal Tracker to manage this. Unfortunately, the Pivotal Tracker project is private so you may be unable to view the contents of the story.

The labels on this github issue will be updated when the story is started.

z4ce commented 6 years ago

Hey @filidav I had a quick look at this. You are correct that the pivnet resource generates the metadata file. What is happening is since you aren't using the pivnet resource in your pipelines, this task.sh fails.

The upshot for PCF pipelines is that the method being used now ONLY works if the customer is getting their artifacts directly from pivnet. If they are using S3/artifactory/nexus/other they can't use use this task.

cc: @christianang

filidav commented 6 years ago

Thanks Ian

I guess the artifactory job is used as an input to this task and that is why it is failing?

I guess my only option is to go direct to pivnet and hardcode the version I want in the Param file. What version did this change? I was thinking of trying v15 of the pipelines with Artifactory .

David Filippelli

On Sep 22, 2017, at 5:16 PM, Ian Zink notifications@github.com<mailto:notifications@github.com> wrote:

Hey @filidavhttps://github.com/filidav I had a quick look at this. You are correct that the pivnet resource generates the metadata file. What is happening is since you aren't using the pivnet resource in your pipelines, this task.sh fails.

The upshot for PCF pipelines is that the method being used now ONLY works if the customer is getting their artifacts directly from pivnet. If they are using S3/artifactory/nexus/other they can't use use this task. cc: @christiananghttps://github.com/christianang

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/pivotal-cf/pcf-pipelines/issues/192#issuecomment-331562585, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AWJsOgjIbstYHRXkt0NfSVNs1EyFe5cEks5slCObgaJpZM4PhLvp.

STATEMENT OF CONFIDENTIALITY The information contained in this email message and any attachments may be confidential and legally privileged and is intended for the use of the addressee(s) only. If you are not an intended recipient, please: (1) notify me immediately by replying to this message; (2) do not use, disseminate, distribute or reproduce any part of the message or any attachment; and (3) destroy all copies of this message and any attachments.

christianang commented 6 years ago

I see. Not sure which version this changed in, but it does sound like we need to reevaluate how we get the version to support non-pivnet repositories.

svrc commented 6 years ago

@christianang @filidav My recommendation would be that when you populate Artifactory from pivnet, you also push the metadata.json file. Then also get that file on the downstream task. It contains the tile dependency information like stemcell versions. It might require an extra small bash task to move the file to appropriate directory.

abbyachau commented 6 years ago

Hi @filidav, just circling back to see if you were able to follow @svrc-pivotal's suggestion to make this work?

filidav commented 6 years ago

Yes we got it to work.

David Filippelli

On Oct 10, 2017, at 8:38 PM, abbyachau notifications@github.com<mailto:notifications@github.com> wrote:

Hi @filidavhttps://github.com/filidav, just circling back to see if you were able to follow @svrc-pivotalhttps://github.com/svrc-pivotal's suggestion to make this work?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/pivotal-cf/pcf-pipelines/issues/192#issuecomment-335647516, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AWJsOhBuP1m_FnOFvlS0juTQGSJ4U8QTks5srA4EgaJpZM4PhLvp.

STATEMENT OF CONFIDENTIALITY The information contained in this email message and any attachments may be confidential and legally privileged and is intended for the use of the addressee(s) only. If you are not an intended recipient, please: (1) notify me immediately by replying to this message; (2) do not use, disseminate, distribute or reproduce any part of the message or any attachment; and (3) destroy all copies of this message and any attachments.

abbyachau commented 6 years ago

Hi @filidav thanks for letting us know, much appreciated. @svrc-pivotal if you don't mind, we've added your recommendation to the Troubleshooting section on each of our install-pcf README pages.

@z4ce thanks for your help on this one. We don't plan on prioritising this issue because there appears to be a reasonable workaround. If you think otherwise, or if we hear there is a need from the field, we can review.

Closing this issue out as it is resolved but please do not hesitate to ping us/create a new issue if you have any feedback. Thanks!

z4ce commented 6 years ago

@filidav wasn't the workaround you used to actually use the older code that inspected the download? Or are you actually pushing the metadata.json into artifactory?

filidav commented 6 years ago

Pushing the metadata,json and then getting it.

In upgrade ERT pipeline

task.yml

platform: linux

image_resource: type: docker-image source: {repository: czero/cflinuxfs2}

inputs:

params:

run:

macC02T61C9H03Q:upgrade-ert filidav$ cat ../tasks/version-metadata/task.sh

!/bin/bash

set -eux

Copyright 2017-Present Pivotal Software, Inc. All rights reserved.

#

Licensed under the Apache License, Version 2.0 (the "License");

you may not use this file except in compliance with the License.

You may obtain a copy of the License at

#

http://www.apache.org/licenses/LICENSE-2.0

#

Unless required by applicable law or agreed to in writing, software

distributed under the License is distributed on an "AS IS" BASIS,

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

See the License for the specific language governing permissions and

limitations under the License.

function main() {

local cwd cwd="${1}"

cp ${cwd}/pivnet-product/metadata.json ${cwd}/pivnet-product-meta/metadata-$(date '+%s').json } main "${PWD}"

David Filippelli | Platform Engineer, Provisioning + Pipelines, Global Solutions Delivery 250 Bloor Street East | Toronto, Ontario, M4W 1E5 | M. 416-527-3343 David_Filippelli@manulife.commailto:David_Filippelli@manulife.com | manulife.comhttp://www.manulife.com/

Have a question? Want to know how we deliver? Check out our Playbookhttps://mfc.sharepoint.com/sites/GEO/playbook and be a part of the community!

[cid:image001.png@01D34294.DEFD94B0]

From: Ian Zink notifications@github.com Reply-To: pivotal-cf/pcf-pipelines reply@reply.github.com Date: Wednesday, October 11, 2017 at 1:13 PM To: pivotal-cf/pcf-pipelines pcf-pipelines@noreply.github.com Cc: David Filippelli David_Filippelli@manulife.com, Mention mention@noreply.github.com Subject: Re: [pivotal-cf/pcf-pipelines] Using pcf pipelines v0.16 to deploy pcf (#192)

@filidavhttps://github.com/filidav wasn't the workaround you used to actually use the older code that inspected the download? Or are you actually pushing the metadata.json into artifactory?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/pivotal-cf/pcf-pipelines/issues/192#issuecomment-335881407, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AWJsOiT-dr3lxL3dcQ8ykVvVs5z7k5GWks5srPdAgaJpZM4PhLvp.

STATEMENT OF CONFIDENTIALITY The information contained in this email message and any attachments may be confidential and legally privileged and is intended for the use of the addressee(s) only. If you are not an intended recipient, please: (1) notify me immediately by replying to this message; (2) do not use, disseminate, distribute or reproduce any part of the message or any attachment; and (3) destroy all copies of this message and any attachments.

svrc commented 6 years ago

@z4ce we're actually pushing metadata.json into artifactory in this case, to stay current

abbyachau commented 6 years ago

Thanks, follow up story to this issue is here: https://www.pivotaltracker.com/story/show/151900831.

dburkhalter-pivotal commented 6 years ago

https://network.pivotal.io/products/pcf-automation does not exist