search

walter-cd / walter

Tiny deployment pipeline
http://walter-cd.net
Apache License 2.0
438 stars 39 forks source link

Inconsistency with __OUT, __COMBINED #162

Closed erophames closed 8 years ago

erophames commented 8 years ago

Good Day,

I have come across an error that is related to the special variable output. Namely when you use.

pipeline:
  -  stage: run_tests
     type: command
     command: go test ./...
  -  stage: run_linting
     type: command
     command: |
       echo "$(golint ./...)"
  -  stage: run_build
     type: command
     command: echo __COMBINED["run_linting"]

You will not get the echo on the last stage. However when you set the configuration up as the following.

pipeline:
  -  stage_name: run_tests
     type: command
     command: go test ./...
  -  stage_name: run_linting
     type: command
     command: |
       echo "$(golint ./...)"
  -  stage_name: run_build
     type: command
     command: echo __COMBINED["run_linting"]

The output will be the following, which is correct:

14:50:49  INFO [command] exec: run_linting
14:50:49  INFO [run_linting][command] exec output: domain/mapper.go:11:1: exported function ScheduleInstructionToDomain should have comment or be unexported
14:50:49  INFO [command] exec: run_build
14:50:49  INFO [run_build][command] exec output: domain/mapper.go:11:1: exported function ScheduleInstructionToDomain should have comment or be unexported

However you get the following warnings:

14:50:45  WARN property "stage_name" is deprecated. please use "stage" instead.
14:50:45  WARN found property "stage_name"
14:50:45  WARN property "stage_name" is deprecated. please use "stage" instead.
14:50:45  WARN found property "stage_name"
14:50:45  WARN property "stage_name" is deprecated. please use "stage" instead.

It would appear this depreciation warning was placed but not all the places the stage tag is used was updated.

mizzy commented 8 years ago

Thanks for reporting! I'll check it.

erophames commented 8 years ago

Thanks for the quick response. Another question, how would i prevent the next stage from executing based on the COMBINED output? I..e I only want the step to proceed if the COMBINED result was empty.

I see there is a basic idea of variables, are we able to set the state of these variables based on command executions.

For example if i execute golint and there are no issues, i'd like to set a variable say lint_clean and check that on the only_if tag in the next stage.

mizzy commented 8 years ago

Sorry, the warning message is incorrect. stage is wrong and name is correct.

So you should replace stage with name like this:

pipeline:
  -  name: run_tests
     type: command
     command: go test ./...
  -  stage: run_linting
     type: command
     command: |
       echo "$(golint ./...)"
  -  name: run_build
     type: command
     command: echo __COMBINED["run_linting"]

And you should get the echo result.

mizzy commented 8 years ago

Another question, how would i prevent the next stage from executing based on the COMBINED output? I..e I only want the step to proceed if the COMBINED result was empty.

How about this ?

  -  name: run_build
     type: command
     only_if: test -z __COMBINED["run_linting"]
     command: some build command
erophames commented 8 years ago

Thanks, I tried it. It works in principle however i get:

16:39:48  INFO [run_linting][command] exec output: domain/mapper.go:11:1: exported function ScheduleInstructionToDomain should have comment or be unexported
16:39:48  INFO [command] only_if: found "only_if" attribute in stage "run_build"
16:39:48  INFO [command] only_if: run_build
16:39:48  INFO [run_build][command] only_if output: sh: 1: test: domain/mapper.go:11:1:: unexpected operator

With the following:

pipeline:
  -  name: run_tests
     type: command
     command: go test ./...
  -  name: run_linting
     type: command
     command: |
       echo "$(golint ./...)"
  -  name: run_build
     type: command
     command: echo __COMBINED["run_linting"]
     only_if: test -z  __COMBINED["run_linting"]

Not entirely sure what unexpected operator is coming from...

mizzy commented 8 years ago

How about quoting like only_if: test -z '__COMBINED["run_linting"]' ?

erophames commented 8 years ago

No, that does not help. Now the stage is prevented regardless of the outcome.

19:14:21  INFO [command] exec: run_linting
19:14:22  INFO [run_linting][command] exec output: 
19:14:22  INFO [command] only_if: found "only_if" attribute in stage "run_build"
19:14:22  INFO [command] only_if: run_build
19:14:22  WARN [command] only_if err: exit status 1
19:14:22  WARN [command] exec: skipped this stage "run_build", since only_if condition failed
19:14:22  INFO [command] exec: 
19:14:22  INFO [command] exec: 
19:14:22  INFO Finished running pipeline process...
19:14:22  INFO Preparing to run cleanup process...
19:14:22  INFO Finished running cleanup process...
19:14:22  INFO succeded to finish Walter

I fixed the linting issue, so in essence the build process should execute now.

pipeline:
  -  name: run_tests
     type: command
     command: go test ./...
  -  name: run_linting
     type: command
     command: |
       echo "$(golint ./...)"
  -  name: run_build
     type: command
     command: echo __COMBINED["run_linting"]
     only_if: test -z  '__COMBINED["run_linting"]'
mizzy commented 8 years ago

It seems that __COMBINED["run_linting"] includes return character. It's not zero-length, so only_if condition fails.

erophames commented 8 years ago 

pipeline:
  -  name: run_tests
     type: command
     command: go test ./...
  -  name: run_linting
     type: command
     command: golint ./...
  -  name: run_build
     type: command
     command: echo __COMBINED["run_linting"]
     only_if: test "__COMBINED["run_linting"]" = ""

So this seems to work as expected. Passing -z to test causes other errors not related to the testable input.

With test -z:

12:07:10  INFO [command] exec: run_linting
12:07:10  INFO [command] only_if: found "only_if" attribute in stage "run_build"
12:07:10  INFO [command] only_if: run_build
12:07:10  INFO [run_build][command] only_if output: sh: 1: test: unexpected operator
12:07:10  WARN [command] only_if err: exit status 2
12:07:10  WARN [command] exec: skipped this stage "run_build", since only_if condition failed

Just test:

No lint issues:

12:08:03  INFO [command] exec: run_linting
12:08:03  INFO [command] only_if: found "only_if" attribute in stage "run_build"
12:08:03  INFO [command] only_if: run_build
12:08:03  INFO [command] exec: run_build
12:08:03  INFO [run_build][command] exec output: 
12:08:03  INFO [command] exec: 
12:08:03  INFO [command] exec:

With lint issues:

12:08:29  INFO [command] exec: run_linting
12:08:29  INFO [run_linting][command] exec output: domain/mapper.go:11:1: exported function ScheduleInstructionToDomain should have comment or be unexported
12:08:29  INFO [command] only_if: found "only_if" attribute in stage "run_build"
12:08:29  INFO [command] only_if: run_build
12:08:29  WARN [command] only_if err: exit status 1
12:08:29  WARN [command] exec: skipped this stage "run_build", since only_if condition failed
12:08:29  INFO [command] exec: 
12:08:29  INFO [command] exec:
mizzy commented 8 years ago

It's weird that -z does not work, but Walter works as expected now, I close this issue.

Thanks for reporting!