Closed gaigepr closed 5 years ago
Another issue related to this, proteus and go-to-protobuf doesn’t seem to know how to generate proto import statements. This is part of why running proteus with --verbose
shows so many wanrings about ignoring certain types because their proto def could not be found.
@gaigepr -- I created a branch which changes the definition of the Workflow Item such that it is no longer an interface{}. The change is here:
https://github.com/jessesuen/argo/tree/protobuf
You can give that a try to see if it helps.
@jessesuen -- Here are some notes on my attempts:
I cloned your fork and checked out the protobuf
branch. Painless! Proteus is installed from my previous attempts so I did not have to do that.
$ pwd
/Users/gaige/go/src/github.com/jessesuen/argo
$ mkdir proto
$ proteus -f ./proto -p github.com/jessesuen/argo/pkg/apis/workflow/v1alpha1 --verbose
# no output, This usually means nothing happened
$ ls proto
# no output, the dir is empty.
$ proteus -f ./proto -p k8s.io/api/core/v1 --verbose
# no output
$ ls proto
# no output
Interestingly, if I do the following, I get proto generated but of course since it is generated from argoproj/argo
your changes to Item
hasn't been applied.
$ proteus -f ./proto -p github.com/argoproj/argo/pkg/apis/workflow/v1alpha1 --verbose
WARN: ignoring type interface{}
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Time" of package github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1 will be ignored because it was not present on the scan path.
WARN: type "Time" of package github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1 will be ignored because it was not present on the scan path.
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "ContainerPort" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "EnvFromSource" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "EnvVar" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "ResourceRequirements" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "VolumeMount" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "VolumeDevice" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Probe" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Probe" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Lifecycle" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "TerminationMessagePolicy" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "PullPolicy" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecurityContext" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "ContainerPort" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "EnvFromSource" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "EnvVar" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "ResourceRequirements" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "VolumeMount" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "VolumeDevice" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Probe" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Probe" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Lifecycle" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "TerminationMessagePolicy" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "PullPolicy" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecurityContext" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Affinity" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Container" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Toleration" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "UID" of package github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/types will be ignored because it was not present on the scan path.
WARN: type "Time" of package github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1 will be ignored because it was not present on the scan path.
WARN: type "Time" of package github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1 will be ignored because it was not present on the scan path.
WARN: type "OwnerReference" of package github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1 will be ignored because it was not present on the scan path.
WARN: type "Initializers" of package github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1 will be ignored because it was not present on the scan path.
WARN: type "Volume" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "PersistentVolumeClaim" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Affinity" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Toleration" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "LocalObjectReference" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Time" of package github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1 will be ignored because it was not present on the scan path.
WARN: type "Time" of package github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1 will be ignored because it was not present on the scan path.
WARN: type "Volume" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
INFO: Generated proto: proto/github.com/argoproj/argo/pkg/apis/workflow/v1alpha1/generated.proto
INFO: executing protoc: /usr/local/bin/protoc --proto_path=/Users/gaige/go/src:./proto:/Users/gaige/go/src/github.com/gogo/protobuf/protobuf:proto/github.com/argoproj/argo/pkg/apis/workflow/v1alpha1:.
WARN: ignoring type interface{}
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Time" of package github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1 will be ignored because it was not present on the scan path.
WARN: type "Time" of package github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1 will be ignored because it was not present on the scan path.
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecretKeySelector" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "ContainerPort" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "EnvFromSource" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "EnvVar" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "ResourceRequirements" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "VolumeMount" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "VolumeDevice" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Probe" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Probe" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Lifecycle" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "TerminationMessagePolicy" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "PullPolicy" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecurityContext" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "ContainerPort" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "EnvFromSource" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "EnvVar" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "ResourceRequirements" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "VolumeMount" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "VolumeDevice" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Probe" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Probe" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Lifecycle" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "TerminationMessagePolicy" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "PullPolicy" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "SecurityContext" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Affinity" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Container" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Toleration" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "UID" of package github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/types will be ignored because it was not present on the scan path.
WARN: type "Time" of package github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1 will be ignored because it was not present on the scan path.
WARN: type "Time" of package github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1 will be ignored because it was not present on the scan path.
WARN: type "OwnerReference" of package github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1 will be ignored because it was not present on the scan path.
WARN: type "Initializers" of package github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1 will be ignored because it was not present on the scan path.
WARN: type "Volume" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "PersistentVolumeClaim" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Affinity" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Toleration" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "LocalObjectReference" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: type "Time" of package github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1 will be ignored because it was not present on the scan path.
WARN: type "Time" of package github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1 will be ignored because it was not present on the scan path.
WARN: type "Volume" of package github.com/argoproj/argo/vendor/k8s.io/api/core/v1 will be ignored because it was not present on the scan path.
WARN: no RPCs in the given proto file, not generating anything
$ ls proto/
github.com
$ ls -lah proto/github.com/argoproj/argo/pkg/apis/workflow/v1alpha1/generated.proto
-rwxr-xr-x 1 gaige staff 12K May 22 12:29 proto/github.com/argoproj/argo/pkg/apis/workflow/v1alpha1/generated.proto
I am not really sure why proteus
won't generate proto from your fork of argo. I suspect it could have to do with package names / paths. I had some issues when I was trying to generate proto in my own fork of argo before I opened this issue.
Another experiment this time using go-to-protobuf
that doesn't quite succeed.
$ dep ensure -add github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway
Fetching sources...
$ bash ./hack/generate-proto.sh
+ set -o errexit
+ set -o nounset
+ set -o pipefail
+++ dirname ./hack/generate-proto.sh
++ cd ./hack/..
++ pwd
+ PROJECT_ROOT=/Users/gaige/go/src/github.com/argoproj/argo
++ cd /Users/gaige/go/src/github.com/argoproj/argo
++ ls -d -1 ./vendor/k8s.io/code-generator
+ CODEGEN_PKG=./vendor/k8s.io/code-generator
+ PATH=/Users/gaige/go/src/github.com/argoproj/argo/dist:/Users/gaige/.cargo/bin:./node_modules/.bin:/Users/gaige/go/bin:/Users/gaige/.google/google-cloud-sdk/bin:/Users/gaige/.nvm/versions/node/v8.9.2/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
+ go build -i -o dist/go-to-protobuf ./vendor/k8s.io/code-generator/cmd/go-to-protobuf
+ go build -i -o dist/protoc-gen-gogo ./vendor/k8s.io/code-generator/cmd/go-to-protobuf/protoc-gen-gogo
+ PACKAGES=(github.com/argoproj/argo/pkg/apis/workflow/v1alpha1)
+ APIMACHINERY_PKGS=(+k8s.io/apimachinery/pkg/util/intstr +k8s.io/apimachinery/pkg/api/resource +k8s.io/apimachinery/pkg/runtime/schema +k8s.io/apimachinery/pkg/runtime k8s.io/apimachinery/pkg/apis/meta/v1 k8s.io/api/core/v1)
++ IFS=,
++ echo github.com/argoproj/argo/pkg/apis/workflow/v1alpha1
++ IFS=,
++ echo +k8s.io/apimachinery/pkg/util/intstr,+k8s.io/apimachinery/pkg/api/resource,+k8s.io/apimachinery/pkg/runtime/schema,+k8s.io/apimachinery/pkg/runtime,k8s.io/apimachinery/pkg/apis/meta/v1,k8s.io/api/core/v1
+ go-to-protobuf --logtostderr --go-header-file=/Users/gaige/go/src/github.com/argoproj/argo/hack/custom-boilerplate.go.txt --packages=github.com/argoproj/argo/pkg/apis/workflow/v1alpha1 --apimachinery-packages=+k8s.io/apimachinery/pkg/util/intstr,+k8s.io/apimachinery/pkg/api/resource,+k8s.io/apimachinery/pkg/runtime/schema,+k8s.io/apimachinery/pkg/runtime,k8s.io/apimachinery/pkg/apis/meta/v1,k8s.io/api/core/v1 --proto-import=./vendor
2018/05/22 14:02:23 github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/api/resource/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
2018/05/22 14:02:23 github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/runtime/schema/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
2018/05/22 14:02:23 github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/runtime/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
2018/05/22 14:02:23 github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto but not used.
github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
2018/05/22 14:02:24 github.com/argoproj/argo/vendor/k8s.io/api/core/v1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto but not used.
2018/05/22 14:02:26 github.com/argoproj/argo/pkg/apis/workflow/v1alpha1/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
github.com/argoproj/argo/pkg/apis/workflow/v1alpha1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto but not used.
github.com/argoproj/argo/pkg/apis/workflow/v1alpha1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto but not used.
+ go build -i -o dist/protoc-gen-gogofast ./vendor/github.com/gogo/protobuf/protoc-gen-gogofast
+ GOPROTOBINARY=gogofast
+ go build -i -o dist/protoc-gen-grpc-gateway ./vendor/github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway
# github.com/argoproj/argo/vendor/google.golang.org/genproto/googleapis/api/annotations
vendor/google.golang.org/genproto/googleapis/api/annotations/http.pb.go:63:26: undefined: proto.InternalMessageInfo
vendor/google.golang.org/genproto/googleapis/api/annotations/http.pb.go:349:30: undefined: proto.InternalMessageInfo
vendor/google.golang.org/genproto/googleapis/api/annotations/http.pb.go:614:39: undefined: proto.InternalMessageInfo
Another go
$ pwd ; git status
$ pwd ; git status
/Users/gaige/go/src/github.com/argoproj/argo
On branch protobuf
Your branch is up to date with 'origin/protobuf'.
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)
modified: Gopkg.lock
modified: Gopkg.toml
Untracked files:
(use "git add <file>..." to include in what will be committed)
hack/generate-proto.sh
no changes added to commit (use "git add" and/or "git commit -a")
Here are the changes made to the Gopkg.toml
$ git diff Gopkg.toml
diff --git a/Gopkg.toml b/Gopkg.toml
index 3976756..69484b1 100644
--- a/Gopkg.toml
+++ b/Gopkg.toml
@@ -5,6 +5,9 @@ required = [
"k8s.io/code-generator/cmd/lister-gen",
# required by packr build (which we don't import as a package)
"golang.org/x/sync/errgroup",
+ # required for proto generation
+ "github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway",
+ "github.com/golang/protobuf",
]
[[constraint]]
@@ -50,3 +53,11 @@ required = [
[[constraint]]
name = "github.com/spf13/cobra"
branch = "master"
+
+[[constraint]]
+ name = "github.com/grpc-ecosystem/grpc-gateway"
+ version = "1.4.0"
+
+[[constraint]]
+ name = "github.com/gogo/protobuf"
+ version = "0.5.0"
Now, lets try generating stuff again! I have made sure protoc, protoc-gen-go, et al. are updated.
$ bash ./hack/generate-proto.sh
+ set -o errexit
+ set -o nounset
+ set -o pipefail
+++ dirname ./hack/generate-proto.sh
++ cd ./hack/..
++ pwd
+ PROJECT_ROOT=/Users/gaige/go/src/github.com/argoproj/argo
++ cd /Users/gaige/go/src/github.com/argoproj/argo
++ ls -d -1 ./vendor/k8s.io/code-generator
+ CODEGEN_PKG=./vendor/k8s.io/code-generator
+ PATH=/Users/gaige/go/src/github.com/argoproj/argo/dist:/Users/gaige/.cargo/bin:./node_modules/.bin:/Users/gaige/go/bin:/Users/gaige/.google/google-cloud-sdk/bin:/Users/gaige/.nvm/versions/node/v8.9.2/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
+ go build -i -o dist/go-to-protobuf ./vendor/k8s.io/code-generator/cmd/go-to-protobuf
+ go build -i -o dist/protoc-gen-gogo ./vendor/k8s.io/code-generator/cmd/go-to-protobuf/protoc-gen-gogo
+ PACKAGES=(github.com/argoproj/argo/pkg/apis/workflow/v1alpha1)
+ APIMACHINERY_PKGS=(+k8s.io/apimachinery/pkg/util/intstr +k8s.io/apimachinery/pkg/api/resource +k8s.io/apimachinery/pkg/runtime/schema +k8s.io/apimachinery/pkg/runtime k8s.io/apimachinery/pkg/apis/meta/v1 k8s.io/api/core/v1)
++ IFS=,
++ echo github.com/argoproj/argo/pkg/apis/workflow/v1alpha1
++ IFS=,
++ echo +k8s.io/apimachinery/pkg/util/intstr,+k8s.io/apimachinery/pkg/api/resource,+k8s.io/apimachinery/pkg/runtime/schema,+k8s.io/apimachinery/pkg/runtime,k8s.io/apimachinery/pkg/apis/meta/v1,k8s.io/api/core/v1
+ go-to-protobuf --logtostderr --go-header-file=/Users/gaige/go/src/github.com/argoproj/argo/hack/custom-boilerplate.go.txt --packages=github.com/argoproj/argo/pkg/apis/workflow/v1alpha1 --apimachinery-packages=+k8s.io/apimachinery/pkg/util/intstr,+k8s.io/apimachinery/pkg/api/resource,+k8s.io/apimachinery/pkg/runtime/schema,+k8s.io/apimachinery/pkg/runtime,k8s.io/apimachinery/pkg/apis/meta/v1,k8s.io/api/core/v1 --proto-import=./vendor
2018/05/22 14:32:41 github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/api/resource/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
2018/05/22 14:32:41 github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/runtime/schema/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
2018/05/22 14:32:41 github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/runtime/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
2018/05/22 14:32:41 github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto but not used.
github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
2018/05/22 14:32:42 github.com/argoproj/argo/vendor/k8s.io/api/core/v1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto but not used.
2018/05/22 14:32:44 github.com/argoproj/argo/pkg/apis/workflow/v1alpha1/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
github.com/argoproj/argo/pkg/apis/workflow/v1alpha1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto but not used.
github.com/argoproj/argo/pkg/apis/workflow/v1alpha1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto but not used.
+ go build -i -o dist/protoc-gen-gogofast ./vendor/github.com/gogo/protobuf/protoc-gen-gogofast
+ GOPROTOBINARY=gogofast
+ go build -i -o dist/protoc-gen-grpc-gateway ./vendor/github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway
# github.com/argoproj/argo/vendor/google.golang.org/genproto/googleapis/api/annotations
vendor/google.golang.org/genproto/googleapis/api/annotations/http.pb.go:63:26: undefined: proto.InternalMessageInfo
vendor/google.golang.org/genproto/googleapis/api/annotations/http.pb.go:349:30: undefined: proto.InternalMessageInfo
vendor/google.golang.org/genproto/googleapis/api/annotations/http.pb.go:614:39: undefined: proto.InternalMessageInfo
It seems to exit as normal because of some undefined proto messages... But look!
$ git status
On branch protobuf
Your branch is up to date with 'origin/protobuf'.
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)
modified: Gopkg.lock
modified: Gopkg.toml
modified: pkg/apis/workflow/v1alpha1/types.go
Untracked files:
(use "git add <file>..." to include in what will be committed)
hack/generate-proto.sh
pkg/apis/workflow/v1alpha1/generated.pb.go
pkg/apis/workflow/v1alpha1/generated.proto
no changes added to commit (use "git add" and/or "git commit -a")
So let's look at what changed.
$ git diff pkg/apis/workflow/v1alpha1/types.go
diff --git a/pkg/apis/workflow/v1alpha1/types.go b/pkg/apis/workflow/v1alpha1/types.go
index c85d824..29c7238 100644
--- a/pkg/apis/workflow/v1alpha1/types.go
+++ b/pkg/apis/workflow/v1alpha1/types.go
@@ -53,215 +53,215 @@ const (
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
type Workflow struct {
metav1.TypeMeta `json:",inline"`
- metav1.ObjectMeta `json:"metadata"`
- Spec WorkflowSpec `json:"spec"`
- Status WorkflowStatus `json:"status"`
+ metav1.ObjectMeta `json:"metadata" protobuf:"bytes,1,opt,name=metadata"`
+ Spec WorkflowSpec `json:"spec" protobuf:"bytes,2,opt,name=spec"`
+ Status WorkflowStatus `json:"status" protobuf:"bytes,3,opt,name=status"`
}
// WorkflowList is list of Workflow resources
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
type WorkflowList struct {
metav1.TypeMeta `json:",inline"`
- metav1.ListMeta `json:"metadata"`
- Items []Workflow `json:"items"`
+ metav1.ListMeta `json:"metadata" protobuf:"bytes,1,opt,name=metadata"`
+ Items []Workflow `json:"items" protobuf:"bytes,2,rep,name=items"`
}
// WorkflowSpec is the specification of a Workflow.
type WorkflowSpec struct {
// Templates is a list of workflow templates used in a workflow
- Templates []Template `json:"templates"`
+ Templates []Template `json:"templates" protobuf:"bytes,1,rep,name=templates"`
// Entrypoint is a template reference to the starting point of the workflow
- Entrypoint string `json:"entrypoint"`
+ Entrypoint string `json:"entrypoint" protobuf:"bytes,2,opt,name=entrypoint"`
// Arguments contain the parameters and artifacts sent to the workflow entrypoint
// Parameters are referencable globally using the 'workflow' variable prefix.
// e.g. {{workflow.parameters.myparam}}
- Arguments Arguments `json:"arguments,omitempty"`
+ Arguments Arguments `json:"arguments,omitempty" protobuf:"bytes,3,opt,name=arguments"`
// ServiceAccountName is the name of the ServiceAccount to run all pods of the workflow as.
- ServiceAccountName string `json:"serviceAccountName,omitempty"`
+ ServiceAccountName string `json:"serviceAccountName,omitempty" protobuf:"bytes,4,opt,name=serviceAccountName"`
// Volumes is a list of volumes that can be mounted by containers in a workflow.
- Volumes []apiv1.Volume `json:"volumes,omitempty"`
+ Volumes []apiv1.Volume `json:"volumes,omitempty" protobuf:"bytes,5,rep,name=volumes"`
// VolumeClaimTemplates is a list of claims that containers are allowed to reference.
// The Workflow controller will create the claims at the beginning of the workflow
// and delete the claims upon completion of the workflow
- VolumeClaimTemplates []apiv1.PersistentVolumeClaim `json:"volumeClaimTemplates,omitempty"`
+ VolumeClaimTemplates []apiv1.PersistentVolumeClaim `json:"volumeClaimTemplates,omitempty" protobuf:"bytes,6,rep,name=volumeClaimTemplates"`
// Parallelism limits the max total parallel pods that can execute at the same time in a workflow
- Parallelism *int64 `json:"parallelism,omitempty"`
+ Parallelism *int64 `json:"parallelism,omitempty" protobuf:"varint,7,opt,name=parallelism"`
// Suspend will suspend the workflow and prevent execution of any future steps in the workflow
- Suspend *bool `json:"suspend,omitempty"`
+ Suspend *bool `json:"suspend,omitempty" protobuf:"varint,8,opt,name=suspend"`
// NodeSelector is a selector which will result in all pods of the workflow
// to be scheduled on the selected node(s). This is able to be overridden by
// a nodeSelector specified in the template.
- NodeSelector map[string]string `json:"nodeSelector,omitempty"`
+ NodeSelector map[string]string `json:"nodeSelector,omitempty" protobuf:"bytes,9,rep,name=nodeSelector"`
// Affinity sets the scheduling constraints for all pods in the workflow.
// Can be overridden by an affinity specified in the template
- Affinity *apiv1.Affinity `json:"affinity,omitempty"`
+ Affinity *apiv1.Affinity `json:"affinity,omitempty" protobuf:"bytes,10,opt,name=affinity"`
// Tolerations to apply to workflow pods.
- Tolerations []apiv1.Toleration `json:"tolerations,omitempty"`
+ Tolerations []apiv1.Toleration `json:"tolerations,omitempty" protobuf:"bytes,11,rep,name=tolerations"`
// ImagePullSecrets is a list of references to secrets in the same namespace to use for pulling any images
// in pods that reference this ServiceAccount. ImagePullSecrets are distinct from Secrets because Secrets
// can be mounted in the pod, but ImagePullSecrets are only accessed by the kubelet.
// More info: https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod
- ImagePullSecrets []apiv1.LocalObjectReference `json:"imagePullSecrets,omitempty"`
+ ImagePullSecrets []apiv1.LocalObjectReference `json:"imagePullSecrets,omitempty" protobuf:"bytes,12,rep,name=imagePullSecrets"`
// OnExit is a template reference which is invoked at the end of the
// workflow, irrespective of the success, failure, or error of the
// primary workflow.
- OnExit string `json:"onExit,omitempty"`
+ OnExit string `json:"onExit,omitempty" protobuf:"bytes,13,opt,name=onExit"`
}
# ... The rest is omitted for brevity but you get the idea
So that is interesting, it annotated our go structs! Now lets look at those other files that are new.
$ cat pkg/apis/workflow/v1alpha1/generated.proto
// This file was autogenerated by go-to-protobuf. Do not edit it manually!
syntax = 'proto2';
package github.com.argoproj.argo.pkg.apis.workflow.v1alpha1;
import "k8s.io/api/core/v1/generated.proto";
import "k8s.io/apimachinery/pkg/apis/meta/v1/generated.proto";
import "k8s.io/apimachinery/pkg/runtime/generated.proto";
import "k8s.io/apimachinery/pkg/runtime/schema/generated.proto";
import "k8s.io/apimachinery/pkg/util/intstr/generated.proto";
// Package-wide variables from generator "generated".
option go_package = "v1alpha1";
// Arguments to a template
message Arguments {
// Parameters is the list of parameters to pass to the template or workflow
repeated Parameter parameters = 1;
// Artifacts is the list of artifacts to pass to the template or workflow
repeated Artifact artifacts = 2;
}
// Artifact indicates an artifact to place at a specified path
message Artifact {
// name of the artifact. must be unique within a template's inputs/outputs.
optional string name = 1;
// Path is the container path to the artifact
optional string path = 2;
// mode bits to use on this file, must be a value between 0 and 0777
// set when loading input artifacts.
optional int32 mode = 3;
// From allows an artifact to reference an artifact from a previous step
optional string from = 4;
// ArtifactLocation contains the location of the artifact
optional ArtifactLocation artifactLocation = 5;
// GlobalName exports an output artifact to the global scope, making it available as
// '{{workflow.outputs.artifacts.XXXX}} and in workflow.status.outputs.artifacts
optional string globalName = 6;
}
// ArtifactLocation describes a location for a single or multiple artifacts.
// It is used as single artifact in the context of inputs/outputs (e.g. outputs.artifacts.artname).
// It is also used to describe the location of multiple artifacts such as the archive location
// of a single workflow step, which the executor will use as a default location to store its files.
message ArtifactLocation {
// S3 contains S3 artifact location details
optional S3Artifact s3 = 1;
// Git contains git artifact location details
optional GitArtifact git = 2;
// HTTP contains HTTP artifact location details
optional HTTPArtifact http = 3;
// Artifactory contains artifactory artifact location details
optional ArtifactoryArtifact artifactory = 4;
// Raw contains raw artifact location details
optional RawArtifact raw = 5;
}
// ArtifactoryArtifact is the location of an artifactory artifact
message ArtifactoryArtifact {
// URL of the artifact
optional string url = 1;
optional ArtifactoryAuth artifactoryAuth = 2;
}
// ArtifactoryAuth describes the secret selectors required for authenticating to artifactory
message ArtifactoryAuth {
// UsernameSecret is the secret selector to the repository username
optional k8s.io.api.core.v1.SecretKeySelector usernameSecret = 1;
// PasswordSecret is the secret selector to the repository password
optional k8s.io.api.core.v1.SecretKeySelector passwordSecret = 2;
}
// DAGTask represents a node in the graph during DAG execution
message DAGTask {
// Name is the name of the target
optional string name = 1;
// Name of template to execute
optional string template = 2;
// Arguments are the parameter and artifact arguments to the template
optional Arguments arguments = 3;
// Dependencies are name of other targets which this depends on
repeated string dependencies = 4;
}
// DAGTemplate is a template subtype for directed acyclic graph templates
message DAGTemplate {
// Target are one or more names of targets to execute in a DAG
optional string target = 1;
// Tasks are a list of DAG tasks
repeated DAGTask tasks = 2;
}
// GitArtifact is the location of an git artifact
message GitArtifact {
// Repo is the git repository
optional string repo = 1;
// Revision is the git commit, tag, branch to checkout
optional string revision = 2;
// UsernameSecret is the secret selector to the repository username
optional k8s.io.api.core.v1.SecretKeySelector usernameSecret = 3;
// PasswordSecret is the secret selector to the repository password
optional k8s.io.api.core.v1.SecretKeySelector passwordSecret = 4;
}
// HTTPArtifact allows an file served on HTTP to be placed as an input artifact in a container
message HTTPArtifact {
// URL of the artifact
optional string url = 1;
}
// Inputs are the mechanism for passing parameters, artifacts, volumes from one template to another
message Inputs {
// Parameters are a list of parameters passed as inputs
repeated Parameter parameters = 1;
// Artifact are a list of artifacts passed as inputs
repeated Artifact artifacts = 2;
}
// Item expands a single workflow step into multiple parallel steps
// The value of Item can be a map, string, bool, or number
//
// +protobuf=true
// +protobuf.options.(gogoproto.goproto_stringer)=false
// +k8s:openapi-gen=true
message Item {
optional int64 type = 1;
optional string numVal = 2;
optional bool boolVal = 3;
optional string strVal = 4;
map<string, ItemValue> mapVal = 5;
}
// +protobuf=true
// +protobuf.options.(gogoproto.goproto_stringer)=false
// +k8s:openapi-gen=true
message ItemValue {
optional int64 type = 1;
optional string numVal = 2;
optional bool boolVal = 3;
optional string strVal = 4;
}
// Pod metdata
message Metadata {
map<string, string> annotations = 1;
map<string, string> labels = 2;
}
// NodeStatus contains status information about an individual node in the workflow
// +k8s:openapi-gen=false
message NodeStatus {
// ID is a unique identifier of a node within the worklow
// It is implemented as a hash of the node name, which makes the ID deterministic
optional string id = 1;
// Name is unique name in the node tree used to generate the node ID
optional string name = 2;
// DisplayName is a human readable representation of the node. Unique within a template boundary
optional string displayName = 3;
// Type indicates type of node
optional string type = 4;
// TemplateName is the template name which this node corresponds to. Not applicable to virtual nodes (e.g. Retry, StepGroup)
optional string templateName = 5;
// Phase a simple, high-level summary of where the node is in its lifecycle.
// Can be used as a state machine.
optional string phase = 6;
// BoundaryID indicates the node ID of the associated template root node in which this node belongs to
optional string boundaryID = 7;
// A human readable message indicating details about why the node is in this condition.
optional string message = 8;
// Time at which this node started
optional k8s.io.apimachinery.pkg.apis.meta.v1.Time startedAt = 9;
// Time at which this node completed
optional k8s.io.apimachinery.pkg.apis.meta.v1.Time finishedAt = 10;
// PodIP captures the IP of the pod for daemoned steps
optional string podIP = 11;
// Daemoned tracks whether or not this node was daemoned and need to be terminated
optional bool daemoned = 12;
// Inputs captures input parameter values and artifact locations supplied to this template invocation
optional Inputs inputs = 13;
// Outputs captures output parameter values and artifact locations produced by this template invocation
optional Outputs outputs = 14;
// Children is a list of child node IDs
repeated string children = 15;
// OutboundNodes tracks the node IDs which are considered "outbound" nodes to a template invocation.
// For every invocation of a template, there are nodes which we considered as "outbound". Essentially,
// these are last nodes in the execution sequence to run, before the template is considered completed.
// These nodes are then connected as parents to a following step.
//
// In the case of single pod steps (i.e. container, script, resource templates), this list will be nil
// since the pod itself is already considered the "outbound" node.
// In the case of DAGs, outbound nodes are the "target" tasks (tasks with no children).
// In the case of steps, outbound nodes are all the containers involved in the last step group.
// NOTE: since templates are composable, the list of outbound nodes are carried upwards when
// a DAG/steps template invokes another DAG/steps template. In other words, the outbound nodes of
// a template, will be a superset of the outbound nodes of its last children.
repeated string outboundNodes = 16;
}
// Outputs hold parameters, artifacts, and results from a step
message Outputs {
// Parameters holds the list of output parameters produced by a step
repeated Parameter parameters = 1;
// Artifacts holds the list of output artifacts produced by a step
repeated Artifact artifacts = 2;
// Result holds the result (stdout) of a script template
optional string result = 3;
}
// Parameter indicate a passed string parameter to a service template with an optional default value
message Parameter {
// Name is the parameter name
optional string name = 1;
// Default is the default value to use for an input parameter if a value was not supplied
optional string default = 2;
// Value is the literal value to use for the parameter.
// If specified in the context of an input parameter, the value takes precedence over any passed values
optional string value = 3;
// ValueFrom is the source for the output parameter's value
optional ValueFrom valueFrom = 4;
// GlobalName exports an output parameter to the global scope, making it available as
// '{{workflow.outputs.parameters.XXXX}} and in workflow.status.outputs.parameters
optional string globalName = 5;
}
// RawArtifact allows raw string content to be placed as an artifact in a container
message RawArtifact {
// Data is the string contents of the artifact
optional string data = 1;
}
// ResourceTemplate is a template subtype to manipulate kubernetes resources
message ResourceTemplate {
// Action is the action to perform to the resource.
// Must be one of: get, create, apply, delete, replace
optional string action = 1;
// Manifest contains the kubernetes manifest
optional string manifest = 2;
// SuccessCondition is a label selector expression which describes the conditions
// of the k8s resource in which it is acceptable to proceed to the following step
optional string successCondition = 3;
// FailureCondition is a label selector expression which describes the conditions
// of the k8s resource in which the step was considered failed
optional string failureCondition = 4;
}
// RetryStrategy provides controls on how to retry a workflow step
message RetryStrategy {
// Limit is the maximum number of attempts when retrying a container
optional int32 limit = 1;
}
// S3Artifact is the location of an S3 artifact
message S3Artifact {
optional S3Bucket s3Bucket = 1;
// Key is the key in the bucket where the artifact resides
optional string key = 2;
}
// S3Bucket contains the access information required for interfacing with an S3 bucket
message S3Bucket {
// Endpoint is the hostname of the bucket endpoint
optional string endpoint = 1;
// Bucket is the name of the bucket
optional string bucket = 2;
// Region contains the optional bucket region
optional string region = 3;
// Insecure will connect to the service with TLS
optional bool insecure = 4;
// AccessKeySecret is the secret selector to the bucket's access key
optional k8s.io.api.core.v1.SecretKeySelector accessKeySecret = 5;
// SecretKeySecret is the secret selector to the bucket's secret key
optional k8s.io.api.core.v1.SecretKeySelector secretKeySecret = 6;
}
// ScriptTemplate is a template subtype to enable scripting through code steps
message ScriptTemplate {
optional k8s.io.api.core.v1.Container container = 1;
// Source contains the source code of the script to execute
optional string source = 2;
}
// Sidecar is a container which runs alongside the main container
message Sidecar {
optional k8s.io.api.core.v1.Container container = 1;
// MirrorVolumeMounts will mount the same volumes specified in the main container
// to the sidecar (including artifacts), at the same mountPaths. This enables
// dind daemon to partially see the same filesystem as the main container in
// order to use features such as docker volume binding
optional bool mirrorVolumeMounts = 2;
}
// SuspendTemplate is a template subtype to suspend a workflow at a predetermined point in time
message SuspendTemplate {
}
// Template is a reusable and composable unit of execution in a workflow
message Template {
// Name is the name of the template
optional string name = 1;
// Inputs describe what inputs parameters and artifacts are supplied to this template
optional Inputs inputs = 2;
// Outputs describe the parameters and artifacts that this template produces
optional Outputs outputs = 3;
// NodeSelector is a selector to schedule this step of the workflow to be
// run on the selected node(s). Overrides the selector set at the workflow level.
map<string, string> nodeSelector = 4;
// Affinity sets the pod's scheduling constraints
// Overrides the affinity set at the workflow level (if any)
optional k8s.io.api.core.v1.Affinity affinity = 5;
// Metdata sets the pods's metadata, i.e. annotations and labels
optional Metadata metadata = 6;
// Deamon will allow a workflow to proceed to the next step so long as the container reaches readiness
optional bool daemon = 7;
// Steps define a series of sequential/parallel workflow steps
repeated WorkflowStep steps = 8;
// Container is the main container image to run in the pod
optional k8s.io.api.core.v1.Container container = 9;
// Script runs a portion of code against an interpreter
optional ScriptTemplate script = 10;
// Resource template subtype which can run k8s resources
optional ResourceTemplate resource = 11;
// DAG template subtype which runs a DAG
optional DAGTemplate dag = 12;
// Suspend template subtype which can suspend a workflow when reaching the step
optional SuspendTemplate suspend = 13;
// Sidecars is a list of containers which run alongside the main container
// Sidecars are automatically killed when the main container completes
repeated Sidecar sidecars = 14;
// Location in which all files related to the step will be stored (logs, artifacts, etc...).
// Can be overridden by individual items in Outputs. If omitted, will use the default
// artifact repository location configured in the controller, appended with the
// <workflowname>/<nodename> in the key.
optional ArtifactLocation archiveLocation = 15;
// Optional duration in seconds relative to the StartTime that the pod may be active on a node
// before the system actively tries to terminate the pod; value must be positive integer
// This field is only applicable to container and script templates.
optional int64 activeDeadlineSeconds = 16;
// RetryStrategy describes how to retry a template when it fails
optional RetryStrategy retryStrategy = 17;
// Parallelism limits the max total parallel pods that can execute at the same time within the
// boundaries of this template invocation. If additional steps/dag templates are invoked, the
// pods created by those templates will not be counted towards this total.
optional int64 parallelism = 18;
// Tolerations to apply to workflow pods.
repeated k8s.io.api.core.v1.Toleration tolerations = 19;
}
// ValueFrom describes a location in which to obtain the value to a parameter
message ValueFrom {
// Path in the container to retrieve an output parameter value from in container templates
optional string path = 1;
// JSONPath of a resource to retrieve an output parameter value from in resource templates
optional string jsonPath = 2;
// JQFilter expression against the resource object in resource templates
optional string jqFilter = 3;
// Parameter reference to a step or dag task in which to retrieve an output parameter value from
// (e.g. '{{steps.mystep.outputs.myparam}}')
optional string parameter = 4;
}
// Workflow is the definition of a workflow resource
// +genclient
// +genclient:noStatus
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
message Workflow {
optional k8s.io.apimachinery.pkg.apis.meta.v1.ObjectMeta metadata = 1;
optional WorkflowSpec spec = 2;
optional WorkflowStatus status = 3;
}
// WorkflowList is list of Workflow resources
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
message WorkflowList {
optional k8s.io.apimachinery.pkg.apis.meta.v1.ListMeta metadata = 1;
repeated Workflow items = 2;
}
// WorkflowSpec is the specification of a Workflow.
message WorkflowSpec {
// Templates is a list of workflow templates used in a workflow
repeated Template templates = 1;
// Entrypoint is a template reference to the starting point of the workflow
optional string entrypoint = 2;
// Arguments contain the parameters and artifacts sent to the workflow entrypoint
// Parameters are referencable globally using the 'workflow' variable prefix.
// e.g. {{workflow.parameters.myparam}}
optional Arguments arguments = 3;
// ServiceAccountName is the name of the ServiceAccount to run all pods of the workflow as.
optional string serviceAccountName = 4;
// Volumes is a list of volumes that can be mounted by containers in a workflow.
repeated k8s.io.api.core.v1.Volume volumes = 5;
// VolumeClaimTemplates is a list of claims that containers are allowed to reference.
// The Workflow controller will create the claims at the beginning of the workflow
// and delete the claims upon completion of the workflow
repeated k8s.io.api.core.v1.PersistentVolumeClaim volumeClaimTemplates = 6;
// Parallelism limits the max total parallel pods that can execute at the same time in a workflow
optional int64 parallelism = 7;
// Suspend will suspend the workflow and prevent execution of any future steps in the workflow
optional bool suspend = 8;
// NodeSelector is a selector which will result in all pods of the workflow
// to be scheduled on the selected node(s). This is able to be overridden by
// a nodeSelector specified in the template.
map<string, string> nodeSelector = 9;
// Affinity sets the scheduling constraints for all pods in the workflow.
// Can be overridden by an affinity specified in the template
optional k8s.io.api.core.v1.Affinity affinity = 10;
// Tolerations to apply to workflow pods.
repeated k8s.io.api.core.v1.Toleration tolerations = 11;
// ImagePullSecrets is a list of references to secrets in the same namespace to use for pulling any images
// in pods that reference this ServiceAccount. ImagePullSecrets are distinct from Secrets because Secrets
// can be mounted in the pod, but ImagePullSecrets are only accessed by the kubelet.
// More info: https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod
repeated k8s.io.api.core.v1.LocalObjectReference imagePullSecrets = 12;
// OnExit is a template reference which is invoked at the end of the
// workflow, irrespective of the success, failure, or error of the
// primary workflow.
optional string onExit = 13;
}
// WorkflowStatus contains overall status information about a workflow
// +k8s:openapi-gen=false
message WorkflowStatus {
// Phase a simple, high-level summary of where the workflow is in its lifecycle.
optional string phase = 1;
// Time at which this workflow started
optional k8s.io.apimachinery.pkg.apis.meta.v1.Time startedAt = 2;
// Time at which this workflow completed
optional k8s.io.apimachinery.pkg.apis.meta.v1.Time finishedAt = 3;
// A human readable message indicating details about why the workflow is in this condition.
optional string message = 4;
// Nodes is a mapping between a node ID and the node's status.
map<string, NodeStatus> nodes = 5;
// PersistentVolumeClaims tracks all PVCs that were created as part of the workflow.
// The contents of this list are drained at the end of the workflow.
repeated k8s.io.api.core.v1.Volume persistentVolumeClaims = 6;
// Outputs captures output values and artifact locations produced by the workflow via global outputs
optional Outputs outputs = 7;
}
// WorkflowStep is a reference to a template to execute in a series of step
message WorkflowStep {
// Name of the step
optional string name = 1;
// Template is a reference to the template to execute as the step
optional string template = 2;
// Arguments hold arguments to the template
optional Arguments arguments = 3;
// WithItems expands a step into multiple parallel steps from the items in the list
repeated Item withItems = 4;
// WithParam expands a step into from the value in the parameter
optional string withParam = 5;
// When is an expression in which the step should conditionally execute
optional string when = 6;
}
So, partial progress but the bash script doesn't run to completion so I cannot attest to the quality of the proto.
Another experiment. This one removes references to grpc-gateway from the hack/generate-proto.sh
script.
$ bash hack/generate-proto.sh
+ set -o errexit
+ set -o nounset
+ set -o pipefail
+++ dirname hack/generate-proto.sh
++ cd hack/..
++ pwd
+ PROJECT_ROOT=/Users/gaige/go/src/github.com/argoproj/argo
++ cd /Users/gaige/go/src/github.com/argoproj/argo
++ ls -d -1 ./vendor/k8s.io/code-generator
+ CODEGEN_PKG=./vendor/k8s.io/code-generator
+ PATH=/Users/gaige/go/src/github.com/argoproj/argo/dist:/Users/gaige/.cargo/bin:./node_modules/.bin:/Users/gaige/go/bin:/Users/gaige/.google/google-cloud-sdk/bin:/Users/gaige/.nvm/versions/node/v8.9.2/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
+ go build -i -o dist/go-to-protobuf ./vendor/k8s.io/code-generator/cmd/go-to-protobuf
+ go build -i -o dist/protoc-gen-gogo ./vendor/k8s.io/code-generator/cmd/go-to-protobuf/protoc-gen-gogo
+ PACKAGES=(github.com/argoproj/argo/pkg/apis/workflow/v1alpha1)
+ APIMACHINERY_PKGS=(+k8s.io/apimachinery/pkg/util/intstr +k8s.io/apimachinery/pkg/api/resource +k8s.io/apimachinery/pkg/runtime/schema +k8s.io/apimachinery/pkg/runtime k8s.io/apimachinery/pkg/apis/meta/v1 k8s.io/api/core/v1)
++ IFS=,
++ echo github.com/argoproj/argo/pkg/apis/workflow/v1alpha1
++ IFS=,
++ echo +k8s.io/apimachinery/pkg/util/intstr,+k8s.io/apimachinery/pkg/api/resource,+k8s.io/apimachinery/pkg/runtime/schema,+k8s.io/apimachinery/pkg/runtime,k8s.io/apimachinery/pkg/apis/meta/v1,k8s.io/api/core/v1
+ go-to-protobuf --logtostderr --go-header-file=/Users/gaige/go/src/github.com/argoproj/argo/hack/custom-boilerplate.go.txt --packages=github.com/argoproj/argo/pkg/apis/workflow/v1alpha1 --apimachinery-packages=+k8s.io/apimachinery/pkg/util/intstr,+k8s.io/apimachinery/pkg/api/resource,+k8s.io/apimachinery/pkg/runtime/schema,+k8s.io/apimachinery/pkg/runtime,k8s.io/apimachinery/pkg/apis/meta/v1,k8s.io/api/core/v1 --proto-import=./vendor
2018/05/22 16:25:19 github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/api/resource/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
2018/05/22 16:25:19 github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/runtime/schema/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
2018/05/22 16:25:20 github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/runtime/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
2018/05/22 16:25:20 github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto but not used.
github.com/argoproj/argo/vendor/k8s.io/apimachinery/pkg/apis/meta/v1/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
2018/05/22 16:25:21 github.com/argoproj/argo/vendor/k8s.io/api/core/v1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto but not used.
2018/05/22 16:25:23 github.com/argoproj/argo/pkg/apis/workflow/v1alpha1/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
github.com/argoproj/argo/pkg/apis/workflow/v1alpha1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto but not used.
github.com/argoproj/argo/pkg/apis/workflow/v1alpha1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto but not used.
+ go build -i -o dist/protoc-gen-gogofast ./vendor/github.com/gogo/protobuf/protoc-gen-gogofast
+ GOPROTOBINARY=gogofast
++ find /Users/gaige/go/src/github.com/argoproj/argo '(' -name '*.proto' -and -path '*/server/*' -or -path '*/reposerver/*' -and -name '*.proto' ')'
+ PROTO_FILES='/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/argoproj/argo-cd/reposerver/repository/repository.proto
/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/argoproj/argo-cd/server/application/application.proto
/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/argoproj/argo-cd/server/cluster/cluster.proto
/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/argoproj/argo-cd/server/repository/repository.proto
/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/argoproj/argo-cd/server/version/version.proto'
+ for i in '${PROTO_FILES}'
+ '[' -f /.dockerenv ']'
+ GOOGLE_PROTO_API_PATH=/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis
+ GOGO_PROTOBUF_PATH=/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/gogo/protobuf
+ protoc -I/Users/gaige/go/src/github.com/argoproj/argo -I/usr/local/include -I./vendor -I/Users/gaige/go/src -I/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis -I/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/gogo/protobuf --gogofast_out=plugins=grpc:/Users/gaige/go/src /Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/argoproj/argo-cd/reposerver/repository/repository.proto
vendor/github.com/argoproj/argo-cd/reposerver/repository/repository.proto: warning: Import k8s.io/api/core/v1/generated.proto but not used.
+ for i in '${PROTO_FILES}'
+ '[' -f /.dockerenv ']'
+ GOOGLE_PROTO_API_PATH=/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis
+ GOGO_PROTOBUF_PATH=/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/gogo/protobuf
+ protoc -I/Users/gaige/go/src/github.com/argoproj/argo -I/usr/local/include -I./vendor -I/Users/gaige/go/src -I/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis -I/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/gogo/protobuf --gogofast_out=plugins=grpc:/Users/gaige/go/src /Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/argoproj/argo-cd/server/application/application.proto
+ for i in '${PROTO_FILES}'
+ '[' -f /.dockerenv ']'
+ GOOGLE_PROTO_API_PATH=/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis
+ GOGO_PROTOBUF_PATH=/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/gogo/protobuf
+ protoc -I/Users/gaige/go/src/github.com/argoproj/argo -I/usr/local/include -I./vendor -I/Users/gaige/go/src -I/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis -I/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/gogo/protobuf --gogofast_out=plugins=grpc:/Users/gaige/go/src /Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/argoproj/argo-cd/server/cluster/cluster.proto
+ for i in '${PROTO_FILES}'
+ '[' -f /.dockerenv ']'
+ GOOGLE_PROTO_API_PATH=/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis
+ GOGO_PROTOBUF_PATH=/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/gogo/protobuf
+ protoc -I/Users/gaige/go/src/github.com/argoproj/argo -I/usr/local/include -I./vendor -I/Users/gaige/go/src -I/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis -I/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/gogo/protobuf --gogofast_out=plugins=grpc:/Users/gaige/go/src /Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/argoproj/argo-cd/server/repository/repository.proto
vendor/github.com/argoproj/argo-cd/server/repository/repository.proto: warning: Import k8s.io/api/core/v1/generated.proto but not used.
+ for i in '${PROTO_FILES}'
+ '[' -f /.dockerenv ']'
+ GOOGLE_PROTO_API_PATH=/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis
+ GOGO_PROTOBUF_PATH=/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/gogo/protobuf
+ protoc -I/Users/gaige/go/src/github.com/argoproj/argo -I/usr/local/include -I./vendor -I/Users/gaige/go/src -I/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis -I/Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/gogo/protobuf --gogofast_out=plugins=grpc:/Users/gaige/go/src /Users/gaige/go/src/github.com/argoproj/argo/vendor/github.com/argoproj/argo-cd/server/version/version.proto
YAHOO! The script runs to completion and does not complain about anything being undefined.
Is this a BUG REPORT or FEATURE REQUEST?:
This is a FEATURE REQUEST. At present, I can successfully use proteus to generate a
.proto
fromargoproj/argo/pkg/apis/workflow/v1alpha1
but there are a few problems left to solve.The various
k8s.io
packages that argo imports must also have.proto
files generated if the.proto
generated for argo itself is going to be a complete representation of the go definitions. This has not worked so far in my experiments. Oftenproteus
segfaults when trying to run onk8s.io
packages and other times it generates mostly empty.proto
files.You can try too:
Take a look inside
/home/user/go/src/github.com/argoproj/argo/proto
for what it outputs, if it even runs...Here is what I have been doing to generate
.proto
from argo'sworkflow
package.This produces a
.proto
file under./proto/github.com/argoproj/argo/pkg/apis/workflow/v1alpha1/generated.proto
which is great! It also generates apkg/apis/workflow/v1alpha1/generated.pb.go
.However, there are a few problems. Not all the
message
s have all their fields. For example,v1alpha1.Item
is defined as follows:But the resulting message lacks any fields... This has to do with an incompatibility between go's
interface{}
type and protoc (including gogo/protoc). One solution might be to use theAny
proto type from google. See this stackoverflow thread.How to reproduce it (as minimally and precisely as possible):
./proto
For what its worth, I also tried using go-to-protobuf and had essentially 0 success.
Environment: