cisco-ie / pipeline-gnmi

A Model-Driven Telemetry collector based on the open-source tool pipeline
Apache License 2.0
32 stars 6 forks source link

Add support for Go modules #5

Closed nleiva closed 5 years ago

nleiva commented 5 years ago

We need to migrate to Go modules (go.mod and go.sum). Pipeline is currently using Glide for dependency management which is no longer maintained.

In order to do so, we first need to remove the changes go generate makes to jsonpb as depicted in vendor.patch. EmitUInt64Unquoted is not part of the official protobuf library, so we need to replicate the Marshaler struct of github.com/golang/protobuf/jsonpb with this field on it. "A little copying is better than a little dependency".

I will first attempt to do this, before updating dependencies and bring Go 1.12 support as well.

remingtonc commented 5 years ago

@nleiva This appears to get more complex as we introduce modules. For example...

rm -rf vendor/
rm go.mod go.sum
export GO111MODULE=on
go mod init
go test -v ./

Errors out:

[pipeline-gnmi] go test -v ./                                                                                                               go-modules  ✗ ✭
# github.com/cisco-ie/pipeline-gnmi [github.com/cisco-ie/pipeline-gnmi.test]
./codec_gpb_test.go:33:3: unknown field 'EmitUInt64Unquoted' in struct literal of type jsonpb.Marshaler
./codec_gpb_test.go:39:3: unknown field 'EmitUInt64Unquoted' in struct literal of type jsonpb.Marshaler
FAIL    github.com/cisco-ie/pipeline-gnmi [build failed]

Fixing references:

# github.com/cisco-ie/pipeline-gnmi [github.com/cisco-ie/pipeline-gnmi.test]
./codec_gpb_test.go:35:3: cannot use promoted field Marshaler.EmitDefaults in struct literal of type Marshaler
./codec_gpb_test.go:36:3: cannot use promoted field Marshaler.OrigName in struct literal of type Marshaler
./codec_gpb_test.go:41:3: cannot use promoted field Marshaler.EmitDefaults in struct literal of type Marshaler
./codec_gpb_test.go:42:3: cannot use promoted field Marshaler.OrigName in struct literal of type Marshaler
FAIL    github.com/cisco-ie/pipeline-gnmi [build failed]
remingtonc commented 5 years ago

Fixed function itself but a test is now failing which appears related to the unmarshaling. Tracking in branch go-modules.

remingtonc commented 5 years ago

I am especially inclined to look at #15 with these jsonpb issues.

nleiva commented 5 years ago
[pipeline-gnmi] go test -v ./                                                                                                               go-modules  ✗ ✭
# github.com/cisco-ie/pipeline-gnmi [github.com/cisco-ie/pipeline-gnmi.test]
./codec_gpb_test.go:33:3: unknown field 'EmitUInt64Unquoted' in struct literal of type jsonpb.Marshaler
./codec_gpb_test.go:39:3: unknown field 'EmitUInt64Unquoted' in struct literal of type jsonpb.Marshaler
FAIL    github.com/cisco-ie/pipeline-gnmi [build failed]

Yeah, codec_gpb_test.go still references to github.com/golang/protobuf/jsonpb. I forgot to make the changes in the test files (to reference the local jsonpb struct instead), my bad. I will fix this in the next couple of days.

# github.com/cisco-ie/pipeline-gnmi [github.com/cisco-ie/pipeline-gnmi.test]
./codec_gpb_test.go:35:3: cannot use promoted field Marshaler.EmitDefaults in struct literal of type Marshaler
./codec_gpb_test.go:36:3: cannot use promoted field Marshaler.OrigName in struct literal of type Marshaler
./codec_gpb_test.go:41:3: cannot use promoted field Marshaler.EmitDefaults in struct literal of type Marshaler
./codec_gpb_test.go:42:3: cannot use promoted field Marshaler.OrigName in struct literal of type Marshaler
FAIL    github.com/cisco-ie/pipeline-gnmi [build failed]

We cannot access the embedded or promoted fields in a struct literal. You either do it like you did:

marshallerEmitString := &Marshaler{
    EmitUInt64Unquoted: false,
    Marshaler: jsonpb.Marshaler{
        EmitDefaults: true,
        OrigName:     true,
    },
}

or

marshallerEmitString := Marshaler{}
marshallerEmitString.EmitUInt64Unquoted = false
marshallerEmitString.EmitDefaults = true
marshallerEmitString.OrigName = true
nleiva commented 5 years ago

Fixed function itself but a test is now failing which appears related to the unmarshaling. Tracking in branch go-modules.

I'll take a look at it.