search

grpc-ecosystem / grpc-gateway

gRPC to JSON proxy generator following the gRPC HTTP spec
https://grpc-ecosystem.github.io/grpc-gateway/
BSD 3-Clause "New" or "Revised" License
18.21k stars 2.24k forks source link

descriptor.File should not contain nil SourceCodeInfo #3542

Closed ebilling closed 1 year ago

ebilling commented 1 year ago

🐛 Bug Report

As of v2.17.0 we have started getting errors when running protoc-gen-openapiv2.

We get the message: “descriptor.File should not contain nil SourceCodeInfo” with no other details about the issue.

It seems to be coming from here: https://github.com/grpc-ecosystem/grpc-gateway/blob/main/protoc-gen-openapiv2/internal/genopenapi/template.go#L2333

If I revert back to v2.16.2, everything works fine. I can't tell if this is a problem or not.

To Reproduce

  1. protoc --plugin protoc-gen-openapiv2=protoc-gen-openapiv2 --openapiv2_out="./openapiV2" -I=./proto:$(GOOGLE_DIR)/api:$(ANNOTATIONS_DIR) *.proto

Expected behavior

Works like v2.16.2

Alternatively, tell us which file is having the issue.

Actual Behavior

results in a set of these messages for each proto file:

descriptor.File should not contain nil SourceCodeInfo descriptor.File should not contain nil SourceCodeInfo descriptor.File should not contain nil SourceCodeInfo descriptor.File should not contain nil SourceCodeInfo descriptor.File should not contain nil SourceCodeInfo descriptor.File should not contain nil SourceCodeInfo descriptor.File should not contain nil SourceCodeInfo descriptor.File should not contain nil SourceCodeInfo descriptor.File should not contain nil SourceCodeInfo

Your Environment

Mac OS - Darwin Kernel Version 22.6.0: Wed Jul 5 22:22:52 PDT 2023; root:xnu-8796.141.3~6/RELEASE_ARM64_T8103 arm64 go version go1.20.7 darwin/arm64

johanbrandhorst commented 1 year ago

Thanks for the bug report, this is definitely concerning. It doesn't seem to be happening in our own tests, so I'm not sure what could be going on here. This code also hasn't changed in the last 6 years as far as I can tell. What version of protoc are you using? I'll have a look again to see if any changes stand out to me but at the moment I'm at a loss.

ebilling commented 1 year ago

protoc --version libprotoc 23.2

ebilling commented 1 year ago

I'm forking the code to see if I can at least print the filename that is causing the problem and get more details about why this is happening.

johanbrandhorst commented 1 year ago

Ah, I expect it's https://github.com/grpc-ecosystem/grpc-gateway/commit/a0ab79a99c319d7ba9b0af39998189cb32401bf9

johanbrandhorst commented 1 year ago

Looks like we might need to explicitly set the source code info like we used to

johanbrandhorst commented 1 year ago

CC @aimuz

johanbrandhorst commented 1 year ago

Can you try building from set-source-code-info-explicitly? E.g.

$ go install github.com/grpc-ecosystem/grpc-gateway/v2/cmd/protoc-gen-grpc-gateway@set-source-code-info-explicitly
$ protoc ...
ebilling commented 1 year ago

That fixes the issue. Thank you.

ebilling commented 1 year ago

Just an FYI, I did my change to report the errant files and got this when running make install && make clean && make generate

go install github.com/bufbuild/buf/cmd/buf@v1.14.0
go install \
        ./protoc-gen-openapiv2 \
        ./protoc-gen-grpc-gateway
find . -type f -name '*.pb.go' -delete
find . -type f -name '*.swagger.json' -delete
find . -type f -name '*.pb.gw.go' -delete
rm -f examples/internal/clients/echo/client.go examples/internal/clients/echo/response.go examples/internal/clients/echo/configuration.go examples/internal/clients/echo/api_echo_service.go examples/internal/clients/echo/model_examplepb_simple_message.go examples/internal/clients/echo/model_examplepb_embedded.go examples/internal/clients/abe/model_a_bit_of_everything_nested.go examples/internal/clients/abe/api_a_bit_of_everything_service.go examples/internal/clients/abe/client.go examples/internal/clients/abe/api_camel_case_service_name.go examples/internal/clients/abe/configuration.go examples/internal/clients/abe/api_echo_rpc.go examples/internal/clients/abe/model_examplepb_a_bit_of_everything.go examples/internal/clients/abe/model_examplepb_a_bit_of_everything_repeated.go examples/internal/clients/abe/model_examplepb_body.go examples/internal/clients/abe/model_examplepb_numeric_enum.go examples/internal/clients/abe/model_examplepb_update_v2_request.go examples/internal/clients/abe/model_message_path_enum_nested_path_enum.go examples/internal/clients/abe/model_nested_deep_enum.go examples/internal/clients/abe/model_pathenum_path_enum.go examples/internal/clients/abe/model_protobuf_field_mask.go examples/internal/clients/abe/response.go examples/internal/clients/unannotatedecho/client.go examples/internal/clients/unannotatedecho/response.go examples/internal/clients/unannotatedecho/configuration.go examples/internal/clients/unannotatedecho/model_examplepb_unannotated_simple_message.go examples/internal/clients/unannotatedecho/api_unannotated_echo_service.go examples/internal/clients/responsebody/client.go examples/internal/clients/responsebody/response.go examples/internal/clients/responsebody/configuration.go examples/internal/clients/responsebody/model_examplepb_repeated_response_body_out.go examples/internal/clients/responsebody/model_examplepb_repeated_response_body_out_response.go examples/internal/clients/responsebody/model_examplepb_repeated_response_strings.go examples/internal/clients/responsebody/model_examplepb_response_body_out.go examples/internal/clients/responsebody/model_examplepb_response_body_out_response.go examples/internal/clients/responsebody/model_response_response_type.go examples/internal/clients/responsebody/api_response_body_service.go examples/internal/clients/generateunboundmethods/client.go examples/internal/clients/generateunboundmethods/response.go examples/internal/clients/generateunboundmethods/configuration.go examples/internal/clients/generateunboundmethods/model_examplepb_generate_unbound_methods_simple_message.go examples/internal/clients/generateunboundmethods/api_generate_unbound_methods.go
# These generation steps are run in order so that later steps can
# overwrite files produced by previous steps, if necessary.
buf generate
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
# Remove generated gateway in runtime tests, causes import cycle
rm ./runtime/internal/examplepb/non_standard_names.pb.gw.go
# Remove generated_input.proto files, bazel genrule relies on these
# *not* being generated (to avoid conflicts).
rm ./examples/internal/proto/examplepb/generated_input.pb.go
rm ./examples/internal/proto/examplepb/generated_input_grpc.pb.go
rm ./examples/internal/proto/examplepb/generated_input.pb.gw.go
buf generate \
        --template ./examples/internal/proto/examplepb/openapi_merge.buf.gen.yaml \
        --path ./examples/internal/proto/examplepb/openapi_merge_a.proto \
        --path ./examples/internal/proto/examplepb/openapi_merge_b.proto
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
buf generate \
        --template ./examples/internal/proto/examplepb/standalone_echo_service.buf.gen.yaml \
        --path examples/internal/proto/examplepb/unannotated_echo_service.proto
mv examples/internal/proto/examplepb/unannotated_echo_service.pb.gw.go examples/internal/proto/standalone/
buf generate \
        --template ./examples/internal/proto/examplepb/unannotated_echo_service.buf.gen.yaml \
        --path examples/internal/proto/examplepb/unannotated_echo_service.proto
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
buf generate \
        --template ./examples/internal/proto/examplepb/generate_unbound_methods.buf.gen.yaml \
        --path examples/internal/proto/examplepb/generate_unbound_methods.proto
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
buf generate \
        --template ./examples/internal/proto/examplepb/use_go_template.buf.gen.yaml \
        --path examples/internal/proto/examplepb/use_go_template.proto
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
buf generate \
        --template ./examples/internal/proto/examplepb/ignore_comment.buf.gen.yaml \
        --path examples/internal/proto/examplepb/ignore_comment.proto
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
buf generate \
        --template ./examples/internal/proto/examplepb/visibility_rule_preview_echo_service.buf.gen.yaml \
        --path examples/internal/proto/examplepb/visibility_rule_echo_service.proto
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
mv examples/internal/proto/examplepb/visibility_rule_echo_service.swagger.json examples/internal/proto/examplepb/visibility_rule_preview_echo_service.swagger.json
buf generate \
        --template ./examples/internal/proto/examplepb/visibility_rule_internal_echo_service.buf.gen.yaml \
        --path examples/internal/proto/examplepb/visibility_rule_echo_service.proto
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
mv examples/internal/proto/examplepb/visibility_rule_echo_service.swagger.json examples/internal/proto/examplepb/visibility_rule_internal_echo_service.swagger.json
buf generate \
        --template ./examples/internal/proto/examplepb/visibility_rule_none_echo_service.buf.gen.yaml \
        --path examples/internal/proto/examplepb/visibility_rule_echo_service.proto
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
mv examples/internal/proto/examplepb/visibility_rule_echo_service.swagger.json examples/internal/proto/examplepb/visibility_rule_none_echo_service.swagger.json
buf generate \
        --template ./examples/internal/proto/examplepb/visibility_rule_preview_and_internal_echo_service.buf.gen.yaml \
        --path examples/internal/proto/examplepb/visibility_rule_echo_service.proto
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
mv examples/internal/proto/examplepb/visibility_rule_echo_service.swagger.json examples/internal/proto/examplepb/visibility_rule_preview_and_internal_echo_service.swagger.json
buf generate \
        --template ./examples/internal/proto/examplepb/visibility_rule_enums_as_ints_echo_service.buf.gen.yaml \
        --path examples/internal/proto/examplepb/visibility_rule_echo_service.proto
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/rpc/status.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
google/protobuf/any.proto descriptor.File should not contain nil SourceCodeInfo
mv examples/internal/proto/examplepb/visibility_rule_echo_service.swagger.json examples/internal/proto/examplepb/visibility_rule_enums_as_ints_echo_service.swagger.json
swagger-codegen generate -i examples/internal/proto/examplepb/echo_service.swagger.json \
        -l go -o examples/internal/clients/echo --additional-properties packageName=echo
make: swagger-codegen: No such file or directory
make: *** [examples/internal/clients/echo/client.go] Error 1
johanbrandhorst commented 1 year ago

I'll release 2.17.1 with this fix, thank you!

aimuz commented 1 year ago

nice