auto-complete etc support for protobuf generated code

[cnsgsz]

Description of the bug:

Clion cannot find protobuf generated code.

from xxx import foo_pb2

# Without "noinspection PyUnresolvedReferences", clion warns "cannot find `FooProto` in foo_pb2.py.
pb = foo_pb2.FooProto.FromString(record.numpy())

This also makes auto-complete etc not possible for protobuf code.

Which category does this issue belong to?

CLion, PyCharm

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Define any proto and use py_proto_library that comes with the protobuf library.

Example from tensorflow's workspace files:

    tf_http_archive(
        name = "com_google_protobuf",
        patch_file = ["//third_party/protobuf:protobuf.patch"],
        sha256 = "f66073dee0bc159157b0bd7f502d7d1ee0bc76b3c1eac9836927511bdc4b3fc1",
        strip_prefix = "protobuf-3.21.9",
        system_build_file = "//third_party/systemlibs:protobuf.BUILD",
        system_link_files = {
            "//third_party/systemlibs:protobuf.bzl": "protobuf.bzl",
            "//third_party/systemlibs:protobuf_deps.bzl": "protobuf_deps.bzl",
        },
        urls = tf_mirror_urls("https://github.com/protocolbuffers/protobuf/archive/v3.21.9.zip"),
    )

In BUILD:

load("@com_google_protobuf//:protobuf.bzl", "py_proto_library")

py_proto_library(
    name = "foo_py_proto",
    srcs = ["foo.proto"],
)

Example foo.proto:

syntax = "proto3";

package xxx;

message FooProto {
...
}

Which Intellij IDE are you using? Please provide the specific version.

Clion 2023.3.4 and PyCharm professional 2023.3.4

What programming languages and tools are you using? Please provide specific versions.

python3.10

What Bazel plugin version are you using?

2024.1.30.0.1-api-version-233

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

It's not clear to me if this is a pycharm/clion issue, or bazel plugin issue, or protobuf (rule) issue. See this pycharm/clion issue.

In my generated foo_pb2.py, I don't see FooProto explicitly defined. globals() might be doing something magical that confuses clion/pycharm?

# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler.  DO NOT EDIT!
# source: xxx/foo.proto
"""Generated protocol buffer code."""
from google.protobuf.internal import builder as _builder
from google.protobuf import descriptor as _descriptor
from google.protobuf import descriptor_pool as _descriptor_pool
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports)

_sym_db = _symbol_database.Default()

DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\x14...proto3')

_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, globals())
_builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'xxx.foo_pb2', globals())
if _descriptor._USE_C_DESCRIPTORS == False:

  DESCRIPTOR._options = None
...
# @@protoc_insertion_point(module_scope)

[tpasternak]

@ujohnny @LeFrosch do you have capacity to handle it?

[tpasternak]

I can confirm I can reproduce the problem, however sadly I have to assign P3 as I don't have capacity to fix it soon

[tpasternak]

However, I think this might be caused by the fact that py_proto_library from creates the class dynamically as you mentioned, so I'm not sure if this will ever work. Are there any other generators that create regular python class files?

[tpasternak]

Btw I'm not sure if we can do a lot about that on IJ side, however it seems that you could configure your project in a way it would produce more ide-readable python proto outputs. Btw there is a related ticked in the protocolbuffers project https://github.com/protocolbuffers/protobuf/issues/16115