Bug: Empty string in object is not serialized

benjamingorman commented 6 months ago

Summary

When a 'repeated string' type field in a protobuf object is used, and an empty string appended to it via the python-betterproto object, the empty string is not present in serialized output.

An empty string is a requirement in some protobuf definitions, for example Google pprof format which requires an empty string to be present at the beginning of the string table https://github.com/google/pprof/blob/main/proto/profile.proto

Reproduction Steps

Example code to reproduce:

example.proto

syntax = "proto3";

package hello;

// Greeting represents a message you can tell a user.
message Greeting {
  repeated string messages = 1;
}

codegen run with protoc -I . --python_betterproto_out=lib example.proto

main.py

import base64
from lib.hello import Greeting

test = Greeting()

# The empty string appended here does not appear in serialized output
test.messages.append("")
test.messages.append("foo")
print(test)
print(bytes(test))

with open("example.pb", "wb") as fh:
    fh.write(bytes(test))

with open("example.b64", "wb") as fh:
    fh.write(base64.encodebytes(bytes(test)))

example.b64: CgNmb28=

This is with python-betterproto 1.2.5 I wanted to test on the latest version of betterproto however after cloning the repo and running poetry install I could not:

$ poetry run python -m grpc_tools.protoc -I . --python_betterproto_out=lib example.proto
--python_betterproto_out: protoc-gen-python_betterproto: Plugin failed with status code 1.

I also wanted to try adding a test case for this to the repo however I cannot currently run the tests:

PS C:\Users\dude_\Projects\python-betterproto> poetry run pytest tests/test_streams.py
================================================ test session starts =================================================
platform win32 -- Python 3.10.6, pytest-6.2.5, py-1.11.0, pluggy-1.2.0
rootdir: C:\Users\dude_\Projects\python-betterproto, configfile: pytest.ini
plugins: asyncio-0.12.0, cov-2.12.1, mock-3.11.1
collected 0 items / 1 error

======================================================= ERRORS =======================================================
_______________________________________ ERROR collecting tests/test_streams.py _______________________________________
ImportError while importing test module 'C:\Users\dude_\Projects\python-betterproto\tests\test_streams.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
..\..\.pyenv\pyenv-win\versions\3.10.6\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests\test_streams.py:11: in <module>
    from tests.output_betterproto import (
E   ModuleNotFoundError: No module named 'tests.output_betterproto'
============================================== short test summary info ===============================================
ERROR tests/test_streams.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================== 1 error in 0.15s ==================================================

Any idea why the tests.output_betterproto module would be missing?

Expected Results

The empty string appended should have appeared in the serialized protobuf.

Actual Results

The empty string is not present in the serialized protobuf.

System Information

libprotoc 26.1 Python 3.10.6 Name: betterproto Version: 1.2.5 Summary: A better Protobuf / gRPC generator & library Home-page: http://github.com/danielgtaylor/python-betterproto Author: Daniel G. Taylor Author-email: danielgtaylor@gmail.com License: MIT Location: c:\users\dude_\projects\proto-test.venv\lib\site-packages Requires: grpclib, stringcase Required-by:

Checklist

[X] I have searched the issues for duplicates.
[X] I have shown the entire traceback, if possible.
[X] I have verified this issue occurs on the latest prelease of betterproto which can be installed using pip install -U --pre betterproto, if possible.

Gobot1234 commented 6 months ago

This is fixed on v2

Gobot1234 commented 6 months ago

For the outputs being missing you need to run Poe generate iirc or something to that effect

benjamingorman commented 6 months ago

Ok, good to know.

I haven't found a way to be able to run v2 though. Is there something I can do to get more information out of this command so I can debug?

$ poetry run python -m grpc_tools.protoc -I . --python_betterproto_out=lib example.proto
--python_betterproto_out: protoc-gen-python_betterproto: Plugin failed with status code 1.

danielgtaylor / python-betterproto