danielgtaylor / python-betterproto

Clean, modern, Python 3.6+ code generator & library for Protobuf 3 and async gRPC
MIT License
1.56k stars 218 forks source link

Infinite recursion with `int64_field` #620

Closed BryanFauble closed 1 month ago

BryanFauble commented 2 months ago

Summary

Getting the bytes of an int64_field is leading to infinite recursion

Reproduction Steps

Script to reproduce:

import betterproto

int64_field: int = betterproto.int64_field(1)

int64_field = 1
print(bytes(int64_field))

# 1 second in nanoseconds
int64_field = 1000000000

print(int64_field)
# This fails
print(bytes(int64_field))

The area of issue is likely in this part of the code: https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/__init__.py#L470-L498

Expected Results

The bytes to represent the number are returned.

Actual Results

Using the script, after running the second print(bytes(int64_field)) infinite recursion seems to occur. It will continue to print out the following string forever:

\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00

System Information

protoc --version; python --version; pip show betterproto
libprotoc 27.2
Python 3.12.4
Name: betterproto
Version: 2.0.0b7
Summary: A better Protobuf / gRPC generator & library
Home-page: https://github.com/danielgtaylor/python-betterproto
Author: Daniel G. Taylor
Author-email: danielgtaylor@gmail.com
License: MIT
Location: /home/bfauble/.local/share/virtualenvs/my-virtual-env-FjWt4rrD/lib/python3.12/site-packages
Requires: grpclib, python-dateutil, typing-extensions
Required-by: 

Checklist

Gobot1234 commented 2 months ago

This isn't how you're supposed to use messages. Please provide more of your code. This definitely works for the intended use case. You may have not instantiated the Message

BryanFauble commented 2 months ago

This isn't how you're supposed to use messages. Please provide more of your code. This definitely works for the intended use case. You may have not instantiated the Message

This is the minimum required code to showcase this example. I can show more if it's still needed, let me know. It's possibly related to how something like:

@dataclass(eq=False, repr=False)
class AnyValue(betterproto.Message):
    string_value: str = betterproto.string_field(1, group="value")
    bool_value: bool = betterproto.bool_field(2, group="value")
    int_value: int = betterproto.int64_field(3, group="value")
    double_value: float = betterproto.double_field(4, group="value")
    array_value: "ArrayValue" = betterproto.message_field(5, group="value")
    kvlist_value: "KeyValueList" = betterproto.message_field(6, group="value")
    bytes_value: bytes = betterproto.bytes_field(7, group="value")
BryanFauble commented 2 months ago

This definitely works for the intended use case. You may have not instantiated the Message

Is there a unit test that shows this functionality is working?

BryanFauble commented 2 months ago

Full example:

from dataclasses import dataclass
from typing import List
import betterproto

@dataclass(eq=False, repr=False)
class AnyValue(betterproto.Message):
    string_value: str = betterproto.string_field(1, group="value")
    bool_value: bool = betterproto.bool_field(2, group="value")
    int_value: int = betterproto.int64_field(3, group="value")
    double_value: float = betterproto.double_field(4, group="value")
    bytes_value: bytes = betterproto.bytes_field(7, group="value")

@dataclass(eq=False, repr=False)
class KeyValue(betterproto.Message):
    key: str = betterproto.string_field(1)
    value: "AnyValue" = betterproto.message_field(2)

@dataclass(eq=False, repr=False)
class ObjectWrapper(betterproto.Message):
    content: bytes = betterproto.bytes_field(1)

@dataclass(eq=False, repr=False)
class InnerObject(betterproto.Message):
    attributes: List[KeyValue] = betterproto.message_field(1)

key_value = KeyValue(key="my_key_value", value=1000000000)

inner_object = InnerObject(attributes=[key_value])

# This fails
my_wrapper = ObjectWrapper(content=bytes(inner_object))
print(my_wrapper)
Gobot1234 commented 1 month ago

Your issue here is on the key_value = KeyValue(key="my_key_value", value=1000000000) line. You should use key_value = KeyValue(key="my_key_value", value=AnyValue(int_value=1000000000)). I'd recommend setting up type checking so you avoid issues like this in the future