Library request: Google Protocol Buffers (protobuf) repeated fields

[arthur-tacca]

It would be great to add support for repeated fields in Google Protocol Buffers.

Protocol buffers let you define "messages", which are basically like C structs, which have fields that are numeric, string or nested messages. From these definitions you generate code for any of several languages including C++, and this generated code includes methods for serialization to a language-neutral binary format. This library is used in Google projects like gRPC and TensorFlow, but also in lots of non-Google projects like Caffe and Mosh.

The fields can be single or repeated. Unfortunately, the C++ generated code for repeated fields uses their own custom containers. Depending of the type within, you either only see the first element in the debugger, or (usually) just a void* that points to the first element in the container. It would be great if debug visualizers for both types of classes were added to this plugin.

For intrinsic types, the template class google::protobuf::RepeatedField<T> is used. This stores the elements in a contiguous block, like a std::vector. This block is of type RepeatedField<T>::Rep, and the container includes a member of type Rep*. The static type of Rep finishes with a one-element array, which of course is really a multi-element array at runtime, so only the first element shows up in the debugger.

For string and nested messages types, the template class google::protobuf::RepeatedPtrField<T> is used. This also has a nested Rep type that includes an array (of static type length one), except that this is now an array of pointers rather than a direct array of objects. But wait - this is actually done in a non-templated base class called google::protobuf::RepeatedPtrFieldBase, so the static type of the final element of Rep is just void*.

Thanks for considering it!

[arthur-tacca]

I ended up just doing this myself; it was actually pretty easy. Here's the natvis that does the trick. Still, it would be handy to have it in the plugin, for easy installation.

(If this doesn't make it into the plugin, here is a message for anyone that comes across this when searching for it: Just make a new file in your visual studio solution, put it in any directory and give it any name you like but with a .natvis extension, and paste this XML in. That's it.)

<?xml version="1.0" encoding="utf-8"?>
<AutoVisualizer xmlns="http://schemas.microsoft.com/vstudio/debugger/natvis/2010">
  <Type Name="google::protobuf::RepeatedField&lt;*&gt;">
    <DisplayString>{{ size={current_size_} }}</DisplayString>
    <Expand>
        <Item Name="[size]" ExcludeView="simple">current_size_</Item>
        <Item Name="[capacity]" ExcludeView="simple">total_size_</Item>
        <Item Name="[arena]" ExcludeView="simple" Condition="rep_ != 0">rep_->arena</Item>
        <ArrayItems Condition="rep_ != 0">
            <Size>current_size_</Size>
            <ValuePointer>rep_->elements</ValuePointer>
        </ArrayItems>
    </Expand>
  </Type>
  <Type Name="google::protobuf::RepeatedPtrField&lt;*&gt;">
    <DisplayString>{{ size={current_size_} }}</DisplayString>
    <Expand>
        <Item Name="[size]" ExcludeView="simple">current_size_</Item>
        <Item Name="[capacity]" ExcludeView="simple">total_size_</Item>
        <Item Name="[arena]" ExcludeView="simple">arena_</Item>
        <ArrayItems Condition="rep_ != 0">
            <Size>current_size_</Size>
            <ValuePointer>($T1**)rep_->elements</ValuePointer>
        </ArrayItems>
    </Expand>
  </Type>
</AutoVisualizer>

[KindDragon]

Nice. Thank you

[btodd1968]

@arthur-tacca thank you for the post. I found that with google protobufs 3.6.1 it appears that RepeatedField may have since changed. Here is an updated/modified version of the natvis file that I got to work with the most recent protobuf implementation. Also, for newcomers, you can just drag this file into Documents/Visual Studio 2017/Visualizers/

<AutoVisualizer xmlns="http://schemas.microsoft.com/vstudio/debugger/natvis/2010">
  <Type Name="google::protobuf::RepeatedField&lt;*&gt;">
    <DisplayString>{{ size={current_size_} }}</DisplayString>
    <Expand>
      <Item Name="[size]" ExcludeView="simple">current_size_</Item>
      <Item Name="[capacity]" ExcludeView="simple">total_size_</Item>
      <ArrayItems Condition="ptr_.rep != 0">
        <Size>current_size_</Size>
        <ValuePointer>($T1*)ptr_.rep->elements</ValuePointer>
      </ArrayItems>
    </Expand>
  </Type>
  <Type Name="google::protobuf::RepeatedPtrField&lt;*&gt;">
    <DisplayString>{{ size={current_size_} }}</DisplayString>
    <Expand>
      <Item Name="[size]" ExcludeView="simple">current_size_</Item>
      <Item Name="[capacity]" ExcludeView="simple">total_size_</Item>
      <ArrayItems Condition="rep_ != 0">
        <Size>current_size_</Size>
        <ValuePointer>($T1**)rep_->elements</ValuePointer>
      </ArrayItems>
    </Expand>
  </Type>
</AutoVisualizer>

[danieljennings]

Here's an updated version compatible with Protobuf 3.15.3. Thank you to earlier posters for the starting point.

  <Type Name="google::protobuf::RepeatedField&lt;*&gt;">
    <DisplayString>{{ size={current_size_} }}</DisplayString>
    <Expand>
      <Item Name="[size]" ExcludeView="simple">current_size_</Item>
      <Item Name="[capacity]" ExcludeView="simple">total_size_</Item>
      <Item Name="[arena]" Condition="total_size_ == 0 &amp;&amp; arena_or_elements_" ExcludeView="simple">arena_or_elements_</Item>
      <ArrayItems Condition="total_size_ != 0">
        <Size>current_size_</Size>
        <ValuePointer>($T1*)arena_or_elements_</ValuePointer>
      </ArrayItems>
    </Expand>
  </Type>
  <Type Name="google::protobuf::RepeatedPtrField&lt;*&gt;">
    <DisplayString>{{ size={current_size_} }}</DisplayString>
    <Expand>
      <Item Name="[size]" ExcludeView="simple">current_size_</Item>
      <Item Name="[capacity]" ExcludeView="simple">total_size_</Item>
      <Item Name="[arena]" Condition="arena_" ExcludeView="simple">arena_</Item>
      <ArrayItems Condition="rep_ != 0">
        <Size>current_size_</Size>
        <ValuePointer>($T1**)rep_->elements</ValuePointer>
      </ArrayItems>
    </Expand>
  </Type>