Can GetSchema() be made to suggest a proto file for lists or dictionaries?

[GoogleCodeExporter]

Hi,

I am sending Lists and Dictionaries over the wire from .NET to Python, so I 
need to generate .proto files for these kinds of objects (so I can generate the 
corresponding python message framework). Would it be possible for you to 
support generating these via the GetSchema() method?

Thanks! :)

Franchesca

Original issue reported on code.google.com by Franches...@gmail.com on 27 Aug 2012 at 2:24

[GoogleCodeExporter]

That should already work. Do you have an example of what fails?

Original comment by marc.gravell on 27 Aug 2012 at 3:50

[GoogleCodeExporter]

I use the following code:

string testdict = runtimeTypeModel.GetSchema(typeof(Dictionary<string, int>));
string testList = runtimeTypeModel.GetSchema(typeof(List<string>));

and get the following string output for both calls to get schema:

"package System.Collections.Generic;\r\n\r\n"

I am using revision 584 from trunk

Original comment by Franches...@gmail.com on 28 Aug 2012 at 1:01

[GoogleCodeExporter]

Ultimately, lists/dictionaries aren't top-level message type. IMO, the correct 
thing to do here is to declare a message that *encapsulates* those (i.e. *has* 
a list or dictionary), and walk that. Is there a specific scenario you had in 
mind for this?

Original comment by marc.gravell on 29 Aug 2012 at 9:18

[GoogleCodeExporter]

Yes, up to this point I have been manually making .proto files to wrap 
collections, but now more flexibility is required. The scenario I have is 
specific to opening up RPC to non .NET clients. At the moment developers using 
my framework can define an interface that describes the service contract (in 
much the same way that you can for WCF). Some of the method signatures that 
people are using either take dictionaries and lists as either parameters, or 
will return some kind of collection / nested collections. 

I have a codegen tool that makes python clients for .NET services, and I have 
tried making my own codegen to create the .proto file (wrapping the collection 
in a message, with the items as a repeated element), but it starts to get 
really complicated and unreliable when I have collections / nested collections 
with generic parameters of multiple types that have overlapping proto message 
names. I would have to generate the .proto message for the collection, then 
call getschema for each of the generic parameters, then parse all of those to 
fix duplicate messages. I am hoping that this might be something you can 
support directly (saving me a world of pain >.<). 

Original comment by Franches...@gmail.com on 29 Aug 2012 at 10:13

[GoogleCodeExporter]

> "but it starts to get really complicated and unreliable when I have 
collections / nested collections with generic parameters of multiple types that 
have overlapping proto message names"

Yes, I get to face the same complexity ;p Still, try r588 and let me know how 
you get on.

Original comment by marc.gravell on 29 Aug 2012 at 11:20

• Changed state: Fixed

[GoogleCodeExporter]

Oh, awesome, thanks!!!

It works great for lists, and even lists of lists.

For Dictionaries I am getting the proto message wrapper, but not the messages 
representing the types of the generic parameters, e.g. this example below does 
not contain the message for the Cat type, but a list of Cat works fine:

message Dictionary_String_Cat {
   repeated KeyValuePair_String_Cat items = 1;
}
message KeyValuePair_String_Cat {
   optional string Key = 1;
   optional Cat Value = 2;
}

A dictionary of dictionaries currently only goes 1 level deep :

message Dictionary_String_Dictionary`2 {
   repeated KeyValuePair_String_Dictionary`2 items = 1;
}
message KeyValuePair_String_Dictionary`2 {
   optional string Key = 1;
   optional Dictionary_String_Cat Value = 2;
}

Thanks again, this is going to make my life so much easier!!! :D

Original comment by Franches...@gmail.com on 29 Aug 2012 at 12:03

[GoogleCodeExporter]

Try r589 then. 

Original comment by marc.gravell on 29 Aug 2012 at 12:33

[GoogleCodeExporter]

Ok, this one looks much better, but still a slight issue. When the generic 
parameter is a derived type, the key value pair message should point to the 
message of the base type rather than the derived type. Here is an example 
.proto showing the issue, where the Cat type is derived from the Animal type:

package MyLib;
import "bcl.proto" // schema for protobuf-net's handling of core .NET types

message Animal {
   optional int32 NumberOfLegs = 3 [default = 0];
   // the following represent sub-types; at most 1 should have a value
   optional Cat Cat = 1;
}
message Cat {
   optional string Breed = 1;
   optional bcl.DateTime Dob = 2;
}
message Dictionary_String_Cat {
   repeated KeyValuePair_String_Cat items = 1;
}
message KeyValuePair_String_Cat {
   optional string Key = 1;
   optional Cat Value = 2;
}

Original comment by Franches...@gmail.com on 29 Aug 2012 at 1:42

[GoogleCodeExporter]

Yes, you are absolutely right. See, all these subtle cases are why I had it 
marked as "experimental" ;p

Try r590, and keep being picky - it forces me to improve the logic.

Original comment by marc.gravell on 29 Aug 2012 at 2:14

[GoogleCodeExporter]

Oh brilliant! It all seems to work fine now, even for dictionaries of 
dictionaries containing a derived type that is keyed by another type derived 
from the same base class (my most complicated test case for this).

Thanks a million for all your help! :D

Original comment by Franches...@gmail.com on 29 Aug 2012 at 2:54

[GoogleCodeExporter]

Original comment by marc.gravell on 29 Aug 2012 at 4:31

• Changed state: Verified