Why not use conventianal sequential-based indexes?

[GoogleCodeExporter]

Hi, I prefer your approach of using DataContract's for defining the protobuf 
model (as opposed to the other solutions), but I really don't get why you don't 
use the conventional approach for defining the field indexes. i.e. use the 
sequential layout of the properties as the index. 

In this way it would have a 'zero-intrusion' on existing DTO classes and you 
would be able to 'retain the same hidden indexes' just by adding fields at the 
end of each DataContract.

I think you would get a lot more adoption if you reduce the effort required for 
existing code-bases.

Original issue reported on code.google.com by demis.be...@gmail.com on 30 Nov 2009 at 5:52

[GoogleCodeExporter]

There is no such ordering:

http://msdn.microsoft.com/en-us/library/aky14axb.aspx
"The GetProperties method does not return properties in a particular order, 
such as 
alphabetical or declaration order. Your code must not depend on the order in 
which 
properties are returned, because that order varies."

http://msdn.microsoft.com/en-us/library/ch9714z3.aspx
"The GetFields method does not return fields in a particular order, such as 
alphabetical or declaration order. Your code must not depend on the order in 
which 
fields are returned, because that order varies."

You can, however, use implicit alphabetical ordering if you choose (without 
having to 
add your own markers), but that is of course brittle if you add an 
"AardvarkCount" 
property. Have I misunderstood your meaning?

Original comment by marc.gravell on 30 Nov 2009 at 9:02

[GoogleCodeExporter]

Yeah I did mean Declaration order but your right in that the API in theory does 
not look like it would 
support it. Though in practice I've always found for single inheritance classes 
(like most DTO's) that it 
always comes back in declaration order.

I still think you should try to supporting index-less dto's by either using 
alphabetic ordering or maybe 
something unconventional like parsing the source code or inspecting the IL with 
Cecil. 

Personally for people that don't provide indexes I would still try to infer one 
using the GetProperties() 
index order. If it causes problems they can always add indexes to them. If it 
causes problems they can 
always go back and specify indexes later.

Original comment by demis.be...@gmail.com on 30 Nov 2009 at 9:40

[GoogleCodeExporter]

Alphabetic is supported; for example:

[ProtoContract(ImplicitFields = ImplicitFields.AllPublic, InferTagFromName = 
true)]

There is also a mechanism to set this globally if needed. I am in the process 
of adding 
more documentation, but yes: this isn't very clear. I /guess/ I could add an 
"at your 
own risk declaration order", but the documentation **explicitly** states that 
this is 
unsafe, which makes me... hesitate.

Original comment by marc.gravell on 30 Nov 2009 at 9:50

[GoogleCodeExporter]

Actually yeah, I think a global initializer would be more useful. Maybe you 
could 
inject a SortOrder behaviour so we can control our own sorting while providing 
some 
common options out of the box. This way you can document the risk of the 
different 
behaviours.

I think the easiest you can make it fit for existing code-bases the more 
adoption you 
will likely have as devs will most likely prototype it first by integrating it 
into 
their existing app and if they see that protobuf-net is providing valuable 
gains than 
they can go the extra mile and go back and index all their fields.

While I'm on the subject I think you should also provide a Service 
Interface/Gateway 
over HTTP as I believe it would be the most useful. It may not be the quickest 
but 
its got the best chance to work over firewalls.

Original comment by demis.be...@gmail.com on 30 Nov 2009 at 10:38

[GoogleCodeExporter]

That'll be:

ProtoBuf.Serializer.GlobalOptions.InferTagFromName = true;

and:

ProtoBuf.ServiceModel.Client.ProtoClient<T>

then...

Original comment by marc.gravell on 30 Nov 2009 at 11:00

[GoogleCodeExporter]

Hey tnx will check it out.

Original comment by demis.be...@gmail.com on 30 Nov 2009 at 11:14

[GoogleCodeExporter]

Hi, 

Is it possible to have a global option for 'ImplicitFields = 
ImplicitFields.AllPublic' so people can avoid having to decorate their 
POCO models with 'ProtoContract' and hence needing a dependency on ProtoBuf, 
i.e:

[ProtoContract(ImplicitFields = ImplicitFields.AllPublic, InferTagFromName = 
true)]

Original comment by demis.be...@gmail.com on 20 Jan 2010 at 11:28

[GoogleCodeExporter]

If the issue is a dependency, you might also try:

    [XmlType] / [XmlElement(Order = n)]

or

    [DataContract] / [DataMember(Order = n)]

which work too... would that suffice? Combined with GlobalOptions possibly.

Original comment by marc.gravell on 21 Jan 2010 at 12:08

[GoogleCodeExporter]

Hi the rest of my class is a DataContract i.e. with [DataContract] and 
[DataMember]'s 
but wont serialize without the [ProtoContract]? 

Weird, I'll look deeper into it tomorrow.

Original comment by demis.be...@gmail.com on 21 Jan 2010 at 12:31

[GoogleCodeExporter]

In most builds it won't use [DataContract] if you are using the .NET 2.0 
version of the 
code. I have recently changed this so that it will recognise this attribute (by 
name) 
in either the 2.0 or 3.0 build. I can't remember if that change is in the 
current 
release binary, but try checking you have the 3.0 build.

Original comment by marc.gravell on 21 Jan 2010 at 5:28

[GoogleCodeExporter]

HI Marc,

So i am in the same boat as Demis, I don't want to add a  "protobuf" dependency 
on 
all my existing contracts that use DataMember/DataContract attributes. In that 
case, 
can I merely keep my Contracts as is without adding this the "order" property  
to my 
DataMember fields and simply having a global option set for 

"ProtoBuf.Serializer.GlobalOptions.InferTagFromName = true;"

Will this work?

Original comment by sangeeta...@gmail.com on 22 Jan 2010 at 7:41

[GoogleCodeExporter]

That should do it, yes; have you tried it?

Original comment by marc.gravell on 24 Jan 2010 at 10:16

[GoogleCodeExporter]

Not yet, I am facing some serialization issues when i am trying to plug my 
existing 
code with the protocol buffers. I have a service framework ,in which the 
request 
contracts look similar to a WCF contract however every contract derives from a 
base 
class "Request". The Responses are also  WCF contracts that derive from a base 
"Response" object. The framework is generalized to take in a "Request" Type and 
output a "Response" Type. when I plug in the protocol buffer serialization, I 
only 
seem to get the base class values serialized , the derived ones come back with 
default field values. Can you suggest me what I might be doing wrong.

This is what I am doing currently. I have decorated my Request/Response classes 
with 
"ProtoContract(ImplicitFields = ImplicitFields.AllPublic) and i have set the 
following Global Option: "ProtoBuf.Serializer.GlobalOptions.InferTagFromName = 
true;"

Original comment by sangeeta...@gmail.com on 25 Jan 2010 at 5:52

[GoogleCodeExporter]

Oh and by the way , I cant afford the decorate the base class with the 
"include" 
attribute that declares all the derived types, since that wont be extensible in 
my 
framework. 

Original comment by sangeeta...@gmail.com on 25 Jan 2010 at 5:54

[GoogleCodeExporter]

Then at the moment it simply isn't going to recognise your data. Sorry, but at 
the 
current time it doesn't support dynamic (runtime) subclasses - it needs to know 
them at 
compile. I could probably add support for some runtime way of fetching the 
subclasses, 
but you'd need to be able to *reliably* nominate the same id (tag) against each 
expected subclass - is this possible? i.e. a "FooRequest" might be tag 16 at 
the 
server, and *must* be tag 16 a the client. Likewise, if you have saved data on 
disk or 
in the database, it must be tag 16 today, and tag 16 in 6 weeks time when you 
try to 
load it back?

Original comment by marc.gravell on 25 Jan 2010 at 6:45

[GoogleCodeExporter]

Can we add a property on "protocontract" attribute that specifies the base 
class if 
any? Since the derived class will always know at compile time its base

Original comment by sangeeta...@gmail.com on 25 Jan 2010 at 4:22

[GoogleCodeExporter]

A bigger problem is the base-class knowing its subclasses. Are they in the same 
assembly?

Original comment by marc.gravell on 25 Jan 2010 at 4:35

[GoogleCodeExporter]

no they arent :( The base is in a common dll and the derived classes reside in 
their 
own assemblies. 

Original comment by sangeeta...@gmail.com on 25 Jan 2010 at 4:40

[GoogleCodeExporter]

One thing i could do it work on creating tags that ties to my existing 
serialization 
framework ordering. So then I don't need to worry about tags going out of sync. 
But I 
am not sure how much of a refactor that would be on the Protobuf-net dll. 

Original comment by sangeeta...@gmail.com on 25 Jan 2010 at 8:56

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Hey Mark, 

I was wondering if it would be ok to support IOC type pattern to get tags for 
all the 
serializable fields in a Type? The reason being, I had refactored protobuf-net 
to 
support proto serialization with our companys internal serialization framework. 
But 
inorder to sync with your newer version, I would have to re wire code to match 
it. If 
there was anyway in your MetaType.ApplyDefaultBehaviour() method, IOC pattern 
could 
be used to inject customized way to get all serializable fields within the 
type,  
with their Tag, Name, data format , etc information . This way anyone can 
inject 
their own way of getting tags  .

I am currently working to get this done and i am attaching a shelveset for 
review. 
The basic idea is if there was an interface :

public interface ITagGenerator 
    {
        List<IMemberInfo> GetMembersWithTags(Type type) ;
    }

where IMemberInfo is 

public interface IMemberInfo 
    {
        MemberInfo Member { get; set;}
        int Tag { get; set;}
        bool IsRequired { get; set; }
        DataFormat DataFormat { get; set; }
        string Name { get; set; }
    }

And the code at MetaType.ApplyDefaultBehaviour() could be:

 foreach (IMemberInfo member in TagGenerator. GetMembersWithTags())
            {
                ValueMember vm = ApplyDefaultBehaviour(family, member);
                if (vm != null)
                {
                    Add(vm);
                }
            }

and inside the method :

private ValueMember ApplyDefaultBehaviour(AttributeFamily family, MemberInfo 
member)

you can skip getting tag  information for each field and move on to getting the 
effective type and its serializer

Please let me know your thoughts.

Original comment by sangeeta...@gmail.com on 11 Apr 2010 at 8:48

Attachments:

• DefaultTagGenerator.cs
• GlobalOptions.cs
• MetaType.cs

[GoogleCodeExporter]

Adding the new code that works with array instead of list

Original comment by sangeeta...@gmail.com on 13 Apr 2010 at 11:22

Attachments:

• DefaultTagGenerator.cs
• MetaType.cs
• GlobalOptions.cs