null values should be treated as no value

[GoogleCodeExporter]

refer to the following post (description of the problem and proposed solution):
http://groups.google.com/group/protobuf/browse_thread/thread/e3909450a1fb1a21

Original issue reported on code.google.com by eliran.bivas on 14 Nov 2008 at 8:57

[GoogleCodeExporter]

This is a suggested patch to add checks for null value assignments.
The behavior is: throw NPE when the value is a required field, or invoke 
clearField
if its optional.

Recurring fields are also supported.

Original comment by aant...@gmail.com on 24 Nov 2008 at 1:36

Attachments:

• null_check.diff

[GoogleCodeExporter]

Can you send me this patch for review via codereview.appspot.com?

Original comment by kenton+protobuf@google.com on 24 Nov 2008 at 9:42

[GoogleCodeExporter]

Original comment by kenton+protobuf@google.com on 24 Nov 2008 at 9:43

• Changed state: Accepted

[GoogleCodeExporter]

Any news when/if this patch will be approved?

Original comment by jerrycattell on 16 Dec 2008 at 12:03

[GoogleCodeExporter]

The next version (2.0.4, I suppose) will throw NPE when you set a field to 
null.  
This was implemented recently in our internal code branch and will reach the 
SVN code 
next time I cross-integrate.

Apologies to aantono:  I never received notification that you had uploaded your 
patch 
to codereview.appspot.com, though I see it is there now.  I guess it's too late 
now 
since the competing internal change is already done.

Original comment by kenton+protobuf@google.com on 16 Dec 2008 at 12:11

[GoogleCodeExporter]

Throwig an NPE is certainly better than what it currently does and is the 
correct behavior for required fields.  
However, for optional fields, I think client code would be greatly simplified 
if setting a value to null would 
actually be the same as calling clear on the field.

Original comment by jerrycattell on 16 Dec 2008 at 5:15

[GoogleCodeExporter]

No worries... As long as it gets into the code, doesn't matter who wrote it. :)
I agree with jerrycattell that it would be nice to treat the optional fields 
with the 
same behavior as clearing them, since throwing an NPE is not really warranted 
for 
optionals.

Original comment by aant...@gmail.com on 16 Dec 2008 at 4:10

[GoogleCodeExporter]

There's disagreement about this.  I suspect that in most cases where null is 
passed 
to a setter, it is done by accident due to a bug in the code.  I think simply 
unsetting the field in this case is somewhat surprising behavior that would be 
likely 
to cause confusion, making it much harder to figure out what's really going on.

I can see how there are some specific use cases where clearing the field on 
null 
makes sense, but I don't know if these are common enough to outweigh the 
debugging 
difficulty in other cases.  I would prefer to take the safer route and throw 
NPE.  
It's possible to change our mind later, whereas it's not possible to change our 
mind 
once we go the other way.

Thanks for the input, though!

Original comment by kenton+protobuf@google.com on 16 Dec 2008 at 7:53

[GoogleCodeExporter]

I had to look at the protobuf java source code to figure out what the problem 
was. 
At the very least, a more descriptive error msg would be much more helpful.

Original comment by marc2...@gmail.com on 27 Jan 2009 at 5:11

[GoogleCodeExporter]

Yes, in the next version the setters themselves will throw NullPointerException 
if 
you pass null to them.  This change is already in SVN.

Original comment by kenton+protobuf@google.com on 27 Jan 2009 at 11:07

[GoogleCodeExporter]

(Fix is in SVN and will be in 2.0.4.)

Original comment by kenton+protobuf@google.com on 7 Feb 2009 at 3:01

• Changed state: Fixed
• Added labels: FixedIn-2.0.4

[GoogleCodeExporter]

This behaviour really should be mentioned in the Java tutorial as well.

For me, I'd rather the set(null) effectively clear the field, as I use JPA 
entities
to create messages and it's normal to have nullable (optional) columns.

Original comment by abru...@gmail.com on 12 Feb 2009 at 8:35

[GoogleCodeExporter]

I'm with abruegl on this one...
While it is acceptable to throw an NPE for reqired fields, the optional fields 
should
accept nulls, otherwise it makes the client's code very messy (you have to 
check for
null in every case) and it really defeats the notion of the field being 
optional. 
Since optinal does imply that the data might not be there, in java this is done 
via
null.  It also helps to have this, when working with various mapping frameworks 
like
JPA, or ORMs like Hibernate or iBatis.

The supplied patch has exactly that behavior, so the code is already there... :)
The CLA has been submitted a while back, so should not be a problem on that 
front as
well.

Original comment by aant...@gmail.com on 4 Mar 2009 at 7:32

[GoogleCodeExporter]

See my previous comment on why we decided to go with NPE.  I'm sorry that it 
makes 
your code more complicated but other people have other needs.

I'd be open to the idea of setOrClearFoo() accessors which behave the way you 
want -- 
*if* they do not add too much to the size of the resulting .class files.

Original comment by kenton+protobuf@google.com on 4 Mar 2009 at 8:04

[GoogleCodeExporter]

The problem with that is that it does not follow the standard Java Beans 
conventions
(which many of the mapping frameworks use), so having setOrClear*() method 
would not
buy much, unless you write all of the mapping code your self.

Would you be open to intorducing an option to the proto files which would 
indicate
what nullable behavior should be applied to setters?

Something like 
option java_nullable_setters = true;

Original comment by aant...@gmail.com on 4 Mar 2009 at 8:46

[GoogleCodeExporter]

Sorry, we have too many options.  They aren't free.

I'm not familiar with these "mapping frameworks".  What do they do?

Original comment by kenton+protobuf@google.com on 4 Mar 2009 at 8:53

[GoogleCodeExporter]

It has been common in my career to transform a model of one type into a model of
another type.  Doing this with an automated framework, like Dozer but there are 
many
others, limits the likelihood of developer mistakes.  Maybe you have the 
advantage of
being able to use the protobuf model object throughout the system, but a lot of 
other
users such as myself are going to be converting our applications to using 
protobufs
for communication, but it will be too costly to replace our existing models.  
Having
the protobuf follow JavaBeans standards seems reasonable to me.

I am not clear on why you want to throw NPEs for optional fields.  How would it 
be
hard to debug that you had called set with null?  If you look at the code and 
clear
is not being called, are there other options to clear this out?  

Original comment by jacob.to...@gmail.com on 4 Mar 2009 at 9:08

[GoogleCodeExporter]

Throwing a NPE for setting an optional field doesn't make sense to me.  I vote 
for
having protobuf clear the optional field you are trying to set, instead of 
throwing
an exception.

Original comment by smulli...@gmail.com on 4 Mar 2009 at 9:28

[GoogleCodeExporter]

Ken, it sounds like Java may not be a language that you've spent most of your 
time
developing in as you seem like you may not be too familiar with the JavaBeans
standard.  A JavaBean just means that all of the properties in the class are
manipulated with getter and setter methods that follow a certain naming 
convention
(getPropertyName():Type, setPropertyName(newValue:Type)

The origination of that standard stems back to Java AWT - the first windowing 
toolkit
for desktop Java.  The standardization meant that tools developers - like for 
IDEs,
etc - could discover the bean properties and whether they were readable and/or
writable by performing reflection and looking for methods or pairs of methods 
that
follow that convention. A metadata API called BeanInfo was built into Java 
perhaps
even from day 1 to support this model.

Since then, tons of projects and frameworks utilize this pattern to make it 
easier to
glue different components in your application together with things like XML or
specialized graphical IDEs.  A great example is Spring's BeanFactory.  With the
BeanFactory, you create an xml snippet like

<bean name="mybean" class="mypackage.MyClass">
   <property name="foo" value="myval"/>
   <property name="bar" value="1.5"/>
</bean>

Spring looks for the setter method named setFoo and invokes it with the value 
"myval".

Some good examples that could break if you're using ProtoBuf is Hibernate (a 
popular
object-relational mapping tool) and JSON-RPC or Axis (two remoting frameworks). 
 They
work in a simlar fashion to my spring example except that the values for the 
data
mappings are much more dynamic.  So if you're using Hibernate to map the output 
of a
SQL resultset to a graph of Javabeans and there's a NULL in one of the columns, 
when
Hibernate calls setMyColumn() and passes in null, it will cause a NPE.

I definately agree that a simple option in the .proto file would be an elegant
solution but it sounds like there may be some reasons why that's not possible.  
Love
to hear more on that.

Hope this helps.
---Marc

Original comment by marc2...@gmail.com on 4 Mar 2009 at 9:34

[GoogleCodeExporter]

Thanks, Marc.  Actually, I know about Java Beans, just wasn't sure about the 
term 
"mapping frameworks".  I see what you mean now.

While designing the Java implementation of Protocol Buffers, the question of 
supporting the Java Beans standard came up and the consensus was that we 
shouldn't 
bother.  I believe Josh Bloch in particular recommended against it.  I don't 
remember 
exactly why, but note that the Protocol Buffers model is pretty different from 
the 
Java Beans model.  For example, there actually is no such thing as "null" in 
protocol 
buffers -- when you clear a field, you are really just setting it to its 
default 
value.  The field's getter will never return null.  Also, beans do not seem to 
play 
nicely with the immutable-objects-with-separate-builders pattern.

Would it be possible to write an implementation of java.beans.BeanInfo which 
somehow 
wraps a protocol message or builder object?  In other words, write an adaptor 
which 
implements the bean interface in terms of protobuf reflection.  Looking at the 
documentation, it does not look very promising.

Note that the protobuf reflection interface allows you to write generic code 
which 
populates or reads arbitrary protocol buffer objects, completely within the 
protobuf 
model.  But I take it you want to use code that is already written to interact 
with 
beans.

Anyway, I think I'm going to ask Josh what he thinks on this one.

BTW, not a big deal, but my name is Kenton, not Ken.

Original comment by kenton+protobuf@google.com on 5 Mar 2009 at 6:10

[GoogleCodeExporter]

Kenton,

Sounds like there's already quite a bit of consensus on this, but here's my .02 
anyway. 

The fact that the Protocol Buffers model is different from Java caught my 
attention;
given that null has no meaning in Protocol Buffers is central to this given 
that this
is how the Java model interacts with the protobuf model. Given that in Java 
setting a
value to "null" is different from not setting it at all, it seems that we have 
two cases:

1. Required fields:

These fields must have a non-null value *and* must be specified, so default 
values
are moot. In this case, a Java client calling setFoo(null) should immediately be
notified this is invalid via the NPE. Should the Java client merely neglect to 
set
foo to some value, this will be handled later as it should, when the object is 
built. 

2. Optional fields:

Here, the fact that a Java object has no "optional" flag (and thus no possible
default values) implies that the only way to clear out a value in Java is by 
setting
it to null. In a Java object/bean/etc, the very meaning of 'null' is 'No 
Value'. When
the value is cleared out of the Java object, and that object is then converted 
to a
protobuf message, the lack of the value (e.g. null) directly implies that the 
value
in the protobuf should be unset. And thus, IMHO, setFoo(null) on an optional 
field
should absolutely be considered valid and be equivalent to clear(). It's the 
natural
way Java mapping tools specify that no value is set.

Not sure if this helps clarify or not, but - if not -- consider this merely 
another
voice requesting the set...(null) change as requested in the patch above. 
Thanks!

Original comment by scott.pf...@gmail.com on 5 Mar 2009 at 11:36

[GoogleCodeExporter]

Two short comments:

1. Use case: I'd like to be able to easily "copy" an optional field from one 
protobuf
to another by just writing "A.setX(B.getX())", possibly using different method 
names.
 If X was unset in B, this piece of code should unset it in A.

2. I don't think the semantics of setX/getX should depend on whether X is 
required or
optional.  I'd prefer this to be checked at the build() / buildPartial() stage.

Original comment by piotr.zi...@gmail.com on 10 Mar 2009 at 1:11

[GoogleCodeExporter]

Ken, Any words on Josh Bloch? We recently experience the same problem and we 
have to
check null for every option setter to clear the field. It makes the code very 
messy. 

Original comment by milky...@gmail.com on 10 Mar 2009 at 3:58

[GoogleCodeExporter]

Josh and the other local Java experts haven't responded.  I'm going to have to 
go 
track people down physically.

My name still isn't "Ken".

Original comment by kenton+protobuf@google.com on 11 Mar 2009 at 10:04

[GoogleCodeExporter]

Kenton, any news from Josh and others?

Original comment by aant...@gmail.com on 26 Mar 2009 at 1:55

[GoogleCodeExporter]

Sorry for the delay.  I haven't forgotten about this, just have a lot of stuff 
in my 
work queue.

Apparently Josh has locked himself in the fortress of solitude to work on some 
sort 
of "language proposal" which has a "tight deadline", so he's ignoring everyone. 

We're going to keep nagging him.

Frank Yellin -- who used to work on Java at Sun and who reviewed most of the 
Java 
protobuf implementation -- seems to think that the setters as they exist today 
already do not conform to the bean "standard".  Apparently bean setters must 
return 
void, but protobuf setters return Builder (to allow chaining).  Frank says this 
means 
they won't work as beans anyway unless we provide BeanInfo to tell the bean 
implementation how to deal with them.  If we do that, though, then there's no 
reason 
we couldn't tell it to use the hypothetical "setOrClear" methods instead of the 
plain 
"set" methods when accessing as a bean.

So it seems like having separate setOrClearFoo() methods should work.  Anything 
else 
I'm missing?

Original comment by kenton+protobuf@google.com on 26 Mar 2009 at 9:15

[GoogleCodeExporter]

Ken,
I'd say that this would be less-ideal, but still beneficial improvement.
Relaxing the setters for optional would be more ideal, as it fits more with the 
way
other frameworks (hibernate, spring, etc) do their mappings (not all of them 
use the
JavaBeans framework, but just stick to the naming conventions)

Original comment by aant...@gmail.com on 23 Apr 2009 at 4:11

[GoogleCodeExporter]

Kenton, any chance the changes discussed in this issue will be impllemented in 
the
next 2.x release?
I was hoping they would be included in the 2.1, but it does not look like it 
was. :(

Original comment by aant...@gmail.com on 14 May 2009 at 4:22

[GoogleCodeExporter]

Sorry, my todo list was just too full to deal with this for this release, and 
Josh 
Bloch never got back to me despite repeated pestering.  There still doesn't 
seem to be 
any agreement on the right behavior.  I really wish I could get some of the 
local 
experts to at least express some sort of opinion on this.

I will go raid some offices today and force an answer out of people.

Original comment by kenton+protobuf@google.com on 14 May 2009 at 6:20

[GoogleCodeExporter]

Kenton, thanks for the prompt response.
One alternative, I was thinking about, in the interest of allowing people some
flexibility on the behavior of those methods, would be an ability to define 
templates
for the code generation, where people can define the snippets of the method 
bodies, etc.
I've looked at the C code that generates the code, and the templated strings are
already hard-coded in there, so it does seem like a low-effort excercise to get 
them
externalized and allowed to be overwritten by passing in a template file.

Just a thought, but would be glad to hear your opinion/thoughts on it.

Alex

Original comment by aant...@gmail.com on 14 May 2009 at 7:29

[GoogleCodeExporter]

That's an interesting idea but doing it right would be a pretty massive change. 

Simply externalizing the strings isn't enough, since most of them are 
tightly-coupled 
to some set of variables which is computed by the surrounding code.  We'd need 
a 
better story for defining those variables.  I wish we had used a templating 
system 
from the beginning -- it would probably make the protocol compiler much easier 
to 
understand and maintain.  But I'm pretty hesitant to jump into that now.

(Unfortunately I was smothered in email yesterday and today and haven't had a 
chance 
to hunt people down yet.)

Original comment by kenton+protobuf@google.com on 15 May 2009 at 10:27

[GoogleCodeExporter]

Hey Kenton, just talking to Alex (aantono) about protos here at JavaOne. 
Omitting the ability to clear an optional 
field by setting its value to null is more than just a code style problem, and 
so they have to slow down their entire 
I/O system to do wrappers between the beans world and the proto world.

What's the real cost to making this change? It seems like the right thing to 
do, pragmatically speaking.

Original comment by limpbizkit on 3 Jun 2009 at 9:49

[GoogleCodeExporter]

I'm confused.  What kind of a system is made slower by this?  Why would it be 
any 
faster to move the logic into the accessors where it applies to everyone?

Frank Yellin and I -- the two primary developers of Java protocol buffers -- 
both do 
not like the idea of treating "null" and "not present" as the same thing.  They 
are 
not the same thing.  The following code, for example, would assert-fail with 
this 
behavior:

  myBuilder.setFoo(null);
  assert myBuilder.getFoo() == null;

I think this would be very confusing -- who would ever expect that assert to 
fail?  I 
know that there have been people before who expected protocol buffers to have 
real 
support for "null" -- i.e. you could set something to null, serialize, parse, 
and it 
would still be null at the other end -- and only figured out that this was not 
the 
case when they tried it and it threw NPE.  If we used the set-null-means-clear 
behavior, these people would not get an NPE at all; instead, they'd see the 
field be 
non-null on the receiving end and would probably be very confused.

Original comment by kenton+protobuf@google.com on 4 Jun 2009 at 9:19

[GoogleCodeExporter]

Kenton, I totally agree. 
I think the most suitable solution is to be able to receive exactly what was 
sent.
Meaning, if the server send null, receive null on the client side as well. This 
way
we will better represent the data passed over the wire.
Another point we'll be able to achieve with that is when converting the proto 
to OUR
local Java bean we'll be able to apply any Java bean assumptions (null clears 
the
value / null is a value).

Hope the fix will be added to next release, this sounds important to many 
protobuf
users/developers.

Original comment by eliran.bivas on 4 Jun 2009 at 9:50

[GoogleCodeExporter]

The protocol buffers wire format does not define a way to encode "null".  
Furthermore, not all languages have a concept of "null".  So, sending "null" is 
not 
an option.  Sorry.

I think we need to give in to the fact that protocol buffers are not java 
beans.  I 
doubt this is the only way that the two differ.  Any attempt to convert between 
them 
needs to be done at a higher layer and needs to consider all of the differences 
between the formats.  You should be able to write a general algorithm which 
uses 
protocol buffer reflection to implement whatever sort of conversion you want, 
without 
requiring any changes to the generated accessors nor lots of special-case code.

Original comment by kenton+protobuf@google.com on 4 Jun 2009 at 10:02

[GoogleCodeExporter]

Kenton, 
I agree with your desire to not treat nulls as clear field directives, but I 
think the 
reality of life is such that in Java "null" and no-value is about the same 
thing.
Based on your example in the previous post, you were saying that when one sets 
a 
field to null, the getter will return the default value because internally it 
would be 
cleared.
But if the application does the check and if the argument is null, it does not 
set the 
value at all, the end effect would be the same.  But with the NPE for optional 
fields, 
this forces to write boiler-plate wrapper code to do the checks for every 
single field 
before setting it, which leads to unnecessary code bloat with no added benefit.

I was talking to limpbizkit at JavaOne and he agrees with the reality vs. 
desire 
argument as well.
So unless we can have a "null" support (which does not seem like happening, nor 
do I 
think should happen), the treatment of optional values in Java would be best 
served 
by not throwing an NPE or by allowing a templating engine which would allow for 
the 
custom definition of this section of the code generation.

Original comment by aant...@gmail.com on 4 Jun 2009 at 11:24

[GoogleCodeExporter]

> this forces to write boiler-plate wrapper code to do the checks for every
> single field before setting it, which leads to unnecessary code bloat with
> no added benefit.

I cannot remember having ever actually seen code which does this "boilerplate". 
 I 
think this is specific to your system, not something that necessarily applies 
to all 
users of protocol buffers.

Can you explain to me exactly what your system does and how it works?  Code 
samples 
would be great.  Maybe we can find an alternative solution that benefits 
everyone.  
If you'd rather e-mail me directly than post it in public, that's fine.

Original comment by kenton+protobuf@google.com on 5 Jun 2009 at 5:27

[GoogleCodeExporter]

Kenton,
I'm in the process of migrating from JSON to Protobuf, and I must say I was very
surprised and disappointed when I came across this behaviour in the Java 
binding. 

Surprised, because, as mentioned by others, nulls in Java are an idiomatic way 
to
flag the absence of a value. I completely expected protobuf-java to treat them 
as
such. Many libraries I've come across have no clear methods, but instead use
set(null), or at least consider it equivalent to clear(). This is really 
standard stuff.

Disappointed, because of the boilerplate then required when your model objects 
have
many optional fields.

I don't know if you ever got an answer to your last comment, anyway here's an 
example.

If your model object is:

public interface Person {
  public String getFirstName();
  public String getLastName();
  public String getAddress1();
  public String getAddress2();
  ...
  public Integer getAge();
}

where some or most fields are optional, then instead of 

Person p = ...;
ProtoPerson pp = ProtoPerson.Builder.newBuilder()
  .setFirstName(p.getFirstName())
  .setLastName(p.getLastName())
  .setAddress1(p.getAddress1())
  .setAddress2(p.getAddress1())
  ...
  .setAge(p.getAge())
  .build();

you have to write:

ProtoPerson.Builder builder = ProtoPerson.Builder.newBuilder();
if (p.getFirstName() != null) builder.setFirstName(p.getFirstName());
if (p.getLastName() != null) builder.setLastName(p.getLastName());
if (p.getAddress1() != null) builder.setAddress1(p.getAddress1());
if (p.getAddress2() != null) builder.setAddress2(p.getAddress2());
...
if (p.getAge() != null) builder.setAge(p.getAge());
ProtoPerson pp = p.build();

Not so great, twice the character count already... It gets worse if you need to 
mix
in some value processing (such as escaping or capitalizing or filtering or 
formatting
or whatever) that you'd rather not perform twice for each field.

ProtoPerson pp = ProtoPerson.Builder.newBuilder()
  .setFirstName(doSomeModeratelyCostlyEscaping(p.getFirstName()))
  .setLastName(doSomeModeratelyCostlyEscaping(p.getLastName()))
  .setAddress1(doSomeModeratelyCostlyEscaping(p.getAddress1()))
  .setAddress2(doSomeModeratelyCostlyEscaping(p.getAddress1()))
  .build();

And you get:

ProtoPerson.Builder builder = ProtoPerson.Builder.newBuilder();
String eFirstName = doSomeModeratelyCostlyEscaping(p.getFirstName())
if (eFirstName != null) builder(p.getFirstName());
String eLastName = doSomeModeratelyCostlyEscaping(p.getLastName())
if (eLastName != null) builder(p.getLastName());
String eAddress1 = doSomeModeratelyCostlyEscaping(p.getAddress1())
if (eAddress1 != null) builder(p.getAddress1());
String eAddress2 = doSomeModeratelyCostlyEscaping(p.getAddress2())
if (eAddress2 != null) builder(p.getAddress2());
ProtoPerson pp = p.build();

which is barely legible.

My actual use case has many such classes with dozens of optional fields.

Granted, it's no big deal but, yuck.

IMHO, there should really be at least an option to have set(null)'s interpreted 
as
clears. I understand that there are no nulls in protobuf, but the point of
protobuf-java is not to implement the Protobuf model in Java. It is to provide 
an API
to use Protobuf from Java, and thus it should be as friendly as possible to Java
developers. And an important part of this is to embrace Java idioms when 
possible.

Original comment by christop...@gmail.com on 27 Jan 2010 at 8:10

[GoogleCodeExporter]

D'oh ! The last block of code should read:

ProtoPerson.Builder builder = ProtoPerson.Builder.newBuilder();
String eFirstName = doSomeModeratelyCostlyEscaping(p.getFirstName())
if (eFirstName != null) builder.setFirstName(eFirstName);
String eLastName = doSomeModeratelyCostlyEscaping(p.getLastName())
if (eLastName != null) builder.setLastName(eLastName);
String eAddress1 = doSomeModeratelyCostlyEscaping(p.getAddress1())
if (eAddress1 != null) builder.setAddress1(eAddress1);
String eAddress2 = doSomeModeratelyCostlyEscaping(p.getAddress2())
if (eAddress2 != null) builder.setAddress2(eAddress2);
ProtoPerson pp = p.build();

Original comment by christop...@gmail.com on 27 Jan 2010 at 8:14

[GoogleCodeExporter]

OK, look:  The vast majority of the people I talk to about this agree that the 
current behavior is correct.  Of course, these people, being happy with the 
status 
quo, don't usually come out to argue about it, which is why you see mostly 
complaints 
on this thread.

Now, I'd of course like to find a solution that pleases everybody.  However, I 
note 
the following:
- After a successful call to setFoo(x), getFoo() should return x.  I refuse to 
implement anything in which this is not the case.
- If getFoo() returns x before serialization, then after serializing and 
re-parsing 
the message, it must still return x.  I refuse to implement anything in which 
this is 
not the case.
- There is no "null" in the protocol buffers encoding.  We cannot add one as 
not only 
would it break compatibility, but it simply would not work in some other 
languages.  
In C++, for example, the getters return references, which cannot be null.  Some 
languages have no concept of null at all.
- A major reason why there is no "null" is because it would tend to lead to 
security 
problems.  If getFoo() could return null, and some caller failed to check for 
that, 
then it would be trivial for an attacker to exploit that by sending a null 
value.  
But, such problems could easily pass normal testing if the test clients never 
happen 
to send null values, which could easily be the case for many apps.  This would 
be 
especially problematic for C++ servers.
- Having getFoo() return null when hasFoo() is false would obviously lead to 
the same 
problems.  Additionally, it would break lots of existing code, and would 
disappoint 
many people who like the fact that they can write 
"getFoo().getBar().getBaz().hasQux()" without having to check hasFoo(), 
hasBar(), or 
hasBaz() explicitly.

Therefore we simply cannot have setFoo(null) do anything except throw NPE.

Now, what can we do instead?  I am open to the idea of introducing a 
setOrClearFoo() 
accessor which has the behavior you want.  Some open questions about this:
- How much will this increase generated code size?  Note that code size is a 
huge 
problem for protocol buffers at present.  The overhead would have to be no more 
than 
a percent or two.
- Should primitive fields also get a setOrClearFoo() accessor, taking the boxed 
type 
as the input?

So far no one has sent me a patch implementing this or examining the code size 
issue.  
Sorry, I don't have time to write it myself.

Note that as of version 2.3.0, you could actually implement setOrClearFoo() via 
a 
code generator plugin which inserts code into the classes generated by protoc.  
See:

http://code.google.com/apis/protocolbuffers/docs/reference/other.html
http://code.google.com/apis/protocolbuffers/docs/reference/java-
generated.html#plugins

Original comment by kenton@google.com on 27 Jan 2010 at 10:44

[GoogleCodeExporter]

Seems to me there is another way that large amounts of boilerplate code can be 
reduced - I've just documented it on my blog:

http://www.codedmike.com/java-nulls-in-protobuf/

Apologies for the horrendous formatting of the code snippets - it appears the 
early versions of Ghost still have a long way to go...

Original comment by mikerobe...@gmail.com on 12 Nov 2013 at 3:29

[GoogleCodeExporter]

For what it's worth, Wire is an alternate implementation of protocol buffers 
for Java. It uses null for absent.
http://corner.squareup.com/2013/08/introducing-wire.html

Original comment by limpbizkit on 13 Nov 2013 at 7:44

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Interesting thread. I love protobuf. I think the foundation of the disagreement 
is in comment #33:
"[We] do not like the idea of treating "null" and "not present" as the same 
thing.  They are not the same thing."

In fact, NULL is the common definition (and intuition) of "missing", at least 
since SQL/RDBMS (60s/70s), if not before

Usually the mismatches come from things like primitive types (say Java int, 
double) that don't support NULL

In our framework we simply treat hasField() == false as NULL

Original comment by osu...@gmail.com on 9 Feb 2014 at 6:51

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

To have null in proto buffer, can workaround be something like wrapping the 
value and type of value(null/non-null) into a message and then send it over. 
Need opinion.

Original comment by ag.coolg...@gmail.com on 15 Apr 2014 at 2:38