Feature request: Add option to @Getter to allow Optional returns for nullable fields

[michaelboyles]

I use Lombok's @Getter whenever possible. Almost all of the pure getters I find myself writing these days are for java.util.Optionals.

private final String foo;

public Optional<String> getFoo()
{
    return Optional.ofNullable(foo);
}

Changing the field declaration to Optional<String> is not a great solution because any setters would expect an Optional. It's also frowned upon to use Optional as a field value (IntelliJ will give you a warning, for example).

It would be nice if @Getter had an option which allowed me to generate this automatically. For example:

@Getter(optional = true) private final String foo;

This would obviously default to false.

[markusjevringgoeuro]

Since a lot of people use @Data and @Value, without explicitly using @Getter, it would be nice if there was some annotation that would be able to produce the same effect for @Nullable fields.

[baha2046]

instead of making such option to java optional only, i recommend a more powerful version, since there has a lot of optional-like implement (e.g. Vavr - Option). So l recommend to make a function which we can define a processor for the getter, in the first floor it is @Getter(processor=Optional.ofNullable), so the value will first put in to method Optional.ofNullable and the result is return by the getter.

[Maaartinus]

@baha2046 @Getter(processor=Optional::ofNullable) would be the correct syntax, but you can't use method references in annotations and there's no way to specify a method, except for the stringly typed one like @Getter(processor="Optional::ofNullable"), which is pretty ugly. This unfortunately makes the more powerful version much less appealing.

[mxandeco]

I don't think you should have any hope on having this added to lombok.

1214 and mail list

pity.

=/

[lpandzic]

@rzwitserloot I assume your stance hasn't changed much about this issue and #1214. If you're following jdk development it's obvious there are mixed stances about this and there's a background agenda regarding converting Optional to a value type that seems to affect some of those stances - https://github.com/rzwitserloot/lombok/issues/1214#issuecomment-261511309.

Are there any technical pitfalls with implementing this feature or is it simply a religious/subjective matter? If it's only the latter, I'd really like lombok team to rise above it and have the flexibility to add this feature. As a compromise, add it under experimental package and document arguments against in javadoc?

[jwilmoth]

FWIW there are other frameworks that have support for optionals (e.g. https://immutables.github.io/immutable.html#optional-attributes). It would be nice to see support added to Lombok which IMHO has a richer and simpler dev experience usage than the alternatives.

[xenoterracide]

although I agree that sometimes Optional can be used in fields ( I specifically like it for writing builders that make decisions based on the presence of whether something is set ), I don't always want it, so I too would like the option to have generated getters return Optional.

[jsbournival]

Retuning Optional for a nullable field should be the default behaviour IMO.

[celiovasconcelos]

The most common usage scenario would be in JPA @Entity

The java.util.Optional does not implement the Serializable interface. For this reason, we should never map an entity attribute as Optional because that will restrict the entity usage.

The solution is move the Optional to the getter.

The whole world would be glad if Lombok helped in this scenario...

[nlwillia]

Retuning Optional for a nullable field should be the default behaviour IMO.

That would be a painful breaking change for a lot of projects. At the very least, there would need to be a lombok.config setting for it, and preferably have it be opt-in rather than opt-out.

[kowalski7cc]

Hello, any news on this issue? I'd love too see this feature!

[walec51]

@rzwitserloot didn't answer the comment https://github.com/rzwitserloot/lombok/issues/1957#issuecomment-471434870 then we can assume that his stance on this topic did not change - Optional will not get any special treating in his repository

the only option left is to do a fork - any one interested?

[Rawi01]

Instead of forking you can also create a new handler that creates these getters. The regular getter annotation will do nothing if there already is a getter.

[walec51]

Getter is just the simplest example where support for Optional is needed

the end game is support in Value and Data

[Vyacheslav-Lapin]

I think, it'll be great to use java.util.Optional class by default, but get support for change it to its analogs. Most of all I mean io.vavr.control.Option class in VAVR library - in my projects I use it instead of java.util.Optional.

[SDiamante13]

I think this feature would be great to add. At least make it experimental so the community can see if it is a viable solution.

[walec51]

@Vyacheslav-Lapin @SDiamante13 read the entire thread before posting... your not getting this feature here

[gpr-indevelopment]

@walec51 you are right... the repo owner posted here that lombok will not be implementing this feature, and PRs for this wont be accepted. He didnt explain why tho...

[ayush-finix]

@walec51 you are right... the repo owner posted here that lombok will not be implementing this feature, and PRs for this wont be accepted. He didnt explain why tho...

@gpr-indevelopment https://github.com/rzwitserloot/lombok/issues/2264 has slightly more explanation. Looks like it's a preference for nullability annotations. I prefer getting compile time errors when I forget to handle null without using a bunch of other tools, but there's very clearly a strong opinion by the repo owner to bypass the type system. So fork or deal with it (manual getters). Here's an intellij template to generate optional getters if it helps

#if($field.modifierStatic)
static ##
#end
Optional<$field.type> ##
#set($name = $StringUtil.capitalizeWithJavaBeanConvention(

    $StringUtil.sanitizeJavaIdentifier($helper.getPropertyName($field, $project))))
#if ($field.boolean && $field.primitive)
is##
#else
get##
#end
${name}() {
return Optional.ofNullable($field.name);
}

[rzwitserloot]

but there's very clearly a strong opinion by the repo owner to bypass the type system.

Careful. That's a little insulting - please don't grossly misrepresent another's opinion.

We hold the strong opinion that it is a terrible idea to explode 30 years of existing libraries by introducing a backwards incompatible fundamental change to the type system.

We do not hold the strong opinion that 'one should bypass the type system'. That's nuts - what do you think @Value and friends are trying to accomplish?

type tags, if done well, would be fantastic. For java, annotations seem like the obvious mechanism to use to tag types. Unfortunately, nobody has done them well, except a few corner cases which aren't popular. Also, the fact that there are tons of competing libraries out there isn't helping matters. But, from a technical point of view, they're great, and lombok already supports a lot of it: Lombok recognizes lots of nullity annotations and goes out of its way to generate code to take this into account. It's not splurged all over the docs because that's the point: annotations can be integrated without tossing out 30 years. I don't NEED to explain how to do it, lombok just more or less silently improves the code it already generated in a backwards compatible way.

Which gets me back to:

but there's very clearly a strong opinion by the repo owner to bypass the type system.

This is wrong. And I take offense to this characterisation.

[ayush-finix]

but there's very clearly a strong opinion by the repo owner to bypass the type system.

Careful. That's a little insulting - please don't grossly misrepresent another's opinion.

We hold the strong opinion that it is a terrible idea to explode 30 years of existing libraries by introducing a backwards incompatible fundamental change to the type system.

We do not hold the strong opinion that 'one should bypass the type system'. That's nuts - what do you think @Value and friends are trying to accomplish?

type tags, if done well, would be fantastic. For java, annotations seem like the obvious mechanism to use to tag types. Unfortunately, nobody has done them well, except a few corner cases which aren't popular. Also, the fact that there are tons of competing libraries out there isn't helping matters. But, from a technical point of view, they're great, and lombok already supports a lot of it: Lombok recognizes lots of nullity annotations and goes out of its way to generate code to take this into account. It's not splurged all over the docs because that's the point: annotations can be integrated without tossing out 30 years. I don't NEED to explain how to do it, lombok just more or less silently improves the code it already generated in a backwards compatible way.

Which gets me back to:

but there's very clearly a strong opinion by the repo owner to bypass the type system.

This is wrong. And I take offense to this characterisation.

My tone was unnecessarily aggressive and I apologize for that.

However, let me at least explain why I don't believe it's a mischaracterization. To your point, the reason you find the annotations lacking is because they cannot really help you fix this problem. Sure they can add runtime checks everywhere or static analysis to try and warn you of NPEs, but the actual problem is that @Nullable String and @NotNull String are the same type according to java's type system and no amount of annotation magic will change this fact.

Which leads to

backwards incompatible fundamental change to the type system

I fail to see how creating a class is a breaking incompatible change to the type system. Was adding ArrayList in java 1.2 a backwards incompatible fundamental change to the type system? Is every single user created class a change to the type system? Clearly not (although, in some sense you could argue it is because you're created new inhabitable types, but what I really mean to say is that none of these change how the rules of the type system of java are applied). We don't need some special nullable variable syntax to solve this problem, and I would strongly oppose adding some completely pointless (and infuriatingly specific to a single use case) kotlin style ? syntax to java. Using a maybe class (of any sort) fundamentally works because a Maybe<T> != T and the java compiler will not let you pass/use a Maybe<T> in a place where T is required. This is what it means to use the type system and no amount of annotations or runtime checks will equal this capability. Maybe Optional was not designed to be the equivalent of a general purpose Maybe monad, but it meets the requirements anyway (with one weird edge case for .map) because all you need is ofNullable and flatMap.

If a String is an infinite cardinality sum type of null | "" | "a" ..., then an Optional<String> is a type of null | NONE | SOME("" | "a" | ...). The important distinction is that in the pure string case, you pretty much have to handle the null, but in the maybe case, the null is always a programming mistake that you don't have to handle (what semantic meaning could you possibly have for guarding against a null optional?). If you guard the edges, it's entirely possible to have no meaningful nulls in your actual code. This does require you to code with the assumption that a type like String must be "" | "a" | ... without null as a value, but this isn't as hard to achieve/assume as you make it out to be.

As an aside, I wonder why people seem so dead set against using Optional as a parameter or field when they are perfectly happy to use List as a parameter and field. People will "happily" call things like Collections.singletonList, but if you say call Optional.of there's massive debate about it. Apparently a collection of 0 to N things is very, very different from a collection of 0 to 1 things.

[randakar]

That last remark pretty much hits it right on the head imho. As far as I am concerned Optional<> is just another type of Collection style class.

That bring said, the reason why people say that is mostly because language designer Brian Goetz says it.

And Brian says it because of two reasons:

• get() misuse. He called that method "Java 8's biggest API mistake" because people used to collections will try to use optionals like collections, getting the overhead of optional without any of the benefit.
• Serializable. Optional does not implement it, unlike Collections do.

(In my view Java should just deprecate Serializable. After decades of books telling people not to use Java serialisation that interface no longer make sense.)

And yes, get() is bad. That one could use deprecation too. And yes, there is a non-zero performance penalty. But.. if you want performance, what are you doing programming in Java?

[rzwitserloot]

To your point, the reason you find the annotations lacking is because they cannot really help you fix this problem.

Nope. That is not the reason I find them lacking. Annotations can entirely fix this problem.

I find them lacking because the popular ones are bad. Specifically, they are annotations for methods and fields (in many ways, this isn't a fault of the annotations themselves; java's first take on annotations didn't allow them to go on any type, only on methods/fields/params), and they do not consider the multivariant nullity (see later for more on that). They also don't use the power of flexibility, a.k.a. they also do not consider the legacy nullity either. A good version of such a library:

1. Is based on ElementType.TYPE_USE
2. Ships with a system to write, in text files or some other specific, non-java language, a list of 'these annotations ought to have been set on these types from commonly used libraries, including java.* itself.
3. Ships with plugins for IDEs and tools to apply what they mean, going so far as to feel entirely integrated into the IDE.
4. The ecosystem has coalesced onto a single set of annotations.
5. These annotations are capable of conveying all 4 variances of nullity (co-, contra-, in-, and legacy).

Nothing out there fits. And this Optional stuff is making coalescing harder to make happen, which is why Optional zealotry annoys me. Optional is a dead end, so the faster everybody realizes this, the faster we can either stop whining about the NUI problem (I'm not so sure it's that big of a deal), or coalesce onto a viable solution.

I shall explain why it is a dead end, and why lombok will not contribute to pushing the community further down this dead end.

Sure they can add runtime checks everywhere or static analysis to try and warn you of NPEs, but the actual problem is that @Nullable String and @NotNull String are the same type according to java's type system and no amount of annotation magic will change this fact.

That isn't an 'actual problem'. Show me how this is an 'actual problem', because I'm having a hard time seeing how this somehow makes it impossible to do what we presumably want, which is to have tools, docs, human eyeballs, and IDEs all in agreement about the need to check for nulls or the need to only pass definitely-not-null things. Surely that's the point of the exercise of Optional, no?

Let me try this specific example: @Override doesn't change the java type system or even the lang spec itself and yet there isn't an IDE, compiler, or linter tool out there that's confused about it. It does its job, and it fixes the problem. In an entirely backwards compatible way, no less. Nullity annotations can do the same thing, and are doing the same thing already: If you turn on nullity annotation checking in e.g. eclipse or intellij, and you are working with a codebase fully loaded up with nullity info and without scenarios that run into the (solvable) issues I complained about before, you will never get surprising NPEs ever again. The problem is, existing libraries don't have this info (but then existing libraries don't have Optional either, so don't go uncork that champagne just yet), which is why nobody does this. Also, the existing libraries fail to consider the full 4-way variance and thus the systems break down once you involve generics and legacy systems. But those are technically fixable problems, and I can explain to you precisely how to get there. "Existing API does not use Optional in their return types" aren't technically fixable, and/or you nor anybody else has explained to me how you fix this.

I fail to see how creating a class is a breaking incompatible change to the type system.

That's because we're clearly using different definitions of 'backwards compatible'. I mean: "Introducing a backwards incompatible change to the language means that existing code must either be changed and/or recompiled, or is forced into being considered obsolete". That last one is less often stated but obviously true. See next section.

Is every single user created class a change to the type system?

I would be less grumpy if you did me the favour of assuming I'm not a complete idiot. That's obviously not what I meant, please don't argue in bad faith.

Take java.util.Map and its get() method. Obviously, if you find Optional a good idea, this method should clearly have signature public Optional<V> get(Object key). There are 3 options I can see:

1. There is nothing wrong with it. Which implies Optional is not, in fact, a good idea, or I'm missing some other nuance about Optional. I'd love to hear how Map.get does not need Optional and never should, but I doubt that is your point of view.

2. Map should be deprecated/obsoleted. That's obviously very, very, very backwards incompatible.

3. You see no particular problem with an ecosystem where some API uses null to convey no-value, and other API uses Optional, and there's nothing wrong with this. Surely I don't need to explain that this is crazy, and worse than a consistent world (even one where NUI values are conveyed with undocumented, runtime-only-checkable null values).

So, take your pick: Optional is either backwards incompatible, or not possible without voodoo magic, or will split the community in twain. Those options suck.

Thus, unless you can explain some 4th option to me, or explain how one of these doesn't suck: Optional is bad, for java, the way you want it: Not backwards compatible in all pragmatic senses of that word.

Was adding ArrayList in java 1.2 a backwards incompatible fundamental change to the type system?

Yes, of course it was. Well, 'fundamental' is up for debate. In fact, it probably wasn't fundamental, and because of that, in my book a painful but acceptable change. It's also partly going back and fixing an error, and doing so barely in time.

Swing existed as an API in those days already, and a ton of swing methods used Vector in their public signatures. This mistake is still with us. Here is the javadoc of swing's DefaultTableModel from j11.

It's got Vector all over the place. While oracle still drags their heels and finding a way to officially convey that Vector is dead as a doornail (they save @Deprecated for things that are fundamentally broken or insecure, such as Thread.stop, not for things which are extremely bad ideas, but work fine). Surely we can all agree that Vector is obsolete, and should absolutely not be used today - it makes your code stand out as a sore thumb, it's got crazy ugly API (2 method names for most things, that do identical things), and they don't fit common java idioms (vectors are always synchronized, not something you generally want).

Just in case someone wants to argue it isn't deprecated/obsolete: Yes it is, and oracle agrees: The term vector is being reused and reclaimed by oracle for JSR338. The authors of that API are on record ("Future Java" podcast, there are no transcripts to search through unfortunately) as doing that because nobody uses vector anymore. I vaguely recall the term 'obsolete' was literally said.

And yet swing still has this crud. That makes it backwards incompatible. Swing is now an ugly API. What is swing supposed to do here?

1. Change Vector to List. This is binary incompatible in any case, source compatible for all uses of Vector in parameters, but source incompatible for any fields and usage in return types. That's already better than what Optional can do (which is source and binary incompatible for all uses).
2. Give up, and declare swing as an obsolete end-of-life library, and start fresh.
3. Release an incompatible version of swing.
4. Stick with the old API, complicating matters when interacting between services (if you want to make a DefaultTableModel, you will have to take your Lists and convert them to a vector first), and making the recommended use of your API forcibly result in code that has obsoleted stuff in there.

These options all suck. That makes ArrayList's introduction a painful transition that really broke some stuff.

It's not fundamental only because java 1.0 did not ship with the fundamental notion that collection types were an intrinsic part of the type system fit for using on API boundaries. That came with 1.1 which brought a full hierarchy (the collections API).

and I would strongly oppose adding some completely pointless (and infuriatingly specific to a single use case) kotlin style ? syntax to java

We are in agreement. The solution is type tags. Write-time ability to be more specific about secondary aspects of a given type is useful. It is useful to be able to convey, in a java type, that you want a 'safe html' string - one that has been escaped or wasn't sourced from user input or some other unauthorized source. One way is to introduce public class SafeHtmlString, but the problem is variance, and it's high time I explained this.

Let's have this method:

/**
 * Finds the first element in {@code list} that matches {@code predicate} and returns it.
 */
public static <T> T findFirst(List<T> list, Predicate<T> predicate) {}

This code is needlessly specific. Let's say I have:

List<Integer> listOfInts = new ArrayList<Integer>();
Predicate<Object> isInteger = x -> x instanceof Integer;

I cannot pass these to findFirst because the variance on those generics are wrong. In this case, the findFirst method does not care about certain abilities that lists have. This doesn't 'expand out' - findFirst doesnt care about the write features of lists (as its implementation never calls clear, set, add, addAll, retain, etc), it only cares about reading, but there are other methods that do. And because findFirst doesn't care, if it tells the type system that it doesn't care, the type system can be more lenient. The correct way to do this is actually:

public static <T> T findFirst(List<? extends T> list, Predicate<? super T> predicate) {}

So far we've already covered 3 variances: Covariant (the list param), Contravariant (the predicate param), and the earlier broken scenario, where we were incorrectly applying invariance.

Null handling is the same way!

And optional stands no chance. There are only 2 things with optional: Yes, or No.

String x; // in Optional world, this cannot hold NUI
Optional<String> x; // potentially NUI: NoValue, Unknown, or Irrelevant

That's 2 too few. Generics has 4:

List<? extends Number>; // covariance
List<? super Number>; // contravariance
List<Number>; // invariance
List; // raw / legacy variance

and these open and close different doors. nullity needs the same thing, and optional cannot deliver.

Using a maybe class (of any sort) fundamentally works because a Maybe != T

And that's why optional is so bad.

Imagine this method:

/**
 * Finds the first element in {@code list} which matches {@code predicate}, and adds it to the end of {@code list}, adding nothing if no element matches.
 */
public void duplicateFirstMatch(List<T> list, Predicate<T> predicate) {}

This method wants covariance for nullity. This method has a fun property: If BOTH the list itself as well as the predicate are capable of dealing with NUI, then the method's job can be performed. If both cannot, then the method can also perform - and the method will never break heap.

Let's say we live in current java land, and NUI is conveyed with null values in my list. I have:

List<String> example = new ArrayList<>(List.of("hey", null, "world"));
Predicate<Object> isNull = x -> x == null;
duplicateFirstMatch(example, isNull);

This would be fine, and would result in my list gaining a 4th element, and it's null, and that's okay. If I were to annotate the nullity states of all things in this example, it was already a List<@Nullable String>, and clearly a Predicate<@Nullable String>. Adding null to it wasn't an issue. But, this code still works if I have a List<@NonNull String>: The duplicateFirstMatch method cannot possibly 'break' the list by adding null to it, because it can only duplicate.

This is just one specific but small example of the notion of variance. It shows up everywhere once you start thinking about tagging types.

That shows rather clearly that your idea of just adding explicit types (Optional<T> in case of NUI. Perhaps SafeHtmlString in case of string) does not work. It both leads to massive backwards incompatiblity, in the sense that widespread use of such types can only occur if giant swathes of existing libraries are obsoleted, and it leads to an inexpressive language, because it forces all code written to be black-or-white, without the option of writing code that operates on multiple things within one typeclass. Such as duplicateFirstMatch which is code that works for either nullity typeclass.

For another really cool example of what typetagging can do, consider the 'Called' concept from the checkerframework. You can write this method:

public class PersonBuilder {
    // all sorts of 'setters', including birthDate(LocalDate bd) { .. }
    public Person build(@Called("birthDate") PersonBuilder this) { return new Person(...); }
}

That's a typetag: It's tweaking the definition of the receiver of the build method (the above is real, live, valid java, today! I know that this thing looks bizarre, but that's valid java). It tweaks it to state: The build() method's receiver must have the tag 'An instance of PersonBuilder, but, code analysis ensured that birthDate was called on this instance before.

This is entirely backwards compatible: Any existing APIs with builders that has merely documented 'please do not call build() before birthDate has been called on a builder, you'll break things'. Only code that intentionally breaks this rule and relies on the exception falling out of the build() method at runtime would be code that is [A] not buggy, and [B] nevertheless broken by this change. That's the kind of 'break of backwards compatibility' that's fine, because surely all are in agreement that such a use of this hypothetical builder API is mostly ridiculous, and at the very least so exotic, it's unlikely to affect much.

[michaelboyles]

@rzwitserloot Thank you for the detailed justification. Your relative silence up until now on what I think is your most requested feature ever was disheartening.

For me, the point of contention is here: "You see no particular problem with an ecosystem where some API uses null to convey no-value, and other API uses Optional, and there's nothing wrong with this. Surely I don't need to explain that this is crazy"

It doesn't seem crazy to me. This utopic ecosystem you dream of where annotations come to our rescue just isn't going to happen. Like you said, many have tried and many have failed. We've had annotations for years. If it hasn't happened yet, it's not going to.

For me, the use of Optional is a form of progressive enhancement. I know Map might return null, but that doesn't stop me from using Optional in all of my APIs. It's adoption does not mean we have to throw the baby out with the bathwater and re-write everything.

Does the use of Optional mean I can completely shut my eyes and know I can never get a NPE ever again? No. I have to be careful of older APIs of course. But it's the same in your utopia, where someone might forget to add an annotation.

Some inconsistency is inevitable. Optional is the closest thing we have to a standard right now.

[randakar]

It would be helpful if a Java language designer addresses this. If there were some overall plan proposed to give the community a direction hashing out the details is just a matter of converting libraries piecemeal, the same way it was done for generics and annotations.

Regardless of the form of this plan, it will be painful.

I think I just want a solution that gives us a path forward, regardless of form, most of all.

The how is the thing we disagree on, I think.

Ideally null does not exist as a value. Those variants Reinier describes only exist because null exists. Optional doesn't address them because it isn't designed to deal with mixed-nullity situations - it's designed to simplify things by removing null from the equation instead.

Backwards compatibility for Optional would look like a slow gradual replacing of methods and classes that are not null-safe, deprecating things step by step. Painful, would take decades, means old code breaks at some point. That's life. Deprecating the null keyword would be the end goal. Not impossible, but hard.

The annotation path would end up not breaking compilation for old code. This is true. But if I see a class with hundreds of warnings because it still uses raw List instances that code is no less painful and in need of fixing than a class that uses null values without any annotations.

Don't get me wrong. Type tags rock. But the type tag path looks like a dead end for the simple reason that there has not been any movement on the issue for decades. The only way forward at this point is a concerted effort on the part of the Java development team to address this.

And they have to. Java is losing it's primacy in the programming world. Google is moving Android to Kotlin for a reason. Java is making great strides forward, I just hope it is enough. This is one of the reasons.

On Sun, Dec 6, 2020, 11:19 Michael Boyles notifications@github.com wrote:

@rzwitserloot https://github.com/rzwitserloot Thank you for the detailed justification. Your relative silence up until now on what I think is your most requested feature ever was disheartening.

For me, the point of contention is here: "You see no particular problem with an ecosystem where some API uses null to convey no-value, and other API uses Optional, and there's nothing wrong with this. Surely I don't need to explain that this is crazy"

It doesn't seem crazy to me. This utopic ecosystem you dream of where annotations come to our rescue just isn't going to happen. Like you said, many have tried and many have failed. We've had annotations for years. If it hasn't happened yet, it's not going to.

For me, the use of Optional is a form of progressive enhancement. I know Map might return null, but that doesn't stop me from using Optional in all of my APIs. It's adoption does not mean we have to throw the baby out with the bathwater and re-write everything.

Does the use of Optional mean I can completely shut my eyes and know I can never get a NPE ever again? No. I have to be careful of older APIs of course. But it's the same in your utopia, where someone might forget to add an annotation.

Some inconsistency is inevitable. Optional is the closest thing we have to a standard right now.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rzwitserloot/lombok/issues/1957#issuecomment-739482325, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABIERIWSDTJEGWBOUGL52TSTNLEXANCNFSM4GGCNXZQ .

[rzwitserloot]

@michaelboyles wrote:

It doesn't seem crazy to me. This utopic ecosystem you dream of where annotations come to our rescue just isn't going to happen

I'm on the fence on this. I can see how my earlier post sounds like I'm 100% on board with the nullity-via-annotations future, but I'm not: I think it is the best and so far, only technically feasible strategy to ever get to a scenario where idiomatic java has type-carried guards against incorrect use of NUI. That's all - I didn't say that I think this future is going to happen. I'm merely saying: It could, and it would be better than what we have. vs. optional, which cannot, and if somehow I'm wrong and it does happen, would be worse than the current status quo. The presence of many competing nullity libraries, probably powered by the unfortunate series of events where JSR305 was originally going to bring these, did not have an easily included (i.e. maven G:A:V target) library, but never got anywhere, presumably because they belatedly realized null is a little more complicated than just @NonNull and @Nullable, has resulted in this cambrian explosion.

And that genie is hard to put back in the box.

But, you've missed a point I was trying to make, I think. Given:

• Let's say there's ojava: A hypothetical java where Optional<T> is idiomatic (all major libraries and tutorials mention it, all major IDEs support it and assume you use it).
• Let's say there is mjava: A hypothetical java where half of all code and libraries uses Optional<T>, and half use null, and this will never change for the entire lifetime of use for this language.
• Let's say there is njava: A hypothetical java where nullity annotations are idiomatic, and capable of conveying variances as well.
• Finally, there is java: The java we have now. runtime only nullity and all, with some smattering of half-supported, incomplete nullity libraries.

Here are my arguments:

• ojava is never going to happen from where we are now.
• mjava is terrible, worse than java.

You seem to be trying to make the argument that njava is also never going to happen. I hope you're wrong on that. Given that you can already kinda do njava in most IDEs, it's clear there are no technical limitations, but there certainly are plenty of community issues (and this obsession with Optional isn't helping). But that does not matter here - mjava is terrible, and ojava is impossible, therefore Optional is bad. I'm giving you njava as an olive branch: If you really want to move to a future java that has type-checked _NUI_, I'm trying to show you the way. If the kind of crazy zealotry that is being spent on vavr, optional, etc would have been spent on an annotation based proper NUI tagging system, then maybe it doesn't seem so unlikely.

But the point is, mjava is worse than java - and that is why lombok will not support it. Lombok'd be making a worse future-java happen.

For me, the use of Optional is a form of progressive enhancement.

That is a cronenberg-ian twisting of the meaning of 'progressive enhancement'. It's entirely backwards incompatible, not idiomatic java, and never will be. It is as far removed from the concept of progressive enhancement as pinguins and polar bears.

Does the use of Optional mean I can completely shut my eyes and know I can never get a NPE ever again? No. I have to be careful of older APIs of course. But it's the same in your utopia, where someone might forget to add an annotation

Nobody is making this argument in either direction, so I'm not sure how this one is relevant.

Optional is the closest thing we have to a standard right now.

Assuming sensible interpretations of the word 'standard', this is incorrect. If you multiply any method signature in existence with how often it is called across the planet year by year, the amount of signatures that convey NUI info via Optional pales in comparison to the amount that convey it with nullity annotations, or systems that auto-add this info by using an external source listing the missing annotations.

[rzwitserloot]

@randakar wrote:

It would be helpful if a Java language designer addresses this.

Brian Goetz has addressed it, what better source could there be? I can channel Brian for a moment - feel free to doublecheck this with him, but I'm 95% sure his current viewpoint of the situation is this:

1. He'd think that Optional has absolutely no hope of being backwards compatible. He wouldn't say it, he doesn't like to say 'never'. But he'd indicate he doesn't see a route, at least.
2. He'd say that this makes optional a bad solution to the problem.
3. He'd say that the 'problem' of NUI is not so incredibly big that the language is going to die soon if not addressed. For context, he'd posit that lack of closures WAS big enough to warrant a 5-alarm fire at some point, but NUI issues are not.
4. Therefore, given that it is hard to do right, and incredibly contentious: He doesn't want to address it right now. A simple value for money analysis: It's an incredibly high cost to get it done, more so because the community will be up in arms than the actual effort of figuring out a good design, and the benefit is low, because likely the majority of the community will end up hating it, because it's either not what they wanted, or they are shooting the messenger because they thought it was simple and think that the actual solution that Brian and co came up with is 'overengineered' and making it more complicated than it needed to be.
   • Thus, not worth spending any time on whatsoever. Better fish to fry: Access to CPU native wide instructions (the new vector API, part of panama - think: "Hey, that CPU has 80-bit native floating point regs, and 512-bit wide registers where you can run ops on in parallel, how do I unlock that from java?") - Valhalla, etc. These are way less contentious (in that the community will like these features way more), and thus both cheaper (less 'fighting' with the community) and more valuable (because more will like it, increasing the odds that the community adopts it).

I think I just want a solution that gives us a path forward, regardless of form, most of all.

Well, my long technical diatribe on all the options available should tell you that there is either no technical solution available (and if you want one, don't ask oracle engineers - come up with one), or, it's annotation-based typetagging. In any case, Optional is not, and cannot ever be, it.

Ideally null does not exist as a value.

What 'null' do you mean, here? The runtime value? The concept of NUI?

If you meant the concept of NUI, then this is an incorrect statement. Trying to make NUI go away is hard, and the few routes one can think of (for example, force every type to list a default value which is aggressively used) sound worse, in that they lead to silent bugs instead of obvious ones, which is definitely worse.

If you meant 'null', the reference value - then this statement is misleading. null is one way to solve NUI, why would it be 'ideal' if it 'did not exist'? That can only be said if some other NUI solution exists that is superior. There isn't one.

If you meant 'null' as an encompassing handwave over the entirety of the current java ecosystem, i.e. both the reference value and the notion that the type system has no information about nullity and never will - okay, maybe. But there are a billion solutions. Optional is one of many solutions, and a really bad solution at that.

Backwards compatibility for Optional would look like a slow gradual replacing

That's not what 'backwards compatibility' means.

But the type tag path looks like a dead end for the simple reason that there has not been any movement on the issue for decades.

Says you. This is a self fulfilling prophecy. Why do you think I fight the good fight and try to explain why Optional is such a bad plan? I can see how a future java where NUI is better handled at write-time is a better java. I want it to happen. Step one is to get the few zealots who care so much about it, to move away from the dead end.

type tags seem dead to you because you're hellbent on Optional. Look a little further and you'll see that actual support is there in IDEs, and getting better. Slowly, perhaps - because Optional is taking a lot of the oxygen out of the room.

And they have to. Java is losing it's primacy in the programming world.

I've been hearing that one for 20 years. So far, it's been horse manure every time it was said. It's horse manure now.

I'll try to change your mind.

java is currently losing a slight slice of relevance because python is used for integrating with GFX-card based AI and math stuff. This is weird because java is much faster and python has zero support for CPU-native 80-bit FP registers and the like just like java has zero support for this. But, python DOES make it WAY easier to integrate with C-libraries provided by e.g. nvidia for kuda and the like. Java has mechanisms to integrate with them but they are unwieldy; few know how, and the few that do, need to spend a ton of time to do a good job. Trying to then roll with the punches and upgrade your 'java facade' in the face of updates to the underlying library is really hard to do right, but kuda and co are changing a lot all the time.

Optional? Not even on the map.

Google chose kotlin for a reason? Of course they did. The reason is: Google LLC v. Oracle America, Inc

I would really appreciate it if we can move away form wishful thinking and logical fallacies for the remainder of this thread. I've hopefully given enough background with these long posts - from here on out, I'd love to see some in-depth technical analyses of what can be done to make optional viable, and failing that, I'll try to acknowledge any posts that include logical fallacies by naming the fallacy and not wasting any more words on the topic.

[michaelboyles]

@rzwitserloot The fact of the matter is that mjava is what we already have. You might not like it, but it's here and here to stay. Streams make heavy use of it. Future parts of the JDK are likely to make heavy use of it. Libraries will increasingly make use of it as well. Take a look at this issue, it is as far I can tell your most requested feature ever. People are using Optional more and more.

Nobody is making this argument in either direction, so I'm not sure how this one is relevant.

This was the only conclusion I could come to for why you would think a mix of null and Optional was bad - because the existence of Optional might be confusing or misleading for developers. If it's not that, then why is it so terrible?

If you multiply any method signature in existence with how often it is called across the planet year by year

This seems like speculation to me. I would be interested to know if there are any stats available. But in any case, I'm fairly confident that the JDK will continue to use it - I have no indication that they consider it a mistake in the way you do - and so adoption will increase as more people are exposed to it. Don't forget, it's still relatively new in the context of the whole language.

That is a cronenberg-ian twisting of the meaning of 'progressive enhancement'. It's entirely backwards incompatible

I'd argue your definition of backwards incompatible is what's actually a twisting of the common definition. But let's not argue over semantics. The point is that Optional can see wider adoption without affecting APIs that were written before it was available. And, once again, I'm afraid that this is going to happen, regardless of your opinion on it.

[kowalski7cc]

I agree that we already have mjava. And I don't think that is in Oracle's plans to deprecate Optional soon. This is what we have now.

[rzwitserloot]

The fact of the matter is that mjava is what we already have.

No we don't. Name me commonly used libraries that use Optional. it's very few. It should stay that way.

but it's here and here to stay.

As wikipedia editors would say, [citation needed].

Libraries will increasingly make use of it as well.

[citation needed]

People are using Optional more and more.

[citation needed]

This seems like speculation to me.

That's an interesting complaint about my reasoning after the paragraph you wrote that immediately preceded this!

Yes, of course it is. But at least one you can falsify, and a clear measurement. Unlike 'People are using it'. That is either meaningless (yes, of course, you can find a single java programmer that does. You can use that to justify the craziest of ideas. Or 'people' is to be taken as 'a significant amount', but then you haven't even bothered to highlight how one would measure this, let alone make a case that this is true. I'm presumably the person that needs convincing, here. You're doing a very bad job at it, what with these fallacies and nebulous, unfalsifiable statements.

If it's not that, then why is it so terrible?

Having more than one way to do something increases the learning curve, engenders style debates, and complicates matters for code caught between 2 different ways of accomplishing the same thing. Therefore, if there are 2 ways of doing things, and you both to be common parlance, you need to explain why it's good to have 2 ways; presumably, there are significant areas where one way is better than the other, and significant areas where the reverse is true. This is clearly not the case with Optional vs. type-tag for representing NUI info in APIs.

I would be interested to know if there are any stats available.

If you put some effort in and showed me falsifiable stuff, for example by writing a crawler for github projects, you may convince me. Which part of this thread is making it look like I'm going to be swayed by a bunch of folks saying 'but, @rzwitserloot, I think it is great, why can't you see it?'

I've given you rational, falsifiable arguments why Optional is bad.

Either waylay them (and not by saying you dont like these arguments - with actual logic (and not fallacies), actual numbers, or alternative technical implementations of things), or don't bother.

I've asked for rational argument and I'm seeing none. I'm done with it for another year.

If you think you have a novel new take on a technical solution, or a specific reason why the technical arguments as stated before aren't inevitable, please comment in this thread.

I don't think I owed you an explanation, but I provided it anyway. The feature request ends here. Any further debate on the merits, or lack thereof, will need to find another venue.

[michaelboyles]

@rzwitserloot

Name me commonly used libraries that use Optional

Streams, HttpClient (Java 11).

The Java ecosystem is already very established, so the fact that there are no major 3rd party libraries using it doesn't mean much. Name some commonly used libraries that sprung up post-Java 8. I can't think of any.

but it's here and here to stay.

Incubator for Java foreign memory access is using it. That's not published yet. So JDK authors are still actively using it. And anyway, since when did Java ever remove anything?

People are using Optional more and more.

You have 150 people who've cared enough to find this issue and upvote it. Once again, your most requested feature ever.

Of course you are free to do what you want, but if you think you are doing the Java community a favour by ignoring this, you are not.

There is also a way to politely refuse a request without being rude. I understand it could be frustrating if you feel like you constantly have to justify your position, but perhaps the fact that you have to do that should be some indication of how far in the minority you are.

Thank you for your library, but this "citation needed" crap is just childish, and paired with your general attitude, you have lost a significant amount of respect from me. You said in a previous comment "I would be less grumpy if you did me the favour of assuming I'm not a complete idiot" and to be honest, I think a lot of people you've replied to in this thread could say the same thing back to you.

[rzwitserloot]

Name some commonly used libraries that sprung up post-Java 8. I can't think of any.

So, Optional is doomed. Just so we're clear, the aim for you is to convince me to add this feature or allow a PR request. You're instead making the case we shouldn't. Well, I, uh, thanks, I guess.

Streams, HttpClient (Java 11).

Optional was originally 'invented' for streams. It's the one place where Optional is more or less fine and you should follow suit: If you are (somehow) writing your own single-value stream terminators, Optional is warranted. It ends there.

You have 150 people who've cared enough to find this issue and upvote it.

Lombok has something like a million users, and this isn't the first time java (and programming in general) has a zealotry movement where folks scream bloody murder and are extremely motivated to complain and cajole. History has usually shown that they were barking up the wrong tree. The point of the story is, 150 votes mean nothing. This is why I want technical details and falsifiable arguments. Appeals to the masses just do not work; zealotry has ruined that for all of us. This isn't your fault in the slightest. Just an unfortunate aspect of debates on programming language features we all have to deal with.

but if you think you are doing the Java community a favour by ignoring this, you are not.

I'm not ignoring it. I'm actively trying to quash its use. I do this, because I think I am doing the java community a favour.

There is also a way to politely refuse a request without being rude.

So, the hour+ I spent to write down the precise technical issues I have with Optional, just giving all interested a venue forwards, comes across as rude?

Yeah, I get that a lot. It might be a cultural thing. I find it frustrating and rude if I spend some time on a seemingly serious feature request, and someone throws a giant beaming smile at me and says: OMG! Wow thank you so much for your feedback!!!! We'll take it under advisement!! and I never hear from it again.

I hope me highlighting that I've spent quite some time on this issue is sufficient for you to realize I'm at least taking this seriously. On the contrary (culture, no doubt), but I find the incessant use of logical fallacies and appeals to emotion in the face of lengthy technical falsifiable explanations rude, because it sure feels like my viewpoints are being treated like a doormat.

At any rate, from my point of view this was the very opposite: I did you the kindness of going into the weeds on precisely why I tend to shoot requests about Optional down to give you an avenue, nigh impossible as it might seem, as to how to make optional happen in lombok.

you have lost a significant amount of respect from me.

@rspilker usually advises me to just close this stuff without comment, and I always get a little upset because that feels disrespectful.

And yet here we are. So very frustrating that this is where we ended up.

Thank you for your library, but this "citation needed" crap is just childish

I've repeatedly highlighted that I want some falsifiable stuff, and I yet get unfalsifiable stuff. Given that I spent more words in this thread to anybody else by quite far, you've misconstrued my intent. It's not childishness. It's me realizing I've overshot my time budget on this issue by quite a large margin and trying to be more efficient to make up for lost ground.

This last comment of yours did, at least, try to give me some falsifiable stuff and actual technical detail - had the conversation gone more in this direction it may have ended up in other places.

As is, the yearly budget for discussions about null/optional is mostly spent, I'm highly doubtful lombok as a project benefits if we spend much more time on this.

[michaelboyles]

150 votes mean nothing

Yes 150 out of 1 million is nothing. But not every user has a GitHub account, or cares enough to seek out a feature request, or will upvote it. "The silent majority". But you are still 150 to 2 against, as far as I can tell. Like I said, your most requested feature ever; if that's not important to you, or indicative of the community consensus then fine, but I can't see anyone who agrees with you. It's plausible that all 150 of us are stupid, of course, and that seems to be the premise from which you are working.

folks scream bloody murder

Sir, no one is screaming, except perhaps you. I feel everyone in this thread has been respectful and has tried to argue in good faith. You left a long explanation and as such left yourself open for rebuttal. I have long come to accept that this request is not happening, and have said multiple times that no one is trying to force you to do anything, but I saw this as an opportunity to maybe find some common understanding. Unfortunately, you don't seem to have any desire to give people the respect of thinking they may know or see something you don't.

I did you the kindness

Yes, and I thanked you for it. Your behaviour after that point has been poor.

I will leave you with that. I can tell you are frustrated by this issue, but frustration is not a good mindset from which to approach a mutually respectful conversation.

[ayush-finix]

Thank you for the explanations, it's really illuminating into your thoughts. I really appreciate the x-java worlds you've laid out.

I guess we fundamentally disagree on the ability for ojava to come about. Spring, hibernate, jackson, are pretty major libraries with optional support and pretty much firmly in the mjava mode, that I can name off the top of my head. Given that most libraries continue to support java versions <8, it might take a long time for the full transition to happen, but I expect apis to gradually change in the third party library world to just intrinsically be optional based or even provide both apis (as many of them currently do to allow multiple version support). Many already split 8+ because of lambdas. With pattern matching (and maybe value types) for java 17, I only expect more third party libraries to move in this direction.

Base sdk support is actually the pain point. As to map.get, you're right, I don't think there are amazing answers, but I'm actually okay with the insanity of mjava as you say, if it means I get to mostly work in ojava and I just need to wrap a few sdk calls.

If you meant 'null', the reference value - then this statement is misleading. null is one way to solve NUI, why would it be 'ideal' if it 'did not exist'? That can only be said if some other NUI solution exists that is superior. There isn't one.

The reason I push this is because I believe ojava is superior. In practice, you can have languages with no null values, since we do have things like haskell and ocaml existing to use as references. I'm clearly biased, but I'd like java to get closer to this, regardless of how feasible the 100% end goal is. I'd actually have no issue with them creating a new Map interface method for Optional<V> get(Object key).

Even if njava came about and was perfectly supported, I'd still be using optionals, for the reason that it's not just about knowing if a value can be null or not, it's also about handling it in a very standardized and generic way. As an example, I hate go code given that this is idiomatic

v1, err := f1()
if err != nil {
    return
}
v2, err := f2(v1)
if err != nil {
    return
}
v3, err := f3(v1, v2)
if err ...
... 

Manually writing the implementation of flatMap for every line of code is not something I want to do. I'll never understand why people think this is good/simple code and not a noise filled mess that fails to convey what you actually want done (I thought people went in to programming to avoid repetitive tasks). Similarly, I want to chain optional methods with flatMap, something that using annotations doesn't help alleviate. Perhaps I'm really not the best person to ask because I also use third party libraries to add the Try class and you'll see signatures like Try<Optional<String>, Exception> method(String a) (although I bet you can figure out some legal inputs and outputs and some meaning through this signature alone). The benefit is that I don't need special ide/annotations/toolchains/ecosystems to make this work; I just need javac and the existing type system.

To one of the examples

And optional stands no chance. There are only 2 things with optional: Yes, or No.

String x; // in Optional world, this cannot hold NUI Optional x; // potentially NUI: NoValue, Unknown, or Irrelevant

That's 2 too few. Generics has 4:

List<? extends Number>; // covariance List<? super Number>; // contravariance List; // invariance List; // raw / legacy variance

This is not a great example because people are going to nitpick on the actual example given that this is legal

Optional<? extends Number>; // covariance
Optional<? super Number>; // contravariance
Optional<Number>; // invariance
Optional; // raw / legacy variance

and doesn't even seem to be your point at all. I'm still highly confused what other options there are for a given value over yes, it has a value or no, it does not have a value. Is there a third state to a normal null check I'm missing? If you're using optionals, why would you represent any data with null? What is this example trying to show?

Without necessarily getting into every single point, I guess the reason that I've even commented on this issue, is that within my own code, I try to work in ojava. I don't need methods that sometimes handle null or allow null, or deal with lists with a null value as a valid element of the list. I do make the assumption that the type String does not have null as a legal value within my own code. Actually, this discussion, more than any other, makes me realize just how arbitrary the don't use Optional as a field/parameter warning really is, so I think I'll actually be okay with just @Value. People in scala and a whole host of functional languages manage to figure it out just fine without any special syntax.

I understand the backwards compatibility concerns you've raised, but I guess my personal feeling is that progress forces sacrifices. I don't expect old java code to break (most of the time [the javax removals in 11 stands out]), but I'd hardly consider being stuck with old apis as a great status quo as well if it's known to cause so many errors. Like the swing example, it's not their fault that they had to work with the tools they were given, but sometimes you just have to make choices that will hurt for a greater good (does anyone really think defaulting to synchronized is a good idea?). Collections, generics, lambdas are all examples that, for better or worse, did obsolete perfectly "fine" old code. Records and pattern matching is probably going to do it again.

[rzwitserloot]

@ayush-finix

Your entire post is wishful thinking and lacks falsifiable statements or logical reasoning. It's a lot of 'I believe' and 'I want' and 'I think'. The term 'personal feeling' shows up multiple times. This is not the venue.

Is there a third state to a normal null check I'm missing?

Yes. A list can either allow strings, or it can't. So why are there 3 ways to write List<String> (String, ? extends String, ? super String)? If that doesn't help you understand it, then reread my example method that duplicates the first element that matches a predicate and write it for me. fully. In ojava. You won't be able to.

[ayush-finix]

@ayush-finix

Your entire post is wishful thinking and lacks falsifiable statements or logical reasoning. It's a lot of 'I believe' and 'I want' and 'I think'. The term 'personal feeling' shows up multiple times. This is not the venue.

I get that there was a lot of personal thoughts in it, but it was to explain my pov, not to be objective truth. But to blow it all off as there are no falsifiable statements or logical reasoning? I'm looking at popular java libraries and other programming languages handling of NUI values. Hell, something like scala has both Option and null as well because of java interop.

Do you really want me to go more in depth into exact library support? I didn't think it was relevant to point out things like spring-data Optional<T> findOne method and jackson's jdk8 module supporting optional. Clearly the libraries went from 0 support (pre optional) to some support (post optional). Whether the growth will continue is an open question. However, it is objectively true that there exist popular java libraries in m-mode for the foreseeable future.

Do you want me to give more examples from other languages that don't have nulls or that do have nulls and maybe types together?

Is your concern about the concept of Optional types or just the implementation of Optional into java? I thought you were just pointing out the implementation pain of changing apis to use Optional, which is what the whole first part of the comment was about how third party libraries were already moving in that direction, but now it seems like you disagree with an Optional type as a concept. If we started the language over, do you think ojava makes sense, theoretically?

Using a maybe class (of any sort) fundamentally works because a Maybe != T

And that's why optional is so bad.

But that's the entire point. It's how you get the actual compiler to ensure correctness without toolchain work or faking dependent types with annotations or adding something like the checker framework to everything. I now understand your point of the backwards compatibility ramifications, but you can also think of the plus side of having nice convenience methods to handling NUI concepts easily (flatMap/map to chain, orElse to unpack) and the plain java compiler helping programmers with their declared intent.

This seems to be our point of contention. You seem to feel that changing existing apis to use optional is a massive burden that could be avoided by adding more toolchain support, while I clearly feel that mixed apis on the path to all optional is acceptable. While I think we can respectfully disagree (and at the end of the day lombok is yours), citing your position as objective and logical while there literally exist languages (scala) that have gone down the second path or new languages that exist on both sides of the case (go vs rust as an example) is disingenuous at best.

Is there a third state to a normal null check I'm missing?

Yes. A list can either allow strings, or it can't. So why are there 3 ways to write List<String> (String, ? extends String, ? super String)?

Subtyping, which seems like an orthogonal issue from nullability or what the types mean.

If that doesn't help you understand it, then reread my example method that duplicates the first element that matches a predicate and write it for me. fully. In ojava. You won't be able to.

Disregarding the point that (in my world) there wouldn't be null values in the list so the implementation of mine and your version would look virtually identical, if you wanted to do the most mechanical of translations:

public <T> List<T> duplicateFirstMatch(List<T> list, Predicate<T> predicate) {}

to

public <T> List<Optional<T>> duplicateFirstMatch(List<Optional<T>> list, Predicate<Optional<? super T>> predicate) {
  return Stream.concat(
      list.stream(),
      list.stream().filter(predicate).findFirst().stream()
  ).collect(Collectors.toList());
}

seems to work just fine? Granted, this

Predicate<Optional<Object>> pred = x -> !x.isPresent()
duplicateFirstMatch(list, pred)

won't work off the top of my head, but why wouldn't you bind

Predicate<Optional<? super String>> pred = x -> !x.isPresent();
duplicateFirstMatch(list, pred);

anyway. I'm sure someone else with better generic knowledge could even get Predicate<Optional<Object>> to work since it should be reasonably similar to a List<List<T>>, but maybe I'm mistaken here. A few comments: There is no intention of protecting against a null list or null predicate, since these are just straight programmer errors in oworld. The Predicate taking an Optional<? super T> forces someone writing the predicate to "handle" the NUI case since the received parameter of the lambda is an Optional. They don't need to null check this since this is also a programmer error in oworld.

[rzwitserloot]

but it was to explain my pov

This is not the venue.

Do you really want me to go more in depth into exact library support?

2022, maybe. As is, the time budget on this has been spent for the 2021 session on nullity debates. But, yeah, that would have been better than a long, unfalsifiable list of your beliefs.

give more examples from other languages

Of course not. Other languages have absolutely no bearing on this issue; it's barely relevant for language design debates for java itself, and is virtually entirely irrelevant for lombok specifically. Talk about java.

but now it seems like you disagree with an Optional type as a concept.

This is incorrect. I've always disagreed with Optional type as a concept. I've stated both:

• Optional in general is not a great solution to NUI, for any language.
• For java in particular, Optional is much worse, in that it has no way to retrofit existing APIs, and java (the ecosystem) has a ton of existing API and a long-standing tradition to not abandon existing API (citation needed, but, the burden of proof isn't on me!)

Subtyping, which seems like an orthogonal issue from nullability or what the types mean.

Then you still do not understand. Read on.

public <T> List<Optional<T>> duplicateFirstMatch(List<Optional<T>> list, Predicate<Optional<? super T>> predicate) {

This is broken. Here, it doesn't work:

List<String> list = ...;
// duplicate first long string
duplicateFirstMatch(list, str -> str.length() > 10);

Before you follow up with: public <T> List<T> duplicateFirstMatch(List<T> list, Predicate<? super T> predicate) {

That one doesn't work either, because that one would fail here:

List<Optional<T>> list = ...;
// duplicate first present entry
duplicateFirstMatch(list, optStr -> str.isPresent());

I want a single method that will work for both. Here is a hypothetical annotation based system that delivers, and works, today, right now, in real java. From checkerframework:

public <@PolyNull T> List<T> duplicateFirstMatch(List<T> list, Predicate<? super T> predicate) {
    for (T t : list) if (predicate.test(t)) {
        list.add(t);
        return;
    }
}

I can call this with List<@Nullable String> and it'll:

• Work (compile, run, and do what you'd think)
• Check (as in, if my dFM code runs, say, t.toLowerCase(), that would cause a compile-time error explaining that I'm risking a nullpointer exception here.
• Read (as in, PolyNull is documented, and tooling and docs can figure out from just the signature that this code works with either variance of nullity, but that the predicate must also be able to deal with it accordingly).

I can also call this with `List<@NonNull String>:

• It works just the same (compiles, runs, and do what you'd think). In particular, the .add(t) call will not cause a 'potentially adding null to a list of non-nulls' error, because the compilation process realizes that t cannot be null.
• Check (as in, if my dFM code runs, say, list.add(null), it will complain).
• Read.

Documentation on @PolyNull

Show me how to do this with Optional.

[ayush-finix]

I'm sorry if it feels like I'm wasting your time. I understand that this issue is not going to change, so if you don't wish to respond, I'd still like to thank you for the time you have taken. At this point, this is more a learning exercise over anything else.

This is incorrect. I've always disagreed with Optional type as a concept. I've stated both:

* Optional in general is not a great solution to NUI, for any language.

* For java in particular, Optional is much worse, in that it has no way to retrofit existing APIs, and java (the ecosystem) has a ton of existing API and a long-standing tradition to not abandon existing API (citation needed, but, the burden of proof isn't on me!)

I understand your thoughts on point 2, and I don't disagree with your assessment to the scope of the changes.

Point 1 seems like an incredibly bold statement to make which absolutely requires justification on your part. Why is a maybe type bad for haskell or rust? This statement seems absurd.

Of course not. Other languages have absolutely no bearing on this issue; it's barely relevant for language design debates for java itself, and is virtually entirely irrelevant for lombok specifically. Talk about java.

How do we discuss Optional in a vacuum? For java, we'd need api changes with no special toolchain support.

Then you still do not understand. Read on.

public <T> List<Optional<T>> duplicateFirstMatch(List<Optional<T>> list, Predicate<Optional<? super T>> predicate) {

This is broken. Here, it doesn't work:

List<String> list = ...;
// duplicate first long string
duplicateFirstMatch(list, str -> str.length() > 10);

Before you follow up with: public <T> List<T> duplicateFirstMatch(List<T> list, Predicate<? super T> predicate) {

That one doesn't work either, because that one would fail here:

List<Optional<T>> list = ...;
// duplicate first present entry
duplicateFirstMatch(list, optStr -> str.isPresent());

Fail in what sense?

public static <T> List<T> duplicateFirstMatch(List<T> list, Predicate<? super T> predicate) {
return Stream.concat(
list.stream(),
list.stream().filter(predicate).findFirst().stream()
).collect(Collectors.toList());
}

with

public static void main(String[] args) {
List<String> example = List.of("1", "2", "12345678901");
System.out.println(example);
example = MyClass.duplicateFirstMatch(example, s -> s.length() > 10);
System.out.println(example);

List<Optional<String>> example2 = List.of(Optional.of("1"), Optional.of("2"), Optional.of("12345678901"));
System.out.println(example2);
example2 = MyClass.duplicateFirstMatch(example2, o -> o.map(s -> s.length() > 10).orElse(false));
System.out.println(example2);

}

gives

[1, 2, 12345678901] [1, 2, 12345678901, 12345678901] [Optional[1], Optional[2], Optional[12345678901]] [Optional[1], Optional[2], Optional[12345678901], Optional[12345678901]]

What am I missing? If you're using Optionals then having a `List<String>` or a `List<Optional<String>>` with a null value is a mistake that you don't need to guard against since there's no semantic case where it means what you want.
So if I see `List<String>`, I can assume there are no null values, and if I see `List<Optional<String>>`, I can assume there are no null values, and if I see any `List<?>` I can assume the list never contains null values, and further that the `List` itself is never null. This it what it means to work in the full ojava world: there are no null values allowed.
I don't need multiple versions of the duplicateFirstMatch method (one for `@Nullable` vs one for `@NotNull`), I just need a single duplicateFirstMatch method with exactly the signature you listed out because we've removed the `@Nullable` one from the scope we have to care about.

[rzwitserloot]

absolutely requires justification on your part

I've provided it. In rather extreme detail. That variance thing, for starters. There's more, but this isn't the venue for a full teardown on NUI systems.

This statement seems absurd.

[citation needed]. Logical reasoning, falsifiable arguments, or don't bother. This is neither.

How do we discuss Optional in a vacuum?

Vacuum? What are you talking about? The concept of NUI can be stated in terms of any number of libraries out there. Something as simple as "how should a language deal with the notion that a call like map.get() needs to convey 'whoops, that value is not in this map in the first place" is a fine place to start discussion.

Feel free to bring the choices of other languages in, but only if you use that to share technical details on what problems they encounter, examples of use, but with the understanding that you lose some in translation (different idioms, different cultures). All you were doing is saying 'Optional is great - these languages use it and they are great'. Most of that statement is neither logical, nor falsifiable. The only part that one can falsify is 'these languages use it'. Sure. But that doesn't make the entire sentence either logical, or falsifiable, thus, it's not an argument and it has zero convincing power.

Fail in what sense?

It doesn't compile. List<String> is not compatible with List<Optional<String>>. Try it. Write it, run javac on it, show me that this works fine. You can't, because it doesn't.

public static <T> List<T> duplicateFirstMatch(List<T> list, Predicate<? super T> predicate) {

That is not the code you wrote before. It also fails - but down the line. I can't add 'non-NUI' typed strings to this list. Contrast to e.g. List<? extends Number> which lets you provide either a List<Number> or a List<Integer>. Two different types; types that are ordinarily completely incompatible with each other (Given List<Number> x = ..; List<Integer> y = ..;, neither x = y; nor y = x; compiles. And yet, I can write a method where you can pass either x, or y. One method. Two incompatible types. And nevertheless, works fine.

I want one method. Two incompatible types (List<String> and List<Optional<String>>, or, to bring it to the actual point, List<However you would like to convey the notion of String that cannot have no-value> and List<However you would like to convey String-or-no-value>) - but with some sort of marking on that method, that the compiler gives me two things:

• You can pass both of these otherwise incompatible types.
• The typing rules applying to that method expand to ensure safety. As in, no matter what the method does, it should both work and not break invariants regardless of which of the various compatible types is actually passed. In the case of List<? extends Number>, the compiler will not let you write list.add(someNumber);. That's to ensure you don't end up adding a double to a List<Integer> by accident. I want this safety as well.

I'm not going to talk about this example any further, we're 10 layers deep and at this point, either I suck at explaining it, you suck at understanding this, or you aren't in the right mindset to consider problems with the language feature you have chosen to champion. More words aren't going to fix any of that.

[ayush-finix]

Okay, in the interest of respecting your wishes, I'm not going to follow up.

There's more, but this isn't the venue for a full teardown on NUI systems.

Where can I see the full teardown you're referencing for my own understanding? I'm still somewhat thrown for a loop as to how Maybe doesn't work in Haskell given it has no subtyping (so the variance problem is literally not a problem) and actually provides lifting functions. I'm honestly not sure what an alternative in that language even is, but let's ignore that and talk about java.

Edit: Found the place where the ideas are listed: https://github.com/rzwitserloot/lombok/wiki/COMPLEXITIES:-Nullity-in-the-type-system

And I think I found our main point of disagreement:

@NonNull String is a subtype of @Nullable String

As you've pointed out, using Optional means that this is not true. this line

I can't add 'non-NUI' typed strings to this list

makes sense from your perspective as you'd have to explicitly box a String into an Optional<String>, which I view as desired and you view as broken. Since there's no shot that java adds a similar system to scala implicits (which would resolve this in a global [not specific for only nullability] sense), this is pretty much the end.

With this edit, I'm done on this, and thanks for your time.

[randakar]

Well, I wanted to write a long, in depth response to the part where @rzwitserloot replies to my post, but I've seen other people have tried before me. And I doubt I'll do better.

@ayush-finix, @michaelboyles, thanks.

@rzwitserloot

I'll only respond to one thing: You seem to think no developer ever uses Optional, but so far all teams I've worked on have contained developers just like that. Usually multiple.

It depends on the ecosystem you're in.

When still stuck in java-ee based old fashioned teams, yes, Optional seems to not have gotten as much traction as with the more modern Spring Boot / Quarkus based projects. In those cases you do run into grumpy people who don't like it. But in modern, young teams? I've seen entire application architectures that mandate oworld.

And lombok. Especially lombok. Because the combination solves a lot of the problems you have when you want to quickly develop applications in an agile fashion. Lombok tackles the boilerplate, Optional and Streams tackle the nigh endless null checking you have to do all over the place. You're looking at massive reductions in LOC required to write even the simplest things.

Frankly, I find it stupifying that your primary objection seems to resolve around arcane cornercases where you want to do mixed typing of generic collections. Needing something like that happens what, maybe once in a hundred projects?

Real world application development of real world API's isn't about having fancy-pants methods like that. It's about dealing with the complexities of a million cornercases, and simply removing null from the equation helps significantly there. Above and beyond all, that's why people want to use it. Declare null anathema, banish it as soon as possible so that your core business logic can be as complicated as it needs to be without also having to deal with null factoring as a complexity multiplier.

This isn't zealotry. It's not ideology. It's about solving real world problems where you have to find the application logic in between 10.000 null checks and even then can't be sure whatever you are handling right now can or cannot be null and therefore, needs yet another null check.

[rzwitserloot]

You seem to think no developer ever uses Optional, but so far all teams I've worked on have contained developers just like that. Usually multiple.

No; I think those developers are mistaken, and are digging a hole in the hope that, eventually, if only they dig long enough, you'll see daylight again. Except, they are digging down, and lombok is not going to give them the shovel.

It depends on the ecosystem you're in.

A java program that rarely uses java.* and doesn't use any other library at all is the one ecosystem where doing something like having an Optional<T> getX() method is sensible. That's far too exotic a use case to be considered boilerplate. Anything else is back to that 'you are only digging deeper' problem.

When still stuck in java-ee based old fashioned teams

See, you seem to think that not using optional is 'old fashioned'.

Optional and Streams tackle the nigh endless null checking you have to do all over the place.

What are you talking about?

You don't get to avoid NUI checks because you use Optional. On the contrary: Because variance is eliminated when using optional, you get more of them.

Optional makes it possible to determine at write/compile time which API calls might return NUI or not. Versus code that doesn't use optional, where this information is still conveyed, but, you'd have to read the docs. Thus, Optional brings you a combination of 'reading the docs via auto-complete dialogs' (which is a great feature to have), and compile-time/write-time checks.

Neither reduces the amount of null checks you do, unless you speak of 'defensive null checking', where you call a method and you have no idea if it could return null or not, so you toss in a null check.

"Doctor it hurts when I press here!" "Well stop doing that then". Don't write defensive null checks. If you aren't sure if an API can return null, that implies you have no real idea what null might mean. Don't write code to deal with cases you don't understand unless that code crashes and logs. An NPE will crash and log for you.

You're looking at massive reductions in LOC required to write even the simplest things.

[citation really really REALLY needed]. A zealot tends to use unfalsifiable hyperbole. Et tu brute? Cut it out. Stop making idiotic claims based on wishes and unicorns. If presence of null is adding "massive LOC", you're doing it wrong.

I find it stupifying that your primary objection seems to resolve around arcane cornercases

My primary objection is that Optional cannot be introduced to the java ecosystem without obsoleting almost every library in java's long and stories existence. The fact that you haven't figured that out from discussion leaves me with the strong feeling that you aren't considering the issues much and just yelling in frustration.

I get that you're frustrated. I'm trying to help you out by explaining things. Apparently, it's not working.

simply removing null from the equation helps significantly there

Also, if we could just wish programs into existence, that'd help significantly.

You can't remove NUI from the equation. It's inherent in the problems programming is trying to solve. Sure, you can solve NUI in a different form. That doesn't get you anywhere near the mythical gains you seem to think it will, though. You're just kicking the can down the road.

Declare null anathema, banish it as soon as possible

This isn't zealotry. It's not ideology.

LOL.

[rzwitserloot]

I've fleshed out my thoughts on this in the wiki: null: Just the tip of the iceberg

It addresses a great many misunderstandings that are being claimed here. A few in no particular order:

@ayush-finix Where can I see the full teardown you're referencing for my own understanding?

I couldn't find a paper, so I wrote it myself, I guess 😛

@ayush-feniks How do we discuss Optional in a vacuum?

Like this.

@randakar Declare null anathema, banish it as soon as possible

This is confusing NUI-the-reference-value and NUI-the-concept, or possibly NUI-the-reference-value and NUI-the-type-tag. The wiki article should help.

[randakar]

You seem to think no developer ever uses Optional, but so far all teams I've worked on have contained developers just like that. Usually multiple.

No; I think those developers are mistaken, and are digging a hole in the hope that, eventually, if only they dig long enough, you'll see daylight again. Except, they are digging down, and lombok is not going to give them the shovel.

The further I dig the more I like where I end up. Maybe you should try it sometime. Seriously. Try it sometime. Fire up a project, preferably something in spring boot world, start using it in all the ways you loathe, and then see where you end up. And then, only then, honestly evaluate what problem you end up having.

It depends on the ecosystem you're in.

A java program that rarely uses java.* and doesn't use any other library at all is the one ecosystem where doing something like having an Optional<T> getX() method is sensible. That's far too exotic a use case to be considered boilerplate. Anything else is back to that 'you are only digging deeper' problem.

Java, Spring Boot, hibernate, jackson, guava, lombok, JPA, apache commons, junit, mockito, restassured. At minimum. And I use Optional<> as field values because in my view it's just a container exactly the same as any collection class.

When still stuck in java-ee based old fashioned teams

See, you seem to think that not using optional is 'old fashioned'.

That's because it is. It's java 6 style thinking.

Optional and Streams tackle the nigh endless null checking you have to do all over the place.

What are you talking about?

The snippet posted here: https://github.com/rzwitserloot/lombok/issues/1957#issuecomment-740239532 That's reality. Lots of bad programmers around who confuse null with empty. Entire frameworks like JAXB that initialize fields in a model with null, and then you end up in codebases where half the code consists of this crud.

You don't get to avoid NUI checks because you use Optional. On the contrary: Because variance is eliminated when using optional, you get more of them.

How does reducing the number of possible values a field can take increase the complexity of a program? Because that's what the approach is here.

Optional makes it possible to determine at write/compile time which API calls might return NUI or not. Versus code that doesn't use optional, where this information is still conveyed, but, you'd have to read the docs. Thus, Optional brings you a combination of 'reading the docs via auto-complete dialogs' (which is a great feature to have), and compile-time/write-time checks.

"You'd have to read the docs" in a 600-file program that has no or nearly no documentation? Don't make me laugh. You do not always have that luxury. Sometimes you have autogenerated code to deal with that doesn't give you that luxury. Sometimes you end up in an old project and you have to clean things up one step, one null check, at a time.

At which point compile-time checking is your only option, and a

Optional.ofNullable(getFoo()).map(foo -> foo.getBar()).map(bar -> bar.getFinalValue()).orElse(defaultValue)

is the only reasonable way to deal with it. Which sure, implies a bunch of null checks, but I no longer have to explicitly write out every single one of them.

Neither reduces the amount of null checks you do, unless you speak of 'defensive null checking', where you call a method and you have no idea if it could return null or not, so you toss in a null check.

"Doctor it hurts when I press here!" "Well stop doing that then". Don't write defensive null checks. If you aren't sure if an API can return null, that implies you have no real idea what null might mean. Don't write code to deal with cases you don't understand unless that code crashes and logs. An NPE will crash and log for you.

I don't have to write defensive null checks if all data that comes in from the outside is coming in nicely prepackaged in Optional fields.

That's the ideal world I want to get to. Field foo is an optional, bar is not, I know right then and there that foo can be empty and I need to deal with it somehow. The next step is dealing with it, as early as possible.

Annotated fields work too for that, but a) you seem to forget not every project has Checker Framework inside. Nor the capacity to configure it, and b) it's nowhere near as explicit. The IDE can only help you a little, whereas with the code version the necessity of dealing with it is clear up-front.

You're looking at massive reductions in LOC required to write even the simplest things.

[citation really really REALLY needed]. A zealot tends to use unfalsifiable hyperbole. Et tu brute? Cut it out. Stop making idiotic claims based on wishes and unicorns. If presence of null is adding "massive LOC", you're doing it wrong.

My last project went down 50% in terms of total lines of code required after I did the conversion to lombok, optionals and streams. I expect my current one (which is much, much worse) to go down more than that. Honestly though, I can easily blame most of that on other factors. But look at that series of .map() invocations up top. Every single one saves you 2 out of 3 lines required for the if( foo != null ) {} version. If you have enough of that, it adds up. And up.

I find it stupifying that your primary objection seems to resolve around arcane cornercases

My primary objection is that Optional cannot be introduced to the java ecosystem without obsoleting almost every library in java's long and stories existence. The fact that you haven't figured that out from discussion leaves me with the strong feeling that you aren't considering the issues much and just yelling in frustration.

I feel like my point of view is being persecuted. I am being called a 'zealot' for no good reason. I am not threatening to burn you at the stake, am I?

And as for 'obsoleting every library' - eh, many libraries are obsolete. Jaxb is obsolete. Serialisation is obsolete. The Date class is obsolete. Hell, everything goes obsolete, if you wait long enough. What's the point? Optional doesn't force those libraries to change. Time does. Change or die. That's life.

But in the meantime, the libraries I do use seem to mostly have no problem with it. Except for, you know, the ones that aren't changing with the times. See that list up top? Almost all of them are there because they support my workflow, and yes, most of them added support for Optional in some way, shape or form.

So mworld is already here. It really is.

The irony is - Lombok mostly doesn't have to support it. Except for this particular feature request (which I'm not even supporting. Not really) lombok doesn't need to know.

I get that you're frustrated. I'm trying to help you out by explaining things. Apparently, it's not working.

The thing I am frustrated about is the persecution of my point of view. The whole "Optional is an anti-pattern" bs. I honestly cannot see how trying to solve a problem by applying a new level of indirection in the form of a container class is suddenly a problem. Collection<Foo> foo is fine, Optional<Foo> foo is the devil? Where is that standpoint coming from? Meanwhile, I am definitely feeling a general sense of loathing and hatred from some people. For no good reason. Calling Optional<> and antipattern while promoting using a class that does exactly the same thing except less well? Great way to make yourself feel superior, but not a great argument.

I know you try to explain, but what I am mostly hearing is that you want to keep null around and deal with situations where null and non-null values can be mixed freely. I however do not. I want to replace null with something that javac is actually capable of catching for me, by saying null is not just a value. I can't go type getFoo().getBar().getRealValue() without risking nullpointer exceptions if any of these are potentially returning null. I have to think about it, when I already have limited headspace for thinking about it. Instead, I want to think about solving my customer's problem, and getFoo() returning an optional is exactly the way to make it clear this thing may not actually be there in whatever model I am working with, while also giving me a tool to deal with that in the form of the various functional methods of that class.

And really, I do not care about the politics. If NUI-annotation world arrives and solves the problem, I will gladly use it. Until then, you can pry Optional from my cold dead fingers, which does solve the problem, today.

simply removing null from the equation helps significantly there

Also, if we could just wish programs into existence, that'd help significantly.

Heh, don't be suprised if that last bit becomes a reality. I've seen talk of AI-driven coding assistance becoming a thing ..

You can't remove NUI from the equation. It's inherent in the problems programming is trying to solve. Sure, you can solve NUI in a different form. That doesn't get you anywhere near the mythical gains you seem to think it will, though. You're just kicking the can down the road.

I don't want to remove it, I want it to become an exception that stands out the second it is a possibility. And then deal with it immediately.

Declare null anathema, banish it as soon as possible

This isn't zealotry. It's not ideology.

LOL.

You can laugh but it's supremely practical. I make it sound like theology but it's simply a way to reduce the implicit invisible complexity of the problem space by making the exceptions explicit.

I've fleshed out my thoughts on this in the wiki: null: Just the tip of the iceberg

Thanks.

Let me state here, for the record, that I really appreciate you doing this. Despite the frustration. Despite talking to each other through a text window instead of for realsies.

I understand how much time this thing is taking you, not to mention energy, and I understand it kinda sucks. I honestly don't know how to get on the same page with you on this issue but the attempt is appreciated.

One thing I'd say on it though is I'd rather not have to annotate everything with @NotNull. That should be the implied default, long term, and @Nullable the exception.

@randakar Declare null anathema, banish it as soon as possible

This is confusing NUI-the-reference-value and NUI-the-concept, or possibly NUI-the-reference-value and NUI-the-type-tag. The wiki article should help.

I'm definitely referring to null, the reference value. I do not, per sé, care about NUI in itself. Just about dealing with it today, in a very practical sense, rather than in some nebulous nworld future.

And really, seriously, I do not think the sky will fall with people using the Optional class the way it (apparently) wasn't intended. It already works today just fine, in mworld.

The only request I am making is to stop the persecution. Stop calling this an anti-pattern when this line of reasoning is entirely legitimate, even if you feel it misguided. Stop acting all superiour (and for the record: This is mostly aimed at the peanut gallery, rather than you - where I know some commenters on other tickets must be watching) about how bad and ugly Optional is and therefore how stupid the people must be proposing this. Stop calling us zealots. (Especially when I can make a pretty good case the reverse is actually true.) That makes you feel good about yourself, maybe. But it doesn't help. You're pushing someone else down into the mud just to make yourself feel good. I am deadly serious, because I lived it.

It's not ugly. It's rather a beautiful, simple way to deal with a very practical issue that pops up all over the place. No extra dependencies or annotations needed. No need to annotate the hell out of all of your fields and method arguments. Just a single coding standard and you're done.

[rzwitserloot]

How does reducing the number of possible values a field can take increase the complexity of a program?

You didn't. A @Nullable String can hold all possible string values and null. An Optional<String> can hold all possible string values and Optional.none(). You gained nothing on the reference side. The only gain optional offers you is in type checking.

I want it to become an exception that stands out the second it is a possibility.

nullity annotations accomplish this as well, are more powerful, and can be introduced without breaking existing libraries.

One thing I'd say on it though is I'd rather not have to annotate everything with @NotNull.

You clearly haven't used these annotation based nullity libraries. Which is a bit rich, what with your opening paragraph in your previous post. It's not necessary at all.

I make it sound like theology

So you agree that you make it sound like theology, but you're upset with me that I'm using the term 'zealot'. Perhaps you should stop making it sound like theology then.

I'm definitely referring to null, the reference value.

Oh, great! Then you have absolutely no need for Optional at all, and the only question left to answer is why you bothered to proselytize in this thread. Optional is for folks who want NUI reference values but want NUIness conveyed in the type system. It's not for folks who think NUI is anathema, to be banished on sight. (your words).

[michaelboyles]

Thanks a lot for the wiki and the effort taken to write it. It's very clear and concise. I'm sure you'd hoped that it would put rest to the discussion for now, so I don't expect any reply to the below. Just a closing statement...

I think the key point of disagreement I have with you is in regard to your definition of backwards compatibility and how that must be handled.

The JDK team almost never make backwards-incompatible changes. I would suggest that a synonym of backwards compatible is "non-breaking change" (semver seems to agree with me, though not explicitly, but through implication of one of the FAQs). Optional is very clearly not a breaking change. All code that worked before Optional still works now; nothing has been broken.

this API is now obsolete: It's just wrong, now. It should be public Optional<V> get(Object o)

Absolutely, and advocates of Optional would agree with that statement. It is wrong by today's standards. But does that mean that Map is broken by the introduction of Optional. The worst you can say about the "legacy" Map interface is that it does not follow modern conventions (you may debate the degree to which the use of Optional is a convention in the wider community, but I'm talking about conventions of the JDK itself, which has proveably used and is continuing to use Optional in new components).

breaking java in two is not worth that

You state this as though it hasn't already happened. Optional has been introduced, and judging by Java's history, will never be rescinded. HttpClient and foreign memory access are using it. The latter is in active development; if they'd realised they'd made a mistake, they would have stopped using it by now.

So from the facts I know, which are that the JDK authors

• are not planning to revise e.g. Map interfaces
• are actively using Optional (outside the context of Streams )

I can infer several things. Namely, that the JDK authors

• are okay with Java being split in two
• think that Java being split in two is actually better than what we had before
• will continue to use Optional, and that the divide will grow larger, at least in the JDK

I'm not meaning to put words in anyone's mouth, but I think those are reasonable inferences based on what I've seen Brian Goetz and Stuart Marks say on the matter, and how I see the JDK evolving. I've seen them acknowledge some mistakes about the design of Optional, but its mere existence is not one of them. If you have any evidence to the contrary, please show me.

I know you wanted falsifiable statements, but these three things are all a matter of opinion. My opinion is that the JDK authors are right: Java is better off with Optional than without it, even if that means a fractured world. You may wish that NUI would have been solved more elegantly, and may hold out hope that one day it still will be. I'm sure you're right that it's technically possible that it could be. But I don't want to deal with crap code while I hold out for that future. We've been proffered with a solution that works* right now and I, and many others, are taking it.

I would not underestimate the influence that the standards established by the JDK have on the Java community. If it's good enough for them, it's going to be good enough for a great deal of Java devs too.

you would probably debate the degree to which "it works", WRT variance etc. You're probably right that it's not optimal. All I can say is that from my experience of actually using it, it works for me*.

[rzwitserloot]

@michaelboyles The worst you can say about the "legacy" Map interface is that it does not follow modern conventions

In other words, that the Map API is now 'obsolete', not by its own choice, but forced upon it because modern conventions in your hypothetical land of unicorns and wishes have changed. I'm not going to debate what backwards compatibility means with you - that's just kibbitzing about words. It's crystal clear that widespread adherence to what Optional is trying to do means j.u.Map is problematic. Whether you call it 'backwards incompatible' or 'obsoleted' or simply 'no longer up to modern conventions', I don't care what words you use - that is bad. Java introduced both generics and lambdas in an update, and both of these introductions went out of their way to ensure that existing API either wasn't affected at all, or, that existing API could update to 'modern conventions' in a backwards compatible way.

It is clear to me that neither you nor anybody else in this thread has managed to come up with a way to make Optional happen without harming APIs like Map.get, and it ends there for me. Optional is veto'ed until you can fix this.

that the JDK authors

One of the most prolific and authoritative JDK authors is Brian Goetz. You can search the web for his opinions on Optional. You're wildly misunderstanding them if you think JDK authors are on board for optional-returning getters, or that they've willingly signed up to splitting the JDK ecosystem in two.

but I think those are reasonable inferences based on what I've seen Brian Goetz and Stuart Marks say on the matter

No, they are not.

I know you wanted falsifiable statements, but these three things are all a matter of opinion.

So why are you posting? This is a feature request thread, not a public vote, as I have made rather clear. I don't actually know of a good venue to hold these discussions, so I've let this go on for a while, but, I've been making it clear we need to drag this back to falsifiable statements. It's gone on long enough - this is not the venue. Further opinion posts are no longer welcome. Show me fleshed out proposals, analyses of the community, suggestions on how existing APIs can follow along without breaking compatibility or feeling obsolete/ 'no longer following modern conventions' - those are okay.

[rzwitserloot]

@randakar wrote:

I'm not crusading against other people there. Just against null, in my own code. That's not zealotry, that's just being thorough.

I just finished deleting a bunch of your posts and the traffic that it caused in another thread because you were crusading for Optional in it (with everybody involved immediately shooting it down as preposterous, which really makes that zealot label stick). It's unfortunate I have to resort to this, but you're banned from this issue. Any further comments from you will be deleted, clearly we've gone beyond any attempt to reason and build towards something resembling a feature request. Please refrain from crusading for Optional in any other issue or PR as well.

[walec51]

this whole discussion boils down to:

1. One part of the Java community thinks Optional adds no value to the language - only makes things more messy and inconsistent and should almost never be user.
2. Second part of the Java community thinks Optional gives them cleaner code and they want to use it extensively.
3. Both of these groups are numerous and have their arguments.
4. @rzwitserloot whats to use his lead position over his popular library to enforce his opinion on the subject over the Java community

@rzwitserloot and his collaborators gave us a great library and have the right to do so. This does not diminish the fact that we should be grateful for the work they put in to it.

That said we can also have an opinion on their hard stance on the subject. I personally think its somewhat immature. I've used to avoid Optional once too but for many of my coworkers it actually made them write cleaner code and was more readable. Now I always allow my teams to decide on a project by project basis do they want to use Optional in every method that returns an optional value or should we avoid it everywhere. I'm ok with either approach as long as it will be consistent in the entire applications code.

Despite your opinion @rzwitserloot there is a possibility of a compromise here. If you would allow us to - for example - to add a global parameter lombok.dreaded-optional-support=true (false by default) and allow us to implement different rules for generating getters on @Nullable attributes when its on (and other enhancements like this). Then we would be all ok - you could still state your opinion on why Optional is bad everywhere in the documentation.

If you are not ok with that @rzwitserloot then I think this discussion should end now. There is no point in dragging it on.

Every one that wants to see Optional support in lombok should stop talking and start coding a fork. Talk is cheap, write the code. We can even be friendly about it and try to keep the code base in sync by just adding Optional support and releasing a library called lombok-with-optional. In the future maven statistics would tell which option proved to be more useful to developers.

[rzwitserloot]

to use his lead position over his popular library to enforce his opinion on the subject over the Java community

Evidently this needs to be mentioned. This is a feature request for Project Lombok. Not a general forum.

Despite your opinion @rzwitserloot there is a possibility of a compromise here.

This thread is a strong indication that it isn't possible.

If you would allow us to - for example - to add a global parameter lombok.dreaded-optional-support=true

I don't see how you can make the promise of maintaining this for 10 years. Which makes the point moot: No can do.

If you are not ok with that @rzwitserloot then I think this discussion should end now. There is no point in dragging it on.

That does seem to be the proper conclusion, yeah.