JDT doesn't react to external annotation paths set by this plugin

[stechio]

This project looks inactive since 2022, so I don't know whether I'm shouting in the desert... :sweat_smile: (Eclipse JDT seems still to support external annotations—although in a buggy and somewhat experimental manner—, so it's not easy to estimate whether this functionality is alive or doomed) Anyway...

Because of the sparse availability of external annotations (as you know, eg, JRE-related *.eea files from lastNPE repo are incomplete), I need to link plain *.eea files to edit them during development through the convenient popup menu command "Annotate" within the corresponding *.class files source view.

Unfortunately, the way Eclipse manages the configuration of external annotation paths is terrible: Eclipse JDT doesn't provide a way to set a common annotation path where to place *.eea files for both JRE and Maven Dependencies (*.eea files for JRE are set at workspace level via "Preferences/Java/Installed JREs" dialog, while for Maven Dependencies are set at project level, forcing users to edit the "Build Path" dialog of each and every project in a workspace!). The only alternative would be the "Search for external annotations in all build path locations" option, but it presumably applies only to external annotations packaged as JARs (thus unsuitable for edit). Icing on the cake, on my freshly-updated installation (Eclipse IDE 4.32, Eclipse JDT 3.19.500) external annotations configuration for Maven Dependencies doesn't work at all ("External annotations" path set within "Java Build Path/Libraries" dialog isn't saved!).

Due to the above-mentioned limitations/bugs in the way Eclipse manages the configuration of external annotation paths, I resorted to this promising plugin (eclipse-external-annotations-m2e-plugin), configured as follows:

1) installed version: 2.0.0.202212171629 2) m2e.jdt.annotationpath property set in the root pom.xml: "${rootdir}/src/main/eea" (all *.eea files in my project are under "src/main/eea" directory — see tree view screenshot here below)

Despite the plugin gloriously worked as expected ("External annotations" paths for both JRE and Maven Dependencies were automatically set on each and every module of my project — see dialog screenshots here below), JDT doesn't react to those paths (nullness errors are still highlighted in my source code, method signatures in Javadoc view are not augmented with nullness annotations, and the popup menu command "Annotate" doesn't show up within the corresponding *.class files source view).

Evidently, there is a severe problem within Eclipse JDT which makes external annotations paths unusable.

Before reporting to JDT maintainers, I would like to know about your use experience and possible suggestions (I wonder why apparently nobody stumbled upon this problem before — is it a recent JDT regression? or nobody is actually using external annotations any more?)

[agentgt]

Yeah its broken (maven + eclipse integration) or at least a good portion of it.

It worked at one point and was great however I was using the workspace location instead of the jars because the workspace location you can easily add annotations in the UI.

I am using it in my opensource libraries: https://github.com/jstachio/jstachio and companies libraries.

The best solution I have found is just to copy all of eea to each project and rely on workspace location and then tell folks to manually set it up. In other words not using this plugin. Yes it sucks ass.

FWIW I have the headless version working for the above opensource library.

https://github.com/jstachio/jstachio/blob/19b99d079ed454be34c6941dc67908c8232a3a3d/api/jstachio/pom.xml#L121-L170

(I wonder why apparently nobody stumbled upon this problem before — is it a recent JDT regression? or nobody is actually using external annotations any more?)

As far as I know the only other major user is @sebthom besides myself.

I'm also the one evangelizing Eclipse null analysis probably more than anyone else at the moment and regularly active on JSpecify and Reddit. I suppose @kevinb9n might be able to vouch for that. This is not a brag btw but more of a cry for help.

I have been meaning to talk to @vorburger about joining the lastnpe org and helping out more including fixing this plugin.

This winter I plan on really cranking hard on getting Eclipse null analysis easier to use as well as make it be able to read astub format as NullAway,Checker and JSpecify reference implementation use that format instead of EEA.

In the meantime I can try to help as best I can with helping you get setup.

[agentgt]

Oh I forgot to say since you got the plugin to load (it doesn't even load for me these days) is that m2e.jdt.annotationpath should be workspace location.

That is it is more like:

<m2e.jdt.annotationpath>/${WORKSPACE_PROJECT_FOLDER_NAME}/src/main/eea</m2e.jdt.annotationpath>

Replace WORKSPACE_PROJECT_FOLDER_NAME with your projects name that shows up in project explorer (usually it is the maven artifact name).

Regular filesystem path and jar I have not had luck with.

EDIT also if you plan on using sealed classes or records you need to use a nightly build or a milestone build as 2024-06 JDT build is severely broken for null analysis.

https://www.eclipse.org/downloads/packages/release/2024-09/m2

[sebthom]

I never used the eclipse-external-annotations-m2e-plugin. For a more complete set of EEAs you can try out https://github.com/vegardit/no-npe#usage

[stechio]

Thanks guys for your help.

@agentgt:

It worked at one point and was great however I was using the workspace location instead of the jars because the workspace location you can easily add annotations in the UI.

That's exactly my own case (workspace location to add annotations via UI).

The best solution I have found is just to copy all of eea to each project and rely on workspace location and then tell folks to manually set it up. In other words not using this plugin. Yes it sucks ass.

As I wrote in my initial comment, my problem is the opposite: Eclipse doesn't let me manually set up the annotations paths (workspace location) in the "Java Build Path/Libraries" dialog, whilst this plugin works! (now I'm even more confused :confused:)

My critical problem is that, after the plugin successfully sets the external annotations paths up (as shown in the "Java Build Path/Libraries" dialog), Eclipse JDT doesn't detect them (no Javadoc view augmentation, no "Annotate" popup menu command for editing, no warning update in source files). Conversely, if I manually set the external annotations path up for JRE at workspace level (i.e., via "Preferences/Java/Installed JREs" dialog), Eclipse JDT detects it (!!).

So, recapping:

• manual setup:
  • JRE (workspace-level "Preferences/Java/Installed JREs" dialog): SUCCESS (nullness analysis works)
  • Maven Dependencies (project-level "Java Build Path/Libraries" dialog): FAILURE (cannot save the paths — after setting them, I reopen the dialog and the paths are still empty!)
• automatic setup (via plugin with m2e.jdt.annotationpath property):
  • JRE (project-level "Java Build Path/Libraries" dialog): FAILURE (path saved (plugin works), but nullness analysis doesn't work!)
  • Maven Dependencies (project-level "Java Build Path/Libraries" dialog): FAILURE (path saved (plugin works), but nullness analysis doesn't work!)

(I wonder why apparently nobody stumbled upon this problem before — is it a recent JDT regression? or nobody is actually using external annotations any more?)

As far as I know the only other major user is @sebthom besides myself.

I'm also the one evangelizing Eclipse null analysis probably more than anyone else at the moment and regularly active on JSpecify and Reddit. I suppose @kevinb9n might be able to vouch for that. This is not a brag btw but more of a cry for help.

As Eclipse is a well-established IDE, I'm puzzled by its apparent lag in nullness analysis adoption :astonished: — is that because most projects rely on IDE-agnostic static analysis solutions like Checker Framework?

JSpecify would be a great remedy to the horrible proliferation of nullness annotation libraries, but it seems a bit languishing, despite being backed by top industry players.

This winter I plan on really cranking hard on getting Eclipse null analysis easier to use as well as make it be able to read astub format as NullAway,Checker and JSpecify reference implementation use that format instead of EEA.

Format interoperability would be important to avoid redundancies in annotation efforts. BUT *fixing external annotation paths setup in Eclipse* (a SINGLE workspace location for ALL annotation files (JRE and Maven Dependencies, whatever the format (*.eea, \.astub, etc))) would be essential***.

Oh I forgot to say since you got the plugin to load (it doesn't even load for me these days) is that m2e.jdt.annotationpath should be workspace location.

I have replaced my m2e.jdt.annotationpath according to your suggestion ("/myproject/src/main/eea", where "myproject" is the actual project name within Eclipse): the plugin correctly updated the "External annotations" paths associated to JRE and Maven Dependencies (as shown in the "Java Build Path/Libraries" dialog), BUT Eclipse JDT still doesn't react to them (!!). Is there any log to verify whether such "External annotations" paths have been loaded by the nullness analyzer of Eclipse JDT? How can I know its status without setting up a remote debugging session?

EDIT also if you plan on using sealed classes or records you need to use a nightly build or a milestone build as 2024-06 JDT build is severely broken for null analysis.

https://www.eclipse.org/downloads/packages/release/2024-09/m2

Nice to know, but in my case I'm targeting good old Java 11, so I'd be happy just if nullness analysis worked at all :grimacing: :sob:

@sebthom:

I never used the eclipse-external-annotations-m2e-plugin. For a more complete set of EEAs you can try out https://github.com/vegardit/no-npe#usage

Thanks for pointing me to your EEAs collection. I'm intrigued by your generator: how can it infer the actual member nullness, since it is informally denoted by javadoc contracts? Does it statically analyze the source code looking for null assignments/checks?

BTW, how do you set up your Eclipse IDE to consume the EEAs? (I'm still struggling to have the external annotations paths work, particularly for Maven Dependencies, as noted here above)

[sebthom]

@sebthom:

I never used the eclipse-external-annotations-m2e-plugin. For a more complete set of EEAs you can try out vegardit/no-npe#usage

Thanks for pointing me to your EEAs collection. I'm intrigued by your generator: how can it infer the actual member nullness, since it is informally denoted by javadoc contracts? Does it statically analyze the source code looking for null assignments/checks?

The EEA Generator does not scan javadoc, however it can interpret different types of annotations (see). It also contains some logic to determine if a field or method has probably null/non-null return/parameter values (see). A lot of EEAs are still maintained manually tho. Manually added/changed EEAs are not overridden by the EEA generator on subsequent runs.

BTW, how do you set up your Eclipse IDE to consume the EEAs? (I'm still struggling to have the external annotations paths work, particularly for Maven Dependencies, as noted here above)

In TM4E for example I just added no-npe as a Maven dependency at scope <provided> (see) and enabled Search for external annotations in all build path locations. To make it work in the CLI with tycho compiler plugin I had to pass -annotationpath CLASSPATH as compiler argument (see).

When I am extending the EEA files I am not relying on the Eclipse ability to add EEAs as I want the EEA files in a specific way (stable sorting for better diffing). So instead for new libraries I generate the EEA files and then manually edit them. For EEAs for e.g. JDK classes I manually update the previously generated EEA files. A big help for editing EEA files is enabling syntax highlighting. I created a TextMate grammer for EEA files which is available via https://github.com/sebthom/extra-syntax-highlighting-eclipse-plugin

While editing EEA files I reference the source folder from the checked-out repo as path for External anotations:

This process works fine for me.

[agentgt]

@stechio

JRE (project-level "Java Build Path/Libraries" dialog): FAILURE (path saved (plugin works), but nullness analysis doesn't work!)

I vaguely remember upgrading recently and null analysis being very broken so I highly recommend you try a milestone or nightly build. That is currently what I'm running because there were so many issues!

I don't know if it will fix the problems you see but there is a good chance it will as I can associate EEA manually with it on a per project (module) basis. Like @sebthom it works fine for me. The Maven stuff does not but I haven't tried for awhile.

As Eclipse is a well-established IDE, I'm puzzled by its apparent lag in nullness analysis adoption 😲 — is that because most projects rely on IDE-agnostic static analysis solutions like Checker Framework?

I would say most projects do nothing as the reality is so many people rely on what IntelliJ's builtin stuff does. Then the ones that do are using headless static analysis on top of it.

Also besides the fact that Eclipse is less used it does one thing that I think might make adoption more difficult. Eclipse assumes unspecified nullness which for the most part you can assume is @Nullable for everything that is not annotated.

Checker on the other hand does the opposite. I actually prefer the Eclipse way but I think it hurts it adoption when you just see the overwhelming shit fest of warnings/errors when you try to make the switch as most code bases do not use null everywhere.

However one big advantage to use both of them is I find incorrect annotations (generally just missing) all the time w/ Checker and then go file a bug PR with them to correct. This is because Eclipse forces you to do something about it (annotate).

JSpecify would be a great remedy to the horrible proliferation of nullness annotation libraries, but it seems a bit languishing, despite being backed by top industry players.

Well it just went 1.0

All I can say is I think it is going to get better. The Java ecosystem is so big that it just moves a little slower but changes are happening.

[vorburger]

I have been meaning to talk to @vorburger about joining the lastnpe org and helping out more including fixing this plugin.

==> https://github.com/lastnpe/eclipse-null-eea-augments/issues/165