Go to definition not working because of semantic db issue

[yashLadha]

Describe the bug

When opening kafka project in neovim with metals, when trying to go for the definition of module it is saying that the semantic db file doesn't exist so we cannot use the go-to definition in the file.

To Reproduce

Steps to reproduce the behavior:

1. Go to AdminZkClient.scala file
2. Click on RackAwareMode and try to go to definition
3. See the error in the metals log file.

[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/upgrade-system-tests-11-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/upgrade-system-tests-25.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/streams-scala.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/upgrade-system-tests-22.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/log4j-appender-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/upgrade-system-tests-21.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/upgrade-system-tests-25-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/streams-examples.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/file-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/tools.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/clients-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/upgrade-system-tests-22-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/runtime-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/streams.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/basic-auth-extension-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/tools-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/streams-scala-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/upgrade-system-tests-26.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/upgrade-system-tests-23-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/json.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/upgrade-system-tests-27-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/upgrade-system-tests-0101.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/runtime.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/upgrade-system-tests-24-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/test-utils-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/mirror-client-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/upgrade-system-tests-0100.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/basic-auth-extension.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/raft-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/mirror-client.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/upgrade-system-tests-0101-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/mirror-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/upgrade-system-tests-0102-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/api-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/log4j-appender.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/upgrade-system-tests-0110-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/json-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/upgrade-system-tests-21-test.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/clients.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/mirror.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/transforms.json'
[D] Loading project from '/Users/yashladha/opensource/kafka/.bloop/upgrade-system-tests-24.json'
[D] Waiting for a connection at local:///var/folders/pr/f422jxqd2h3f897t3wlf3cl80000gp/T/bsp-launcher416309809016785901/bsp.socket...
The server is listening for incoming connections at local:///var/folders/pr/f422jxqd2h3f897t3wlf3cl80000gp/T/bsp-launcher416309809016785901/bsp.socket...
Starting thread that pumps stdin and redirects it to the bsp server...
Starting thread that pumps server stdout and redirects it to the client stdout...
2021.01.13 07:53:39 INFO  tracing is disabled for protocol BSP, to enable tracing of incoming and outgoing JSON messages create an empty file at /Users/yashladha/Library/Caches/org.scalameta.metals/bsp.trace.json
2021.01.13 07:53:39 INFO  time: connected to build server in 0.22s
2021.01.13 07:53:39 INFO  Connected to Build server v1.4.6-15-209c2a5c
2021.01.13 07:53:39 INFO  time: imported build in 0.19s
2021.01.13 07:53:52 INFO  time: indexed workspace in 12s
2021.01.13 07:53:52 INFO  compiling clients (835 java sources)
2021.01.13 07:53:52 WARN  No processor claimed any of these annotations: org.apache.kafka.common.annotation.InterfaceStability.Unstable,java.lang.FunctionalInterface,org.apache.kafka.common.annotation.InterfaceStability.Evolving,java.lang.SafeVarargs
2021.01.13 07:53:52 INFO  time: compiled clients in 0.69s
2021.01.13 07:54:08 ERROR code navigation does not work for the file '/Users/yashladha/opensource/kafka/core/src/main/scala/kafka/zk/AdminZkClient.scala' because the SemanticDB file '/Users/yashladha/opensource/kafka/.bloop/core/build/bloop-bsp-clients-classes/classes-Metals-o9h4IA1_S8muicfKeMCDqw==/META-INF/semanticdb/core/src/main/scala/kafka/zk/AdminZkClient.scala.semanticdb' doesn't exist. There can be many reasons for this error. 
2021.01.13 07:54:09 ERROR code navigation does not work for the file '/Users/yashladha/opensource/kafka/core/src/main/scala/kafka/zk/AdminZkClient.scala' because the SemanticDB file '/Users/yashladha/opensource/kafka/.bloop/core/build/bloop-bsp-clients-classes/classes-Metals-o9h4IA1_S8muicfKeMCDqw==/META-INF/semanticdb/core/src/main/scala/kafka/zk/AdminZkClient.scala.semanticdb' doesn't exist. There can be many reasons for this error. 
2021.01.13 07:54:09 ERROR code navigation does not work for the file '/Users/yashladha/opensource/kafka/core/src/main/scala/kafka/zk/AdminZkClient.scala' because the SemanticDB file '/Users/yashladha/opensource/kafka/.bloop/core/build/bloop-bsp-clients-classes/classes-Metals-o9h4IA1_S8muicfKeMCDqw==/META-INF/semanticdb/core/src/main/scala/kafka/zk/AdminZkClient.scala.semanticdb' doesn't exist. There can be many reasons for this error. 
2021.01.13 07:54:09 ERROR code navigation does not work for the file '/Users/yashladha/opensource/kafka/core/src/main/scala/kafka/zk/AdminZkClient.scala' because the SemanticDB file '/Users/yashladha/opensource/kafka/.bloop/core/build/bloop-bsp-clients-classes/classes-Metals-o9h4IA1_S8muicfKeMCDqw==/META-INF/semanticdb/core/src/main/scala/kafka/zk/AdminZkClient.scala.semanticdb' doesn't exist. There can be many reasons for this error. 

Expected behavior

Ideally, it should go to the definition of that file. Is there something I am missing?

Screenshots

Installation:

• Operating system: macOS
• Editor: Neovim with builtin-lsp
• Metals version: v0.9.8

Additional context

Search terms

[tgodzik]

Thanks for the reporting! The semanticdb files needed for the navigation are produced during the compilation. Did the compilation at any point finish successfully? I see there is a warning about the annotations, did it stop the entire project from being compiled? I am not sure if annotation processor is supported properly in Bloop currently.

[yashLadha]

Yes compilation is completed successfully and jump to definition works for some files as well.

[tgodzik]

So this might ba an issue with annotation processor https://github.com/scalacenter/bloop/issues/962 :thinking:

This will need to be fixed upstream in Bloop and currently compilation will most likely not finish successfully.

[yashLadha]

@tgodzik I see that the linked issue is merged it latest version of metals released with these changes?

[tgodzik]

@yashLadha that was the plan! However, we will need to do a Bloop release also, so not sure timewise if that will come before the bloop release. In case it's later you should be able to just update the Bloop version in Metals

[tgodzik]

@yashLadha you can try out a new Bloop version 1.4.6-33-1c6f6712 in the Metals: Bloop Version settings. I want to confirm some things first before doing a proper release.

[tgodzik]

Checked it out myself and it seems we are getting:

2021.01.28 15:54:18 WARN  No processor claimed any of these annotations: org.apache.kafka.common.annotation.InterfaceStability.Unstable,java.lang.FunctionalInterface,org.apache.kafka.common.annotation.InterfaceStability.Evolving,java.lang.SafeVarargs

not sure how to handle that in Bloop :/

[thinkiny]

same thing happend

WARN No processor claimed any of these annotations: org.apache.kafka.common.annotation.InterfaceStability.Unstable,org.apache.kafka.common.annotation.InterfaceStability.Evolving,java.lang.SafeVarargs

[thinkiny]

why is this removed from v1.0.0 milestone ?

[ahmedriza]

This still happens randomly. There are no compilation errors at all.

[silles79]

i have the same issue, very big project. compiles fine. but in some files code navigation just doesn't work.

[mbesida]

Have the same issues, output log in vs code prints that compilation has finished, but on some files navigation simply doesn't work with exact same problem

[tanishiking]

if someone uses Scala 2.13.9, see https://github.com/scalameta/metals/issues/4449 workaround: revert to 2.13.8 / use the latest SNAPSHOT of Metals / wait for the next Metals release (it's around the corner!)

[tanishiking]

Also, when you hit the same issue, please attach your

• metals (server) version
• scala version
• what kind of symbol you tried to navigate

That information is really helpful for fixing this issue.

what kind of symbol you tried to navigate is important, for example, code navigation doesn't work for mutable in

val x: collection.mutable.Map[String, Int] = ???

(it might navigate to the package object if exists. maybe we should go somewhere anyway 🤔 ). With this information, we can make things better.

---

This issue has two perspectives.

• (1) Why is SemanticDB not found for the file?
• (2) Even if SemanticDB is not found, Metals will fall back to presentation compiler-based navigation; why doesn't it work?

It sounds like a SemanticDB issue (because we see the log saying Code navigation doesn't work because ... SemanticDB file doesn't exist ...), but it's a kind of presentation compiler's code navigation issue.

We might want to add some more words to the log like "falling back to the compiler-based code navigation..." because the current log message seems a bit confusing.