bzlmod not able to make dependencies available in intellij

[soheil1987]

Description of the bug:

Using bzlmod I am putting a list of all my dependencies in MODULE.bazel file. I run bazel run @unpinned_maven//:pin to update my maven_install.json file. everything goes fine and I see no errors. my bazel version is 6.2.1, env is Windows 10 and intellij version is 2022.3.2. I am able to build my project successfully and all test cases pass. My issue is dependencies are not visible during compile time. I am showing this with screenshots. below screenshot shows when I build the project by listing all dependencies in WORKSPACE file. as you can see all dependencies are available and recognizable and this makes them usable in the code.

the next screenshots show when we use MODULE.bazel to manage dependencies aka bzlmod, dependencies are not available and hence can not be usable inside the code.

the last screenshot shows the impact of missing dependencies. this is a serious issue which makes bzlmod unusable in intellij

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

try to import dependencies into a simple intellij project using two different methods. first classic method using listing artifact with maven_install from load("@rules_jvm_external//:defs.bzl", "maven_install"). second method using bzlmod and MODULE.bazel file. you will see in the first method dependencies are available in classpath during compile time and this make them available to be used inside the code in intellij while in the second method dependencies are not available.

Which Intellij IDE are you using? Please provide the specific version.

2022.3.2

What programming languages and tools are you using? Please provide specific versions.

No response

What Bazel plugin version are you using?

v2023.05.30 ASwB Stable

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[jonfreedman]

This is quite a big deal for us - it renders our bazel code unusable in IntelliJ and if we aren't able to resolve means we can't adopt bazel

[alexeagle]

I'll also point out that the Bazel team has committed to bzlmod being the default in a couple months (with Bazel 7.0), and required for everyone in the major version after that (8.0) so the severity of this issue will increase dramatically.

[mai93]

I gave this issue P3 because it was assigned to me and I do not have the capacity to work on it but I can review a PR that fixes it. I think the PR should be based on the google branch so we can import it internally as we would want the change to be available for AS as well as IJ and CLion plugins.

If another maintainer would like to work on it, this is also an option. @tpasternak @blorente

[tpasternak]

Unfortunately I can't estimate when and if I will have time to work on it

[fmeum]

Would anyone be interested in funding work on this issue, e.g. through the IntelliJ or Rules Authors SIG?

[jonfreedman]

What's involved in funding e.g. how much and what do we get (beyond having the issue fixed)

[fmeum]

This was more of a general question to assess interest, both from potential funders and contributors.

The Rules Authors SIG usually pays on an hourly basis (see https://opencollective.com/bazel-rules-authors-sig), but sometimes also assigns bounties to certain tasks or issues. CC @alexeagle

[mai93]

Does this issue affect only ASwB or also IJwB? Can you provide a simple bzlmod based project and steps to reproduce the problem?

[soheil1987]

what are ASwB and IJwB?

[mai93]

ASwB is the Android Studio with Bazel plugin and IJwB is the IntelliJ with Bazel plugin. I wanted to know if this problem is observed in only one IDE or both are affected

[alexeagle]

https://github.com/bazelbuild/rules_jvm_external/tree/master/examples/bzlmod is a canonical, minimal example of how Bazel Java users are meant to use bzlmod, I imagine you can repro easily with that.

[mai93]

Thanks @alexeagle! I tried importing this project in IntelliJ (with Bazel plugin version 2023.07.18.0.2-api-version-231) and I see the external libraries configured. Am I missing any steps before the import or do I need to change something in the example?

[soheil1987]

@alexeagle @mai93 the difference between https://github.com/bazelbuild/rules_jvm_external/tree/master/examples/bzlmod and the example i provided is WORKSPACE file. WORKSPACE file in https://github.com/bazelbuild/rules_jvm_external/blob/master/examples/bzlmod/WORKSPACE is not empty and apparently that's what makes external dependencies available through intellij. I remember you recommended we should keep WORKSPACE file empty, as you did with quant project, when we use bzlmod and ultimately bazel is going to get rid of WORKSPACE files. also in https://github.com/bazelbuild/rules_jvm_external/tree/master/examples/bzlmod there are two WORKSPACE files, what are the differences?

[mai93]

@soheil1987 I emptied the WORKSPACE file and I'm still seeing the external libraries brought in correctly. Can you please check with this project and let me know what I need to change to reproduce the issue?

[fmeum]

I am also seeing what @mai93 is seeing. In fact, this works for me even on the Bazel repo itself, with common --enable_bzlmod added to .bazelrc.

@soheil1987 How are you enabling Bzlmod? build --enable_bzlmod may be problematic, please check that it is indeed common --enable_bzlmod.

What doesn't work is following load references into external repositories, but that's a separate issue I am looking into fixing.

[fmeum]

also in https://github.com/bazelbuild/rules_jvm_external/tree/master/examples/bzlmod there are two WORKSPACE files, what are the differences?

A WORKSPACE.bzlmod file takes precedence over WORKSPACE if Bzlmod is enabled and additionally removes a few repos that are otherwise automatically appended to WORKSPACE.

[soheil1987]

we are using common --enable_bzlmod inside .bazelrc, our bazel version is 6.2.2 and rules_jvm_external version is 5.3.

[soheil1987]

So i am trying to figure out what can be differences with my current project and https://github.com/bazelbuild/rules_jvm_external/tree/master/examples/bzlmod that can cause unavailability of external dependencies on my side. I tried to add a new dependency to https://github.com/bazelbuild/rules_jvm_external/tree/master/examples/bzlmod and I faced this error Error:no such target '@rules_jvm_external~override~maven~maven//:com_sun_xml_ws_jaxws_ri': target 'com_sun_xml_ws_jaxws_ri' not declared in package '' defined by

can you please take the following steps to see if you can replicate the error:

1- check out https://github.com/bazelbuild/rules_jvm_external/tree/master/examples/bzlmod from command prompt or terminal using git clone https://github.com/bazelbuild/rules_jvm_external.git

2- empty the contents of WORKSPACE file but keep the WORKSPACE.bzlmod file intact.

3- open the project in intellij as a bazel project

4- try to add the following dependency to list of artifacts of maven.install part in MODULE.bazel file. "com.sun.xml.ws:jaxws-ri:4.0.1"

5- add it to the list of deps in java_binary target in [https://github.com/bazelbuild/rules_jvm_external/blob/master/examples/bzlmod/java/src/com/github/rules_jvm_external/examples/bzlmod/BUILD] (https://github.com/bazelbuild/rules_jvm_external/blob/master/examples/bzlmod/java/src/com/github/rules_jvm_external/examples/bzlmod/BUILD)

6- sync the project

[fmeum]

@soheil1987 I tried that and it didn't work with that particular new Maven dependency, but only because rules_jvm_external doesn't seem to pick up all its transitive dependencies correctly.

I tried with org.junit.jupiter:junit-jupiter-api:jar:5.10.0 and that worked, although I had to:

1. Add fail_if_repin_required = True and fetch_sources = True to the maven.install tag.
2. Modify the repin repo name because rules_jvm_external's repin error doesn't construct it correctly with Bzlmod.

Nothing related to IntelliJ though that didn't work.

[Wyverald]

/sub

[alexeagle]

From the OP:

env is Windows 10

I wonder if it repros on Windows but not on Mac/Linux?

[mai93]

Unfortunately Windows is still not officially supported by the plugin but we can accept a PR to fix this.

[alexeagle]

Could you point to which platforms are officially supported? I don't see it on the readme

[mai93]

It is available on the plugin documentation site: https://ij.bazel.build/docs/bazel-support.html

[github-actions[bot]]

Thank you for contributing to the IntelliJ repository! This issue has been marked as stale since it has not had any activity in the last 6 months. It will be closed in the next 14 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-maintainer". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

[github-actions[bot]]

This issue has been automatically closed due to inactivity. If you're still interested in pursuing this, please post @bazelbuild/triage in a comment here and we'll take a look. Thanks!

[Sineaggi]

Seems strange to mark this as not planned.

[mai93]

Reponing, I think the last thing we reached on this is we were not able to reproduce the issue, it may be only happening on windows which is not officially supported by the plugin so the issue is given P3 if one of the community is able to reproduce and provide a fix.

[jonfreedman]

I'd be curious to see usage stats of IntelliJ by operating system - I've worked in software development in financial services for almost 20 years and I have never had any choice but to use Windows as the OS for my day job. I can see in the JetBrains dev survey how many monitors and what disk type people use but I couldn't quickly find the OS... https://www.jetbrains.com/lp/devecosystem-2023/lifestyle/

[hb-man]

@jonfreedman this stat is there, but it's in another section! https://www.jetbrains.com/lp/devecosystem-2023/development/

[jonfreedman]

Thanks, so we can probably assume something along the lines of 64% of JetBrains users are on Windows and this plugin does not support them. This is part of the reason why we stopped using bazel.

[bp-tock]

We're reproducing a similar issue using IntelliJ on Mac after upgrading to Bazel 7.1.1. Following expunging and performing a full Bazel sync, most external libraries and generated files are not being recognized by IntelliJ. Interestingly, the application continues to run normally, so this seems more closely related to the IDE / file indexing.

Here's a sample of our "External Libraries" section which typically contains 100+ external sources / generated files but, when encountering the error, appears nearly empty.

The only reliable way we've found to mitigate the issue is to disable use of our remote cache (i.e., build --remote_cache="") in .bazelrc. After doing so - all seems to work as expected.

Happy to provide any additional information that might help debug.

• MacOS version 14.4.1
• IntelliJ IDEA 2024.1 (Community Edition)
  • Build #IC-241.14494.240, built on March 28, 2024
  • Runtime version: 17.0.10+8-b1207.12 aarch64
  • VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
• Bazel for IntelliJ
  • 2024.04.09.0.1-api-version-241

[tpasternak]

hey @bp-tock

t might be caused by the fact that some of the artifacts are not being downloaded from the remote cache. I'd recommend either:

1. tweaking remote_download flags like this one, or
2. tweaking the targets section in your projectview file, so the missing libraries are directly mentioned there

If it doesn't work, please let us know

[bmt-tock]

hey @bp-tock

t might be caused by the fact that some of the artifacts are not being downloaded from the remote cache. I'd recommend either:

1. tweaking remote_download flags like this one, or
2. tweaking the targets section in your projectview file, so the missing libraries are directly mentioned there

If it doesn't work, please let us know

In our case, it does seem like switching to --remote_download_toplevel seems to fix the issue. So I imagine tweaking the specific remote_download flags like you mention will allow us to fine-tune it.

[tpasternak]

So I'm closing the issue for now, feel free to reopen in case something is still wrong