Integrating Sync Cache with Goland's Shared Indexes

[ahmedyarub]

Description of the feature request:

Storing the sync results into Goland's shared indexes to enable sharing with other machines.

What underlying problem are you trying to solve with this feature?

Not requiring to do a Bazel sync on earch machine.

What operating system, Intellij IDE and programming languages are you using? Please provide specific versions.

All. For now we are only interested in Go.

Have you found anything relevant by searching the web?

https://bazel.build/remote/caching and https://www.jetbrains.com/help/go/indexing.html

Any other information, logs, or outputs that you want to share?

No response

[blorente]

While this is a useful feature we'd love to have, we don't have the bandwidth to implement it right now. We're open to PRs, and I'm happy to provide guidance/reviews if I can.

[ahmedyarub]

So I have officially started working on such feature. I'm considering disabling it by default and implementing it incrementally. The method I'm considering is imitating the STARTUP sync type, copying the files from a remote repo and just running a STARTUP sync. I know that we need to make that the cache is in sync with the local files etc... but let's take it one step at a time.

[ahmedyarub]

After thinking thoroughly about it and especially the case of loading a part of a mono-repo, I'll do this first and I think I know what I need to do it.

[ahmedyarub]

1- Add a method that creates a compressed file containing all the intellij-info files. This is to be used by CI/CD tools mainly to populate a central repo. 2- Upon synchronization, the plugin should first parse the targets and then try to get their intellij-info files from the remote cache. If they do exist, then we can run a NO_BUILD sync which creates the project's proto cache. 3- Versioning of cached intellij-info files need to be considered. 4- I'm also considering using the native IntelliJ Shared Indexing

[tpasternak]

BTW isn't it possible possible to share the ide-info files among the machines via regular bazel's remote chace? We only need to ensure the outputs are reproducible so we can get action cache key hit

[ahmedyarub]

BTW isn't it possible possible to share the ide-info files among the machines via regular bazel's remote chace? We only need to ensure the outputs are reproducible so we can get action cache key hit

Sorry I couldn't understand the difference between your and my approach.

[tpasternak]

2- Upon synchronization, the plugin should first parse the targets and then try to get their intellij-info files from the remote cache. If they do exist, then we can run a NO_BUILD sync which creates the project's proto cache.

If you have a bazel remote cache already set up, you can just build the ide-info files in your CI using the plugin's aspect and then they will be downloaded during build phase so in theory you could avoid building anything

[ahmedyarub]

2- Upon synchronization, the plugin should first parse the targets and then try to get their intellij-info files from the remote cache. If they do exist, then we can run a NO_BUILD sync which creates the project's proto cache.

If you have a bazel remote cache already set up, you can just build the ide-info files in your CI using the plugin's aspect and then they will be downloaded during build phase so in theory you could avoid building anything

Nice I'll be working on this then together with the other suggested features related to target management. What about version management? How can we be sure that we are pulling the correct version? I do have my own ideas but I want to hear more from you.

[blorente]

What about version management? How can we be sure that we are pulling the correct version?

If using the Bazel remote cache, the ide-info files generated by the aspect will be cached based on the hashes of the input files. So, if a version changes (either on a third party dependency, on the IDE, or in your source code), the cache key of the ide-info will change, so invalidation is transparent.

I agree with @tpasternak that using Bazel's remote cache is "free" and correct if you already have it set up.