Improving lsp configuration for better developer experience

Describe the enhancement:

Ran into two specific LSP problems that made it hard to work on the fleet-server. The goal of this issue is to investigate and if possible resolve these problems.

Conflicting build flags

The lsp client needs to be configured to recognize the build tags used in the repo. Prior to configuring these tags, we get the following error:

No packages found for open file <path to fleet-server>/fleet-server/testing/e2e/agent_install_test.go.
   This file may be excluded due to its build tags; try adding "-tags=<build tag>" to your gopls "buildFlags" configuration
   See the documentation for more information on working with build tags:
   https://github.com/golang/tools/blob/master/gopls/doc/settings.md#buildflags-string

Providing the lsp with the necessary build flags resolves this problem, but we run into another one that is caused by conflicting build tags used in the packages. For example if you look at handleFileDelivery_test.go and metrics_integration_test.go you can see that they belong to the same api package but have conflicting integration and !integration tags. When we configure the lsp client to recognize one of them, it stops working for the other. The current solution is manually updating the lsp config and restarting the client while working on either one of these files. This situation would be a bigger problem in case a developer needs to switch between two files with conflicting build tags. We should look into ways of improving the way we configure our lsp clients regarding build flags.

Nested modules

Language server couldn't resolve packages in the submodules due to nested go modules. LSP displayed the following error

> error while importing github.com/elastic/fleet-server/testing/e2e/scaffold: build constraints exclude all Go files in <path to fleet-server>/fleet-server/testing/e2e/scaffold

This issue has been addressed in PR #3397, recommending developers to include go.work and go.work.sum files in their local repository clones. These files were previously omitted due to concerns raised in PR #3380. While I recognize the proposed addition may seem minor, I believe it will improve our development experience, albeit in a subtle way.

Describe a specific use case for the enhancement or feature:

The specific use cases have been explained in the section above.

elastic / fleet-server

Improving lsp configuration for better developer experience #3424