Resolve additional dependencies

[nicolaferraro]

When creating complex integrations, it's often needed to have some constants and objects that are contained in the specific jar of the component used. It would be nice if the tooling could be able to add them on the basis of information given by the CLI.

I'd add a CLI command kamel inspect dependencies that when run on a directory inspects all Camel K files and find all dependencies needed by the defined integrations (it can be also scoped to a single file). A possible output would be:

[
  "mvn:org.apache.camel:camel-core:3.1.0",
  "mvn:org.apache.camel:camel-aws-s3:3.1.0"
]

The tooling should monitor Camel K (and configuration) files for changes and invoke the command to check if something has changed.

Do you think this is possible? Or do you have a different proposal for this scenario? @bfitzpat @apupier

[bfitzpat]

That's very interesting. I think it would definitely be useful.

We also have the explicit dependency flag on the command line:

kamel run -d mvn:com.google.guava:guava:26.0-jre -d camel-mina2 Integration.java

I'm wondering what kind of flow you expect between these commands? Also, how this might work with the --save flag?

Just taking a SWAG here, but maybe something like this in the tooling:

• Right-click on a raw (undeployed) integration and click "Inspect Dependencies"
• Display those dependencies in the Apache Camel K output channel
• User then has to note the dependencies and either specify them in the drop-downs or in the task configuration upon deployment

That seems pretty clunky. Maybe this would work better:

• Add a new drop-down option (or setting) that automatically adds dependencies when we start the integration
• Under the covers we call kamel run integration.java -dependencydetect (or something) which uses this new inspect functionality under the covers to add all dependencies to the deployment

And then this behavior also:

• right-click on a deployed integration and select "Inspect Dependencies"
• Display those dependencies in the Apache Camel K output channel
• Pop up a question notification asking if the user wants to update the deployed integration with those dependencies
• Somehow pass the dependencies via a CLI call to update the deployed integration

So yes, I think this would be great for diagnostic information but I think we have to go further maybe?

[bfitzpat]

Or do you have a different idea of how this might flow @nicolaferraro ?

[nicolaferraro]

That's very interesting. I think it would definitely be useful.

We also have the explicit dependency flag on the command line:

kamel run -d mvn:com.google.guava:guava:26.0-jre -d camel-mina2 Integration.java

I'm wondering what kind of flow you expect between these commands? Also, how this might work with the --save flag?

Just taking a SWAG here, but maybe something like this in the tooling:

• Right-click on a raw (undeployed) integration and click "Inspect Dependencies"
• Display those dependencies in the Apache Camel K output channel
• User then has to note the dependencies and either specify them in the drop-downs or in the task configuration upon deployment

That seems pretty clunky. Maybe this would work better:

• Add a new drop-down option (or setting) that automatically adds dependencies when we start the integration
• Under the covers we call kamel run integration.java -dependencydetect (or something) which uses this new inspect functionality under the covers to add all dependencies to the deployment

And then this behavior also:

• right-click on a deployed integration and select "Inspect Dependencies"
• Display those dependencies in the Apache Camel K output channel
• Pop up a question notification asking if the user wants to update the deployed integration with those dependencies
• Somehow pass the dependencies via a CLI call to update the deployed integration

So yes, I think this would be great for diagnostic information but I think we have to go further maybe?

No actually the point of this feature is not to provide a way for users to know the dependencies to pass to the command line. If Camel K knows that to run your code you need a specific library, currently it adds the library automatically without you having to specify it. So the inspect command cannot find something useful for the user.

The point of this is providing more code completion and error highlighting. Currently the IDE provide code completion for Camel routes and other things that are present in the camel core library. But sometimes this is not enough.

E.g. yesterday I had to write this to customize an integration.

That works when you run it in Camel K, because it detects from the main integration that the camel-aws-s3 library needs to be used. But that class is marked with errors in VSCode and there's no code completion for the packages and classes used there.

So the inspect command can technically be used to know what libraries need to be added to the project classpath in order for VSCode to recognize real errors and provide completion.

Of course, kamel inspect cannot be able to understand what libraries you'll use when you pass the dependencies using command line. But if you --save them, then kamel inspect should be able to pass it to the IDE.

We were also thinking to provide a lightweight way to specify dependencies using comments on the code. That would also be some logic embedded in kamel inspect in case.

[bfitzpat]

@apupier is out this week, but it's definitely something that sounds useful to add to the java language support features in the tooling. We can take a look at this next week and get back to you if that's ok?

[nicolaferraro]

@apupier is out this week, but it's definitely something that sounds useful to add to the java language support features in the tooling. We can take a look at this next week and get back to you if that's ok?

Sure, it's not urgent, so better discussing it a bit

[apupier]

That works when you run it in Camel K, because it detects from the main integration that the camel-aws-s3 library needs to be used. But that class is marked with errors in VSCode and there's no code completion for the packages and classes used there.

a manual configuration is possible currently. It requires users to know which jars are needed and download them, then to configure java.project.referencedLibraries in settings of VS Code.

The tooling should monitor Camel K (and configuration) files for changes and invoke the command to check if something has changed.

Do you think this is possible? Or do you have a different proposal for this scenario?

it is possible. it is a bit reinventing a project structure. it can cause some delay in completion availability when several Camel K files with different dependencies are used in the same workspace. We are relying on jdt.ls mechanism, it is providing a single classpath per workspace for non-classical project. So it means updating the classpath everytime we switch the file, with potentially new dependencies to download.

Which configuration files are you mentioning?

[nicolaferraro]

It's not decided yet, but we may add in the future additional discovery mechanisms, such as from "application.properties" key prefixes or annotations in the source files.

Ok, doing it at every change may be a bit too much, but how about doing it periodically? Dependencies do not change frequently.

What set of base dependencies do you currently include?

[apupier]

It's not decided yet, but we may add in the future additional discovery mechanisms, such as from "application.properties" key prefixes or annotations in the source files.

or a largely used descriptor such as a pom file? :-)

on standalone files, for declaring dependencies maybe worth looking to what jbang is proposing https://github.com/maxandersen/jbang#declare-dependencies (i have no idea if it is a good idea or not)

Ok, doing it at every change may be a bit too much, but how about doing it periodically? Dependencies do not change frequently.

If we are updating periodically, based on time, but not on which file is opened, it means that when we switch of Camel K files, the classpath can be completely unrelated (apart from the camel-core-engine)

What set of base dependencies do you currently include?

dependencies of camel-core-engine https://github.com/camel-tooling/vscode-camelk/blob/ea3559c7320b2898364af0b394618dfed0adb25e/resources/maven-project/pom-to-copy-java-dependencies.xml#L28

[nicolaferraro]

Yeah, we didn't want to reinvent another standard, but also didn't want to use maven since it's much more powerful than a simple dependency descriptor..

I think that using a project-global classpath is acceptable at the moment. We auto-discover only camel-xxx dependencies, so it does not matter a lot if integration A forces the extension to put e.g. camel-telegram into the classpath and that becomes available also to integration B: if B uses those classes, than it's because it needs camel-telegram as well.

In the future we can improve it, but currently the important thing is that the IDE does not signal errors for unknown classes that will be later known at runtime.

[apupier]

kamel inspect command to help support of it https://github.com/apache/camel-k/issues/1738

[apupier]

created https://issues.redhat.com/browse/FUSETOOLS2-826 to track it