padreati / rapaio-jupyter-kernel

Java jupyter kernel
MIT License
53 stars 7 forks source link

improve dependency declaration with extended attributes #42

Closed padreati closed 6 months ago

padreati commented 1 year ago

It seems that Maven allows pom modules to have dependencies which are not declared. In order to collect those kind of dependencies the syntax and logic for adding dependencies should be improved.

Use case: org.lwjgl:lwjgl-glfw:3.3.3:natives-macos-arch64 where the last token is a classifier attribute in maven pom.

According to a post from stackoverflow: https://stackoverflow.com/questions/35052763/downloading-lwjgl-natives-with-ivy

padreati commented 1 year ago

An alternative is to use https://github.com/maveniverse/mima as @maxandersen suggested. It needs to be studied and dive deeper into it.

maxandersen commented 1 year ago

The advantage of Mima is it's consistent with the broader maven artifact system. There are edge cases especially in the overlap of maven and gradle published artifacts that ivy does not handle in same ways.

padreati commented 8 months ago

I started to develop a replacement for dependency handling using MIMA. It seems it works fine.

maxandersen commented 8 months ago

Any chance the syntax could be same as in jbang so no need to change ?

I'm also open on adjusting in jbang if there is some case not covered.

padreati commented 8 months ago

It is an interesting question which raised some options and considerations which I took before implementing the magic as they are. I will try to be brief on it, but it will take some text since there are many things to be discussed.

I agree with your option to use // as the start of DEPS command. I think the reason is to be friendly with java compilation and also to be compatible with any Java editor which could be used to write jbang code. However, this works only for one line magic commands. I am wondering what could be used for multi line (or cell) magic commands?

I know, or at least this is my understanding from a limited research over jbang documentation, that you do not have to treat the case of a cell magic command. For notebooks is simple to define a multi-line magic commmand, that would be limited by cell itself. This is not available in a Java file, thus a terminal marker is needed. Additionally, in order to be compatible with Java editors, I suppose it should stay also in a comment. For that there are two options: /**MAGIC_KEY ... */ or /*MAGIC_KEY ... */.

All the above treats the syntax problem. This can be solved with some effort. The benefit would also be that it eventually be compatible with running the same code from jbang with no additional changes. If you have other suggestions for syntax I would gladly hear. Especially because I don't fully like how multi-row magics looks, even if I am not able to identify what it is.

There is something else which is not syntax related and is about how dependency commands behaves.

In jbang there is a single one line command for importing dependencies. If one has to import multiple dependencies, that it write multiple lines or one line with those dependencies enumerated. One problem which I see here is conflict dependencies. What happens when there is a conflicting dependency issue? For illustration one can import two dependencies, both of them depending on the same third package, but with different versions?

I approached this problem in the following way. Dependency magics gives only instructions regarding which dependencies to be added to the current execution, but without actually collecting the dependencies. There is also the possibility to add dependency overrides which instructs the interpreter on how to resolve some conflicts. In the end, there is an one line command which starts actually the resolve process and bringing resources in the execution space.

Solving dependency after each //DEPS command brings some problems which I find hard to solve. For illustration we have the following scenario:

If add dependency do also the collect and resolve, the previous scenario don't work. For example when A is resolved, A and B-1.0 is added to classpath. Even if the override is executed, if it is done later, it would not be possible to unload the dependency from runtime. For some cases it can be solved in the sense that a proper order of declarations could solve that, but it seems to me much harder or error prone.

In the end, thank you a lot for you attention. I will try to make those things compatible, as I like the benefit of using into normal Java editors. I am inclined to make a double syntax available. But I am eager to hear your opinion regarding both syntax and semantic issues.

padreati commented 6 months ago

I solved without mima since I was not able o understand how one can manage conflicts. Postpone for later a study on that topic.