Aether creates zombie folders under .m2

[CeylonMigrationBot]

[@akberc] This is minor from an effect point of view, but maybe not, as it creates folders in a user's existing .m2 repo that is supposed to be read-only.

• add aether as a repo in config through text editor or Eclipse
• add a dummy module - or maybe a misspelling or wrong version (as I stumbled on it)

module one.amod "1.0.0" {
    import org.dummy.some "99.1";
}

• folders created under .m2

[Migrated from ceylon/ceylon-module-resolver#73]

[CeylonMigrationBot]

[@thradec] I guess that this is normal maven/aether behavior, it creates folder even when artifact is not found and store there some meta-data about it.

[CeylonMigrationBot]

[@akberc] Yes, relatively minor issue, and reproducible in maven goals except mvn clean which I think only checks for existence.

However, CMR -

• does not leave meta-data (last tried time and repos tried) in the 'zombie' folders - hence named 'zombie' :). The file name itself tells that there was an attempt and which remote repo was tried.
• requests for 'ceylon.' bypass aether - which is good, but
• - 'ceyloon.' invokes aether - which is how I stumbled on it
• - Herd will soon have modules other than those beginning with 'ceylon' so aether-enabled projects might be looking for ceylon modules, creating junk.

I guess we can

• close or re-visit in the fture
• or tie it to #4734, so that we use the aether 'existence check' only first (local and remote), and go one better than Maven
• change the built-in settings.xml to leave some metadata droppings. But since we have little use for the metadata (unlike Maven, which presumably uses it to throttle traffic)

I would lean to the existence-check only and solve it with #4734, if possible. This way, aether support is 'cleanly' read-only and 'transfer if found'.

[CeylonMigrationBot]

[@quintesse] Again, I think we should be very careful when trying to by-pass existing Maven functionality, if there's some kind of problem with Aether (or Shrinkwrap) we should take it up with the appropriate upstream project instead of trying to work around it ourselves.

Btw I agree this is minor, although rubbish in the .m2 folder isn't nice it's also not a big problem IMO, the .m2 folder is not read-only, it's a cache, and for Maven it contains half the Internet anyway. ;)

[CeylonMigrationBot]

[@gavinking]

Again, I think we should be very careful when trying to by-pass existing Maven functionality, if there's some kind of problem with Aether (or Shrinkwrap) we should take it up with the appropriate upstream project instead of trying to work around it ourselves.

+1

There are things we need to find our own workarounds for, but this isn't one of them.

[CeylonMigrationBot]

[@akberc] Agreed this specific issue is minor and we can close it or schedule it for 1.2 or even 1.2+ ? so that it doesn't get off the radar?

So, just to clear up a few assumptions (above) so that a future release can look at this in more detail, and to document the role of aether and Shrinkwrap:

• Maven is a build tool and we are not building Maven artifacts nor are we using Maven. So, we are not using 'Maven functionality'. We are using an open multi-format repo library (aether) that happens to have a Maven repository format implementation - and that could easily have been extended for Ceylon repo format. The library allows a non-repo-manager mode and existence-check mode etc. which can be used.
• Upstream aether library has all the features needed for our use cases -- so it is perhaps incorrect to say that this and related niggles are a limitation of upstream libraries. We don't use the aether library directly - we use aether through Shrinkwrap which is not ideal for our use case -- that is the main problem that doesn't allow quick one-line fixes/enhancements as with other CMR issues.
• Perhaps the main reason for Shrinkrwrap being involved and CMR being forced to configure it through settings.xml is because of the requirement of Plexus container services lookup and injection, which can be easily bypassed and I think have gone away with the eclipse aether release.

This is just one of many minor niggles including a race condition I've spotted with aether in settings.xml mode (not proven yet) when building a large project. They all add up to slightly less than 'solid support' for Maven. BTW, with Herd, CMR is great, now that the network HEAD issue is fixed.

I think the best approach would be to gather these issues under a meta-issue 'Use aether directly for next release' or something less prescriptive, and then at least they remain open for 1.1 or 1.2, so that they don't fall off the radar. This is assuming that 'aether is the future' as commented a few days ago when discussing 'mvn:http'. If you want, I can oblige ;)

I am also looking at a light-weight Ceylon implementation as a ceylon.build task so that Maven artifacts can be used in a project without aether. That would give another option and also solve multi-repo, proxy and classloading of dependent artifacts.

[CeylonMigrationBot]

[@gavinking] If ShrinkWrap is the problem wouldn't it be better to fix ShrinkWrap?

[CeylonMigrationBot]

[@akberc] Yes, I will take another look at it. Last cursory check was not very encouraging :)

The specific use case for a Ceylon project, with many possible solutions:

• If a module M defined in the module descriptor cannot be found in Ceylon repos, look for a matching jar in configured Maven repos.
• If a jar is not found in local Maven repo but found in a remote Maven repo, copy to local Maven repo. (or Ceylon cache repo)
• Configure inter-jar dependencies for the runtime so that dependent jars of M are transparently loaded in the same classloader.

If you can correct this and/or give high-level direction on the solution, then yes, we can focus on Shrinkwrap, or CMR or ceylon.build.

[CeylonMigrationBot]

[@quintesse] @akberc Like I said before, @alesj , the guy that made the CMR, only "just" specifically switched to using Shrinkwrap from using Aether directly, I'm sure there were reasons but maybe you should get in touch with him to find out. You at least seem to know Aether.

I'm not sure how far Aether goes with respect to supporting Maven, but like I've said before, people that use Maven a lot tend to have a whole environment set up about it, with their own proxies and company-wide repositories etc. Doing anything that does not automatically make use of that would be a big mistake in my opinion. If people have to figure out how to redo their configuration in Ceylon-speak (and probably finding out that Ceylon doesn't support some of the things Maven does) they're going to be put off. I'm also pretty convinced that if we try to do things ourselves (for example: oh we just download it and copy it to the .m2 cache) we're in for a world of hurt.

So please, if there are problem lets first focus on upstream, we have a direct line to the Shrinkwrap developers, before trying to come up with any other ideas.

[CeylonMigrationBot]

[@akberc] Thanks. These discussions on individual issues don't have much value as they go around in circles, therefore it would be a good idea to bring the 2 or 3 use cases together and see whether the toolset supports them.

For the record: @quintesse Don't know where the stuff in the last message came from as I have not advocated any of that. A developer who knows Maven will only have the classpath loading issue which is separate. We all agree on that.

My issues has always been that due to modular structure, Ceylon supports Java through Maven or through import-jar, and developers who don't know, or use, Maven will have problems. The concern is based on first-hand personal experience with developers who checked out Ceylon projects. It may hinder uptake of a beautiful language.

Can't stop ideas ;) , but will work upstream :)

Thank you once again.

[CeylonMigrationBot]

[@quintesse] Although I don't personally see much of a problem here I think we can keep this around until 1.2 as a kind of tracking issue. If nobody complained about it / fixed it by then we can close it.

[CeylonMigrationBot]

[@akberc] Thanks @quintesse - Agree that this is a minor problem, but please keep open as a reminder so that we don't forgot to try to improve it.