Open zainab-ali opened 3 years ago
This is a bug in the way that jar classifiers are handled when searching for POMs in the mill GenIdeaImpl
task.
@kxa14, you mentioned that you weren't sure where to start investigating. I kept a log of the steps taken as I looked into this. I wouldn't usually upload them in a GitHub comment (I doubled back on myself and found a few red herrings, so they aren't too concise), but I think you'll find them useful. They may give you some pointers on how to troubleshoot these sorts of issues.
Checkout and run the master version of mill
According to the mill README, we can run mill with
./mill -i dev.run ~/scala/lsug-website mill.scalalib.GenIdea/idea
This fails with the same error
Add some logging
Looking in the out
directory, where mill writes its outputs, there's a out/mill/scalalib/GenIdea/idea/log
file. This contains messages outputted by the GenIdeaImpl ctx.log
function. Incidentally, these log lines helped me locate the GenIdeaImpl
class without knowing much about the mill codebase.
We can add a log line to sbtLibraryNameFromPom
:
def sbtLibraryNameFromPom(pomPath : os.Path) : String = {
ctx.map(_.log.info(s"Parsing POM $pomPath ..."))
val pom = xmlParseDom(os.read(pomPath)).flatMap(Pom.project).right.get
Running the GenIdea
task again gives the log message:
Parsing POM ~/.cache/coursier/v1/https/repo1.maven.org/maven2/io/netty/netty-transport-native-epoll/4.1.46.Final/netty-transport-native-epoll-4.1.46.Final-linux-x86_64.pom ...
This is the last thing printed before the task fails.
Check the path ~/.cache/coursier...netty-transport-native-epoll-4.1.46.Final-linux-x86_64.pom
The pom file doesn't exist.
Why are we searching for netty-transport-native-epoll
?
Following the code, we search for the pom for libraries in buildDepsPaths
. This seems like a collection of paths to all of our build dependencies. We expect netty-transport-native-epoll
to be a dependency in here. Let's add some log messages to check:
val allBuildLibraries : Set[ResolvedLibrary] =
resolvedLibraries(buildLibraryPaths ++ buildDepsPaths).toSet
buildDepsPaths.map(p => ctx.map(_.log.info(s"buildDepPath $p ...")))
This outputs the log message:
buildDepPath ~/.cache/coursier/v1/https/repo1.maven.org/maven2/io/netty/netty-transport-native-epoll/4.1.46.Final/netty-transport-native-epoll-4.1.46.Final-linux-x86_64.jar ...
So netty-transport-native-epoll
is indeed a dependency.
What is netty-transport-native-epoll
anyway?
Epoll is an event notification syscall. The netty-transport-native-epoll
jar contains JNI bindings for it. Loosely speaking, it's a wrapper around the non-Java shared library files for the syscall. It's probably pulled in by http4s. According to the Netty docs on native transports, it should be treated as an ordinary jar on Maven Central.
Should it have a POM?
According to the Maven Central publishing instructions, all releases should have a POM. The Netty build is no exception.
Hold on a minute. Does it have a POM?
Looking into the ~/.cache/coursier.../4.1.46.Final
directory, there actually is one:
netty-transport-native-epoll-4.1.46.Final-linux-x86_64.jar
netty-transport-native-epoll-4.1.46.Final.pom
netty-transport-native-epoll-4.1.46.Final-sources.jar
It's missing a linux-x86_64
suffix, so is different to the file we're searching for.
What's that linux-x86_64
suffix for?
Maven jars can have classifiers. These are arbitrary suffixes that let a release contain artifacts with different names. Looking back at the Netty build, it has a linux-x86_64
classifier. And looking at the Netty docs on native transports again, we see how the classifier is specified when adding the dependency for sbt:
"io.netty" % "netty-transport-native-epoll" % "${project.version}" classifier "linux-x86_64"
Mill should strip this classifier when looking for a POM.
Why doesn't mill strip the classifier?
The logic in GenIdeaImpl looks for a POM extension with the same filename as the jar:
val withoutExt = path.last.dropRight(path.ext.length + 1)
val pom = path / os.up / s"$withoutExt.pom"
It doesn't cater for classifiers. This is a bug in the mill code.
How do we fix this?
One solution would be to look for any file with a POM extension in the same directory as the jar. Releases should (I think) only have a single POM associated with them, so we couldn't accidentally pick the wrong POM.
Another would be to make the POM field optional. I'm not sure what the repercussions on the intellij idea file would be.
The bug
The
mill mill.scalalib.GenIdea.idea
task fails:To Reproduce
Run
mill mill.scalalib.GenIdea.idea
in the root directory.Version details
Additional information
This is likely due to the
GenIdea.idea
task attempting to inspect netty's POM file:In the mill GenIdea code, we see:
The library
netty-transport-native-epoll
doesn't have a POM on maven.I'm as yet unsure why this is (are POM files necessary for libraries in maven?).
Quite possibly, the
GenIdea.idea
task should skip the POM if it doesn't exist. If so, we should work to fix this in the mill codebase.