`inspect` should show some useful info about modules (500USD Bounty)

[lefou]

---

From the maintainer Li Haoyi: I'm putting a 500USD bounty on this issue, payable by bank transfer on a merged PR implementing this.

---

Currently, mill inspect only works for targets, but not for modules. For module, it just outputs an error message "inspect Cannot find default task to evaluate for module foo".

Instead, it should provide some useful information:

• Scaladoc, source location (filename:linenumber)
• the inherited / mixed-in modules
• the available tasks
• an optionally defined default task
• potentially known sub-modules or cross instances

[Shri333]

@lefou can I take a stab at this? are there any expected challenges/hurdles, or should the task be relatively straightforward?

[lihaoyi]

@Shri333 go for it. It should be relatively straightforward. It will probably involve changes to the mill-moduledefs plugin to save the scaladoc for objects and lazy vals to Java annotations, and then updates to inspect so when it sees a module it uses Java reflect to fetch the corresponding info to display

[Shri333]

I am trying to build mill's source (main). It failed with JDK 21 but worked with JDK 17 (./mill installLocal). I am also not getting any syntax highlighting for .mill files and BSP appears to be inactive in IntelliJ (I built idea files with ./mill -j 0 mill.idea.GenIdea/idea). I also tested with VSCode + Metals and syntax highlighting is inactive there as well. Is there anything I have to do to get the syntax highlighting and build configurations to appear (for debugging)?

[lihaoyi]

@Shri333 for IDE support you can set up a Scala <-> *.mill file association via Settings / Editor / File Types / Scala in Intellij. That should let IntelliJ's BSP support work and do the right thing. I'm working on getting it set up by default in IntelliJ but for now you can configure it manually.

Mill is currently only built and tested using Java 17, so not surprising it doesn't build cleanly on newer versions. We'll get to upgrading our CI at some point

[lefou]

It's not Mill, that's not running with Java 21, it's only some test cases, which exercise older libraries/tools not yet compatible with Java 21.

• see also https://github.com/com-lihaoyi/mill/pull/2767

[Shri333]

Is there a particular debug configuration that you use in IDEA?

[lihaoyi]

@Shri333 typically I run tests from the terminal and use println debugging. I'm not actually that familiar with usage of the IntelliJ debugger haha

[Shri333]

It seems to no longer output an error anymore (the List(foo.run) was added by me)

dshrihan@Shrihans-MacBook-Air example % ../mill/target/mill-release inspect foo
Mill version 0.12.0-RC1-4-172b61-DIRTY22702cc6 is different than configured for this directory!
Configured version is 0.12.0-RC1 (/Users/dshrihan/Projects/example/.mill-version)
[1/1] inspect 
List(foo.run)
foo.run(RunModule.scala:81)
    Runs this module's code in a subprocess and waits for it to finish

    args <str>...

Inputs:
    foo.finalMainClassOpt
    foo.runClasspath
    foo.forkArgs
    foo.forkEnv
    foo.runUseArgsFile
    foo.finalMainClass
    foo.forkWorkingDir

[lihaoyi]

@Shri333 that's because it resolves to the foo.run task on RunModules. But in cases where there is no defaultCommandName it fails, and even in cases where it does have a defaultCommandName, we would like it to display the metadata about the module itself and not just the default command that has been configured

[Shri333]

Ok that makes sense. Thanks!

[Shri333]

@lihaoyi when transforming to copy scala doc into java annotations in the module defs plugin, which type in global covers objects and lazy vals? I did not find ObjectDef or LazyValDef

[lihaoyi]

objects are ModuleDef, lazy vals are ValDef with a flag iirc

[Shri333]

So, there are no changes required to this compiler plugin?

[lihaoyi]

I don't know, you'll have to tell me haha

[Shri333]

Looks like it works for objects. I got the following working:

/**
 * test scaladoc
 */
object hello extends Module {
}

[1/1] inspect 

    test scaladoc
1 targets failed
inspect Cannot find default task to evaluate for module hello

[Shri333]

with the following changes:

        case r: Resolved.Module => {
          val a = r.cls.getAnnotation(classOf[mill.moduledefs.Scaladoc])
          println(mill.util.Util.cleanupScaladoc(a.value).map("\n    " + _).mkString)

[Shri333]

What NamedTask should I use to represent the module information itself? It seems like Command is used for inspecting specific tasks (i.e. inspect bar.run).

[lihaoyi]

The point of inspect is the terminal output, so whayever you do in the implementation is up to you really. Maybe wrap it in an Either[Module, NamedTask] and then match on the either when it comes time to print it?

[Shri333]

Yeah I was thinking about doing that. I kind of wanted to stick to the existing pattern though. I will try that.

[Shri333]

I am trying to push my branch with local changes to create a draft PR, but I do not have access. @lihaoyi can you add me?

ERROR: Permission to com-lihaoyi/mill.git denied to Shri333.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

[lihaoyi]

you need to fork the repo on github and push to your fork to create a pr from that

[Shri333]

What is the easiest way to get file/line info? Module contains millOuterCtx but that is the outer context (not the context for the actual build file with the module I am trying to inspect). Should I pattern match on some base class of Module?

[Shri333]

For example, I tried the following, but it is not perfect

[lihaoyi]

@Shri333 is the module's own Module#millOuterCtx not correct? My understanding is that it is populated in the Module.BaseClass constructor, which is an implicit fileld in by mill.define.Ctx.make which should pull in the sourcecode.File and sourcecode.Line for the module itself and not its parent or anything

[Shri333]

Actually, it looks like it is working for now. It was not working before for some reason (maybe some change fixed it?).

[Shri333]

added support for filename/linenum, scaladoc, inherited modules, module dependencies, default task, and all tasks if you want to take a look @lihaoyi

Example outputs:

/**
 * scaladoc for foo
 * foo is a module that extends MyModule
 */
object foo extends MyModule {
  def moduleDeps = Seq(bar)
  def ivyDeps = Agg(
    ivy"net.sourceforge.argparse4j:argparse4j:0.9.0",
  )
}

[1/1] inspect 
foo(build.mill:12)
    scaladoc for foo
    foo is a module that extends MyModule

Inherited Modules: build_.package_$MyModule, mill.scalalib.JavaModule, mill.scalalib.TestModule$JavaModuleBase, mill.scalalib.bsp.BspModule, mill.define.Module, mill.define.TaskModule, mill.scalalib.RunModule, mill.scalalib.WithZincWorker, mill.scalalib.GenIdeaModule, mill.scalalib.OfflineSupportModule, mill.scalalib.SemanticDbJavaModule, mill.scalalib.CoursierModule

Module Dependencies: bar

Default Task: run

Tasks: foo.test.compileResources, foo.upstreamAssembly, foo.test.jar, foo.assembly, foo.test.sources, foo.javadocOptions, foo.artifactId, foo.upstreamCompileOutput, foo.bspLocalRunClasspath, foo.test.unmanagedClasspath, foo.forkArgs, foo.test.launcher, foo.resources, foo.allSources, foo.compileIvyDeps, foo.test.discoveredTestClasses, foo.test.assembly, foo.docResources, foo.test.mainClass, foo.runIvyDeps, foo.docSources, foo.semanticDbPluginClasspath, foo.test.forkWorkingDir, foo.localRunClasspath, foo.test.compile, foo.test.forkEnv, foo.test.testFramework, foo.test.zincReportCachedProblems, foo.runClasspath, foo.test.testUseArgsFile, foo.test.semanticDbPluginClasspath, foo.platformSuffix, foo.transitiveCompileClasspath, foo.test.artifactName, foo.runUseArgsFile, foo.bspTransitiveCompileClasspath, foo.test.transitiveIvyDeps, foo.allIvyDeps, foo.transitiveCompileIvyDeps, foo.test.compiledClassesAndSemanticDbFiles, foo.test.localRunClasspath, foo.launcher, foo.semanticDbData, foo.test.runUseArgsFile, foo.jar, foo.test.sourceJar, foo.semanticDbScalaVersion, foo.javacOptions, foo.test.bspLocalClasspath, foo.semanticDbVersion, foo.test.localClasspath, foo.upstreamAssembly2, foo.test.semanticDbVersion, foo.compiledClassesAndSemanticDbFiles, foo.test.artifactSuffix, foo.test.testSandboxWorkingDir, foo.test.ideaCompileOutput, foo.zincReportCachedProblems, foo.artifactNameParts, foo.test.upstreamAssembly, foo.test.resolvedIvyDeps, foo.test.allSourceFiles, foo.generatedSources, foo.test.transitiveLocalClasspath, foo.compileResources, foo.test.bspTransitiveCompileClasspath, foo.forkWorkingDir, foo.transitiveLocalClasspath, foo.test.docJarUseArgsFile, foo.sourceJar, foo.test.allIvyDeps, foo.allLocalMainClasses, foo.test.javacOptions, foo.test.bspLocalRunClasspath, foo.test.semanticDbScalaVersion, foo.test.semanticDbEnablePluginScalacOptions, foo.localClasspath, foo.upstreamAssemblyClasspath, foo.forkEnv, foo.test.resolvedRunIvyDeps, foo.allSourceFiles, foo.test.runIvyDeps, foo.bspLocalClasspath, foo.test.testReportXml, foo.test.bspCompileClasspath, foo.test.resources, foo.resolvedRunIvyDeps, foo.test.zincIncrementalCompilation, foo.test.testClasspath, foo.mainClass, foo.test.semanticDbPluginIvyDeps, foo.prependShellScript, foo.resolvedIvyDeps, foo.transitiveIvyDeps, foo.test.transitiveCompileIvyDeps, foo.semanticDbEnablePluginScalacOptions, foo.test.mandatoryIvyDeps, foo.test.artifactId, foo.test.compileClasspath, foo.test.docSources, foo.test.mandatoryJavacOptions, foo.docJar, foo.test.generatedSources, foo.mandatoryIvyDeps, foo.test.prependShellScript, foo.test.upstreamAssemblyClasspath, foo.test.transitiveCompileClasspath, foo.bspTransitiveLocalClasspath, foo.test.upstreamAssembly2, foo.compileClasspath, foo.manifest, foo.test.javadocOptions, foo.mandatoryJavacOptions, foo.test.zincAuxiliaryClassFileExtensions, foo.docJarUseArgsFile, foo.zincAuxiliaryClassFileExtensions, foo.test.runClasspath, foo.test.finalMainClass, foo.test.testCachedArgs, foo.test.semanticDbJavaVersion, foo.test.upstreamCompileOutput, foo.test.bspCompileClassesPath, foo.localCompileClasspath, foo.test.compileIvyDeps, foo.compile, foo.resolvedSemanticDbJavaPluginIvyDeps, foo.unmanagedClasspath, foo.test.forkArgs, foo.finalMainClass, foo.test.semanticDbData, foo.test.allSources, foo.test.testCached, foo.artifactName, foo.ideaCompileOutput, foo.test.docResources, foo.test.docJar, foo.bspCompileClassesPath, foo.test.ivyDeps, foo.test.manifest, foo.test.allLocalMainClasses, foo.zincIncrementalCompilation, foo.semanticDbPluginIvyDeps, foo.bspCompiledClassesAndSemanticDbFiles, foo.artifactSuffix, foo.bspCompileClasspath, foo.test.bspTransitiveLocalClasspath, foo.finalMainClassOpt, foo.test.localCompileClasspath, foo.test.artifactNameParts, foo.ivyDeps, foo.sources, foo.semanticDbJavaVersion, foo.test.bspCompiledClassesAndSemanticDbFiles, foo.test.finalMainClassOpt, foo.test.resolvedSemanticDbJavaPluginIvyDeps, foo.test.platformSuffix

/**
 * test scaladoc
 */
object hello extends Module {
}

[1/1] inspect 
hello(build.mill:29)
    test scaladoc

Inherited Modules: mill.define.Module

[lihaoyi]

@Shri333 I think that looks great!

Some thoughts: a lot of the verbosity comes from transitively resolving stuff that isn't directly written in the code. Let's try

• For Tasks:, let's only display tasks which are directly implemented/overridden in the module if that's possible, on the assumption that those are probably the most interesting tasks to inspect, and anyone who wants a full listing can use ./mill resolve foo.test._

• For inherited modules traits, let's go with direct parent traits only, which would roughly match what is listed in the extends clause in the code. Otherwise the list gets pretty long with how fine-grained our trait hierarchy is, and it also no longer matches what is written in the code (which is only direct parents)

For testing, you should be able to re-use the docannotations test https://github.com/com-lihaoyi/mill/tree/main/integration/feature/docannotations, which already has a module with scaladoc and parent traits and defined classes and so on ready for you to use. Just need to add a few cases using inspect on the modules where previously the test only used inspect on the tasks

[Shri333]

@lihaoyi also, can I use scala reflect instead of java reflect? It was added in scala 2.10, but I am not sure about backwards compat issues.

[lihaoyi]

lets avoid scala reflect; using it for macros is fine, but at rintime it has a pretty big performance hit from running the scala compiler

[Shri333]

Hmm, java reflect works for everything except filtering out methods directly declared on an object. It seems to be an issue with how scala compiles objects into bytecode. Instead of getting the methods directly declared on an object (like foo), I end up getting the following when using t.cls.getDeclaredMethods:

Array(public mill.define.Target build_.package_$foo$.manifest(), public mill.define.Command build_.package_$foo$.run(mill.define.Task), public java.lang.String build_.package_$foo$.toString(), public build_.package_$MyModule$test$ build_.package_$foo$.test(), public mill.define.Target build_.package_$foo$.compile(), public mill.define.Target build_.package_$foo$.resources(), public mill.define.Target build_.package_$foo$.jar(), private static java.lang.Object build_.package_$foo$.$deserializeLambda$(java.lang.invoke.SerializedLambda), public mill.define.Target build_.package_$foo$.mainClass(), public build_.package_ build_.package_$foo$.build_$package_$MyModule$$$outer(), public scala.collection.immutable.Seq build_.package_$foo$.mill$scalalib$JavaModule$$recModuleDeps(), public mill.define.ModuleRef build_.package_$foo$.mill$scalalib$JavaModule$$super$zincWorker(), public mill.define.Target build_.package_$foo$.mill$scalalib$JavaModule$$super$forkArgs(), public mill.define.Target build_.package_$foo$.mill$scalalib$JavaModule$$super$forkEnv(), public mill.define.Command build_.package_$foo$.mill$scalalib$JavaModule$$super$runLocal(mill.define.Task), public mill.define.Command build_.package_$foo$.mill$scalalib$JavaModule$$super$run(mill.define.Task), public mill.define.Command build_.package_$foo$.mill$scalalib$JavaModule$$super$runMain(java.lang.String,scala.collection.immutable.Seq), public scala.collection.immutable.Seq build_.package_$foo$.transitiveModuleCompileModuleDeps(), public mill.define.Target build_.package_$foo$.semanticDbEnablePluginScalacOptions(), public mill.define.Target build_.package_$foo$.resolvedSemanticDbJavaPluginIvyDeps(), public mill.define.Task build_.package_$foo$.runBackgroundTask$default$2(), public mill.define.Target build_.package_$foo$.bspTransitiveCompileClasspath(), public mill.define.Target build_.package_$foo$.bspTransitiveLocalClasspath(), public mill.define.Target build_.package_$foo$.zincAuxiliaryClassFileExtensions(), public mill.define.Target build_.package_$foo$.bspCompiledClassesAndSemanticDbFiles(), public mill.define.Target build_.package_$foo$.compiledClassesAndSemanticDbFiles(), public mill.define.Module$millInternal$ build_.package_$foo$.millInternal(), public scala.collection.immutable.Seq build_.package_$foo$.millModuleDirectChildren(), public mill.define.Task build_.package_$foo$.mapDependencies(), public mill.define.Task build_.package_$foo$.resolveCoursierDependency(), public mill.define.Task build_.package_$foo$.resolvePublishDependency(), public mill.scalalib.bsp.BspBuildTarget build_.package_$foo$.bspBuildTarget(), public mill.define.Task build_.package_$foo$.bspBuildTargetData(), public mill.define.Target build_.package_$foo$.semanticDbPluginClasspath(), public java.lang.String build_.package_$foo$.defaultCommandName(), public final scala.collection.immutable.Seq build_.package_$foo$.moduleDepsChecked(), public scala.collection.immutable.Seq build_.package_$foo$.compileModuleDeps(), public final scala.collection.immutable.Seq build_.package_$foo$.compileModuleDepsChecked(), public scala.collection.immutable.Seq build_.package_$foo$.recursiveModuleDeps(), public scala.collection.immutable.Seq build_.package_$foo$.transitiveModuleDeps(), public scala.collection.immutable.Seq build_.package_$foo$.assemblyRules(), public mill.define.Task build_.package_$foo$.printDepsTree(boolean,mill.define.Task,scala.collection.immutable.List), public scala.Function1 build_.package_$foo$.doRunBackground(os.Path,scala.collection.immutable.Seq,mill.api.AggWrapper$Agg,scala.collection.immutable.Seq,scala.collection.immutable.Map,java.lang.String,os.Path,boolean,scala.Option,scala.collection.immutable.Seq), public boolean build_.package_$foo$.runBackgroundLogToConsole(), public mill.define.Target build_.package_$foo$.semanticDbPluginIvyDeps(), public mill.define.Task build_.package_$foo$.defaultResolver(), public mill.define.Task build_.package_$foo$.resolveDeps(mill.define.Task,boolean), public boolean build_.package_$foo$.resolveDeps$default$2(), public mill.define.Task build_.package_$foo$.resolutionCustomizer(), public mill.define.Task build_.package_$foo$.coursierCacheCustomizer(), public mill.define.Task build_.package_$foo$.ideaJavaModuleFacets(int), public mill.define.Task build_.package_$foo$.ideaConfigFiles(int), public mill.define.Task build_.package_$foo$.runForkedTask(mill.define.Task,mill.define.Task), public mill.define.Task build_.package_$foo$.runForkedTask$default$2(), public mill.define.Task build_.package_$foo$.runLocalTask(mill.define.Task,mill.define.Task), public mill.define.Task build_.package_$foo$.runLocalTask$default$2(), public mill.define.Task build_.package_$foo$.runBackgroundTask(mill.define.Task,mill.define.Task), public java.lang.String build_.package_$foo$.bspDisplayName(), public java.lang.String build_.package_$foo$.bspDisplayName0(), public os.Path build_.package_$foo$.millSourcePath(), public os.Path build_.package_$foo$.intellijModulePath(), public mill.define.Task build_.package_$foo$.repositoriesTask(), public static final mill.define.TargetImpl build_.package_$foo$.$anonfun$ivyDeps$1(build_.package_$foo$), public mill.define.Task build_.package_$foo$.bindDependency(), public mill.define.Ctx$BasePath build_.package_$foo$.millModuleBasePath(), public mill.define.Segments build_.package_$foo$.millModuleSegments(), public mill.define.Ctx$External build_.package_$foo$.millModuleExternal(), public mill.define.Ctx$Foreign build_.package_$foo$.millModuleShared(), public static final mill.api.AggWrapper$Agg build_.package_$foo$.$anonfun$ivyDeps$3(), public static final mill.api.Result build_.package_$foo$.$anonfun$ivyDeps$2(scala.collection.immutable.Seq,mill.api.Ctx), public mill.define.Target build_.package_$foo$.runClasspath(), public mill.scalalib.IvyDepsTreeArgs build_.package_$foo$.ivyDepsTree$default$1(), public mill.define.Command build_.package_$foo$.ivyDepsTree(mill.scalalib.IvyDepsTreeArgs), public mill.define.Task build_.package_$foo$.run$default$1(), public mill.define.Command build_.package_$foo$.runBackground(scala.collection.immutable.Seq), public mill.define.Task build_.package_$foo$.runLocal$default$1(), public mill.define.Command build_.package_$foo$.runMainBackground(java.lang.String,scala.collection.immutable.Seq), public mill.define.Command build_.package_$foo$.runMainLocal(java.lang.String,scala.collection.immutable.Seq), public boolean build_.package_$foo$.showModuleDeps$default$1(), public mill.define.Command build_.package_$foo$.showModuleDeps(boolean), public mill.define.Command build_.package_$foo$.prepareOffline(mainargs.Flag), public mill.define.Target build_.package_$foo$.allSourceFiles(), public mill.define.Target build_.package_$foo$.generatedSources(), public mill.define.Target build_.package_$foo$.platformSuffix(), public mill.define.Target build_.package_$foo$.unmanagedClasspath(), public mill.define.Target build_.package_$foo$.ideaCompileOutput(), public mill.define.Target build_.package_$foo$.artifactNameParts(), public mill.define.Target build_.package_$foo$.bspCompileClasspath(), public mill.define.Target build_.package_$foo$.bspLocalClasspath(), public mill.define.Target build_.package_$foo$.bspLocalRunClasspath(), public mill.define.Target build_.package_$foo$.compileClasspath(), public mill.define.Target build_.package_$foo$.compileIvyDeps(), public mill.define.Target build_.package_$foo$.compileResources(), public mill.define.Target build_.package_$foo$.docJarUseArgsFile(), public mill.define.Target build_.package_$foo$.docResources(), public mill.define.Target build_.package_$foo$.finalMainClass(), public mill.define.Target build_.package_$foo$.finalMainClassOpt(), public mill.define.Target build_.package_$foo$.forkWorkingDir(), public mill.define.Target build_.package_$foo$.javacOptions(), public mill.define.Target build_.package_$foo$.javadocOptions(), public mill.define.Target build_.package_$foo$.localClasspath(), public mill.define.Target build_.package_$foo$.localCompileClasspath(), public mill.define.Target build_.package_$foo$.localRunClasspath(), public mill.define.Target build_.package_$foo$.mandatoryJavacOptions(), public mill.define.Target build_.package_$foo$.prependShellScript(), public mill.define.Target build_.package_$foo$.resolvedIvyDeps(), public mill.define.Target build_.package_$foo$.resolvedRunIvyDeps(), public mill.define.Target build_.package_$foo$.runUseArgsFile(), public mill.define.Target build_.package_$foo$.transitiveCompileClasspath(), public mill.define.Target build_.package_$foo$.transitiveCompileIvyDeps(), public mill.define.Target build_.package_$foo$.transitiveIvyDeps(), public mill.define.Target build_.package_$foo$.transitiveLocalClasspath(), public mill.define.Target build_.package_$foo$.upstreamAssembly(), public mill.define.Target build_.package_$foo$.upstreamAssembly2(), public mill.define.Target build_.package_$foo$.upstreamAssemblyClasspath(), public mill.define.Target build_.package_$foo$.upstreamCompileOutput(), public mill.define.Target build_.package_$foo$.zincIncrementalCompilation(), public mill.define.Target build_.package_$foo$.zincReportCachedProblems(), public mill.define.Target build_.package_$foo$.allLocalMainClasses(), public mill.define.Target build_.package_$foo$.artifactSuffix(), public mill.define.Target build_.package_$foo$.bspCompileClassesPath(), public mill.define.Target build_.package_$foo$.mandatoryIvyDeps(), public mill.define.Target build_.package_$foo$.semanticDbData(), public mill.define.Target build_.package_$foo$.semanticDbScalaVersion(), public mill.define.Target build_.package_$foo$.semanticDbJavaVersion(), public mill.define.Target build_.package_$foo$.semanticDbVersion(), public mill.define.Target build_.package_$foo$.artifactName(), public boolean build_.package_$foo$.skipIdea(), public mill.define.ModuleRef build_.package_$foo$.zincWorker(), public scala.collection.immutable.Seq build_.package_$foo$.moduleDeps(), public mill.define.Command build_.package_$foo$.runLocal(mill.define.Task), public mill.define.Command build_.package_$foo$.runMain(java.lang.String,scala.collection.immutable.Seq), public mill.define.Target build_.package_$foo$.ivyDeps(), public mill.define.Target build_.package_$foo$.runIvyDeps(), public mill.define.Target build_.package_$foo$.sources(), public mill.define.Target build_.package_$foo$.allIvyDeps(), public mill.define.Target build_.package_$foo$.allSources(), public mill.define.Target build_.package_$foo$.assembly(), public mill.define.Target build_.package_$foo$.forkArgs(), public mill.define.Target build_.package_$foo$.forkEnv(), public mill.define.Target build_.package_$foo$.launcher(), public mill.define.Target build_.package_$foo$.sourceJar(), public mill.define.Target build_.package_$foo$.artifactId(), public mill.define.Target build_.package_$foo$.docJar(), public mill.define.Target build_.package_$foo$.docSources(), public mill.define.Task build_.package_$foo$.runner(), private final void build_.package_$foo$.test$lzycompute$1(), private final void build_.package_$foo$.millInternal$lzycompute$2(), public scala.collection.immutable.Seq build_.package_$foo$.mill$define$Module$$millModuleDirectChildrenImpl(), public scala.collection.immutable.Seq build_.package_$foo$.mill$scalalib$JavaModule$$recCompileModuleDeps(), public mill.define.Target build_.package_$foo$.mill$scalalib$JavaModule$$super$localRunClasspath(), public mill.define.Target build_.package_$foo$.mill$scalalib$JavaModule$$super$runClasspath(), public mill.define.Target build_.package_$foo$.mill$scalalib$JavaModule$$super$runUseArgsFile(), public scala.Function1 build_.package_$foo$.mill$scalalib$JavaModule$$super$doRunBackground(os.Path,scala.collection.immutable.Seq,mill.api.AggWrapper$Agg,scala.collection.immutable.Seq,scala.collection.immutable.Map,java.lang.String,os.Path,boolean,scala.Option,scala.collection.immutable.Seq), public boolean build_.package_$foo$.mill$scalalib$JavaModule$$super$runBackgroundLogToConsole(), public mill.define.Command build_.package_$foo$.mill$scalalib$JavaModule$$super$runMainBackground(java.lang.String,scala.collection.immutable.Seq), public mill.define.Command build_.package_$foo$.mill$scalalib$JavaModule$$super$runMainLocal(java.lang.String,scala.collection.immutable.Seq), public mill.define.Target build_.package_$foo$.mill$scalalib$JavaModule$$super$forkWorkingDir(), public mill.define.Command build_.package_$foo$.mill$scalalib$JavaModule$$super$prepareOffline(mainargs.Flag), public mill.scalalib.bsp.BspBuildTarget build_.package_$foo$.mill$scalalib$JavaModule$$super$bspBuildTarget(), private scala.collection.immutable.Seq build_.package_$foo$.mill$scalalib$JavaModule$$recModuleDeps$lzycompute(), private scala.collection.immutable.Seq build_.package_$foo$.mill$scalalib$JavaModule$$recCompileModuleDeps$lzycompute(), private scala.collection.immutable.Seq build_.package_$foo$.mill$define$Module$$millModuleDirectChildrenImpl$lzycompute())

[lihaoyi]

@Shri333 what are you trying to do? If you're not sure how to use java-reflect for Scala/Mill, you can see Reflect.scala which has a bunch of hacky filters and configs that Mill uses for reflecting on stuff. Assuming you're not doing anything fancy, the same techniques should work for you

[Shri333]

I was trying to get all tasks that are explicitly declared on a module. For example, for the module below:

/**
 * scaladoc for foo
 * foo is a module that extends MyModule
 */
object foo extends MyModule {
  def moduleDeps = Seq(bar)
  def ivyDeps = Agg(
    ivy"net.sourceforge.argparse4j:argparse4j:0.9.0",
  )
}

I used

Reflect.reflect(t.cls, classOf[Target[_]], Function.const(true), noParams = true)

But, I got a bunch of extra methods:

ArraySeq(public mill.define.Target build_.package_$foo$.allIvyDeps(), public mill.define.Target build_.package_$foo$.allLocalMainClasses(), public mill.define.Target build_.package_$foo$.allSourceFiles(), public mill.define.Target build_.package_$foo$.allSources(), public mill.define.Target build_.package_$foo$.artifactId(), public mill.define.Target build_.package_$foo$.artifactName(), public mill.define.Target build_.package_$foo$.artifactNameParts(), public mill.define.Target build_.package_$foo$.artifactSuffix(), public mill.define.Target build_.package_$foo$.assembly(), public mill.define.Target build_.package_$foo$.bspCompileClassesPath(), public mill.define.Target build_.package_$foo$.bspCompileClasspath(), public mill.define.Target build_.package_$foo$.bspCompiledClassesAndSemanticDbFiles(), public mill.define.Target build_.package_$foo$.bspLocalClasspath(), public mill.define.Target build_.package_$foo$.bspLocalRunClasspath(), public mill.define.Target build_.package_$foo$.bspTransitiveCompileClasspath(), public mill.define.Target build_.package_$foo$.bspTransitiveLocalClasspath(), public mill.define.Target build_.package_$foo$.compile(), public mill.define.Target build_.package_$foo$.compileClasspath(), public mill.define.Target build_.package_$foo$.compileIvyDeps(), public mill.define.Target build_.package_$foo$.compileResources(), public mill.define.Target build_.package_$foo$.compiledClassesAndSemanticDbFiles(), public mill.define.Target build_.package_$foo$.docJar(), public mill.define.Target build_.package_$foo$.docJarUseArgsFile(), public mill.define.Target build_.package_$foo$.docResources(), public mill.define.Target build_.package_$foo$.docSources(), public mill.define.Target build_.package_$foo$.finalMainClass(), public mill.define.Target build_.package_$foo$.finalMainClassOpt(), public mill.define.Target build_.package_$foo$.forkArgs(), public mill.define.Target build_.package_$foo$.forkEnv(), public mill.define.Target build_.package_$foo$.forkWorkingDir(), public mill.define.Target build_.package_$foo$.generatedSources(), public mill.define.Target build_.package_$foo$.ideaCompileOutput(), public mill.define.Target build_.package_$foo$.ivyDeps(), public mill.define.Target build_.package_$foo$.jar(), public mill.define.Target build_.package_$foo$.javacOptions(), public mill.define.Target build_.package_$foo$.javadocOptions(), public mill.define.Target build_.package_$foo$.launcher(), public mill.define.Target build_.package_$foo$.localClasspath(), public mill.define.Target build_.package_$foo$.localCompileClasspath(), public mill.define.Target build_.package_$foo$.localRunClasspath(), public mill.define.Target build_.package_$foo$.mainClass(), public mill.define.Target build_.package_$foo$.mandatoryIvyDeps(), public mill.define.Target build_.package_$foo$.mandatoryJavacOptions(), public mill.define.Target build_.package_$foo$.manifest(), public mill.define.Target build_.package_$foo$.platformSuffix(), public mill.define.Target build_.package_$foo$.prependShellScript(), public mill.define.Target build_.package_$foo$.resolvedIvyDeps(), public mill.define.Target build_.package_$foo$.resolvedRunIvyDeps(), public mill.define.Target build_.package_$foo$.resolvedSemanticDbJavaPluginIvyDeps(), public mill.define.Target build_.package_$foo$.resources(), public mill.define.Target build_.package_$foo$.runClasspath(), public mill.define.Target build_.package_$foo$.runIvyDeps(), public mill.define.Target build_.package_$foo$.runUseArgsFile(), public mill.define.Target build_.package_$foo$.semanticDbData(), public mill.define.Target build_.package_$foo$.semanticDbEnablePluginScalacOptions(), public mill.define.Target build_.package_$foo$.semanticDbJavaVersion(), public mill.define.Target build_.package_$foo$.semanticDbPluginClasspath(), public mill.define.Target build_.package_$foo$.semanticDbPluginIvyDeps(), public mill.define.Target build_.package_$foo$.semanticDbScalaVersion(), public mill.define.Target build_.package_$foo$.semanticDbVersion(), public mill.define.Target build_.package_$foo$.sourceJar(), public mill.define.Target build_.package_$foo$.sources(), public mill.define.Target build_.package_$foo$.transitiveCompileClasspath(), public mill.define.Target build_.package_$foo$.transitiveCompileIvyDeps(), public mill.define.Target build_.package_$foo$.transitiveIvyDeps(), public mill.define.Target build_.package_$foo$.transitiveLocalClasspath(), public mill.define.Target build_.package_$foo$.unmanagedClasspath(), public mill.define.Target build_.package_$foo$.upstreamAssembly(), public mill.define.Target build_.package_$foo$.upstreamAssembly2(), public mill.define.Target build_.package_$foo$.upstreamAssemblyClasspath(), public mill.define.Target build_.package_$foo$.upstreamCompileOutput(), public mill.define.Target build_.package_$foo$.zincAuxiliaryClassFileExtensions(), public mill.define.Target build_.package_$foo$.zincIncrementalCompilation(), public mill.define.Target build_.package_$foo$.zincReportCachedProblems())

[Shri333]

Also, I added tests and fixed some of the resolve tests, but other resolve tests don't seem to be working correctly. I fixed some of them, but the ones that use selectors do not seem to filter properly

[lihaoyi]

I was trying to get all tasks that are explicitly declared on a module. For example, for the module below:

/**
 * scaladoc for foo
 * foo is a module that extends MyModule
 */
object foo extends MyModule {
  def moduleDeps = Seq(bar)
  def ivyDeps = Agg(
    ivy"net.sourceforge.argparse4j:argparse4j:0.9.0",
  )
}

I used

Reflect.reflect(t.cls, classOf[Target[_]], Function.const(true), noParams = true)

But, I got a bunch of extra methods:

ArraySeq(public mill.define.Target build_.package_$foo$.allIvyDeps(), public mill.define.Target build_.package_$foo$.allLocalMainClasses(), public mill.define.Target build_.package_$foo$.allSourceFiles(), public mill.define.Target build_.package_$foo$.allSources(), public mill.define.Target build_.package_$foo$.artifactId(), public mill.define.Target build_.package_$foo$.artifactName(), public mill.define.Target build_.package_$foo$.artifactNameParts(), public mill.define.Target build_.package_$foo$.artifactSuffix(), public mill.define.Target build_.package_$foo$.assembly(), public mill.define.Target build_.package_$foo$.bspCompileClassesPath(), public mill.define.Target build_.package_$foo$.bspCompileClasspath(), public mill.define.Target build_.package_$foo$.bspCompiledClassesAndSemanticDbFiles(), public mill.define.Target build_.package_$foo$.bspLocalClasspath(), public mill.define.Target build_.package_$foo$.bspLocalRunClasspath(), public mill.define.Target build_.package_$foo$.bspTransitiveCompileClasspath(), public mill.define.Target build_.package_$foo$.bspTransitiveLocalClasspath(), public mill.define.Target build_.package_$foo$.compile(), public mill.define.Target build_.package_$foo$.compileClasspath(), public mill.define.Target build_.package_$foo$.compileIvyDeps(), public mill.define.Target build_.package_$foo$.compileResources(), public mill.define.Target build_.package_$foo$.compiledClassesAndSemanticDbFiles(), public mill.define.Target build_.package_$foo$.docJar(), public mill.define.Target build_.package_$foo$.docJarUseArgsFile(), public mill.define.Target build_.package_$foo$.docResources(), public mill.define.Target build_.package_$foo$.docSources(), public mill.define.Target build_.package_$foo$.finalMainClass(), public mill.define.Target build_.package_$foo$.finalMainClassOpt(), public mill.define.Target build_.package_$foo$.forkArgs(), public mill.define.Target build_.package_$foo$.forkEnv(), public mill.define.Target build_.package_$foo$.forkWorkingDir(), public mill.define.Target build_.package_$foo$.generatedSources(), public mill.define.Target build_.package_$foo$.ideaCompileOutput(), public mill.define.Target build_.package_$foo$.ivyDeps(), public mill.define.Target build_.package_$foo$.jar(), public mill.define.Target build_.package_$foo$.javacOptions(), public mill.define.Target build_.package_$foo$.javadocOptions(), public mill.define.Target build_.package_$foo$.launcher(), public mill.define.Target build_.package_$foo$.localClasspath(), public mill.define.Target build_.package_$foo$.localCompileClasspath(), public mill.define.Target build_.package_$foo$.localRunClasspath(), public mill.define.Target build_.package_$foo$.mainClass(), public mill.define.Target build_.package_$foo$.mandatoryIvyDeps(), public mill.define.Target build_.package_$foo$.mandatoryJavacOptions(), public mill.define.Target build_.package_$foo$.manifest(), public mill.define.Target build_.package_$foo$.platformSuffix(), public mill.define.Target build_.package_$foo$.prependShellScript(), public mill.define.Target build_.package_$foo$.resolvedIvyDeps(), public mill.define.Target build_.package_$foo$.resolvedRunIvyDeps(), public mill.define.Target build_.package_$foo$.resolvedSemanticDbJavaPluginIvyDeps(), public mill.define.Target build_.package_$foo$.resources(), public mill.define.Target build_.package_$foo$.runClasspath(), public mill.define.Target build_.package_$foo$.runIvyDeps(), public mill.define.Target build_.package_$foo$.runUseArgsFile(), public mill.define.Target build_.package_$foo$.semanticDbData(), public mill.define.Target build_.package_$foo$.semanticDbEnablePluginScalacOptions(), public mill.define.Target build_.package_$foo$.semanticDbJavaVersion(), public mill.define.Target build_.package_$foo$.semanticDbPluginClasspath(), public mill.define.Target build_.package_$foo$.semanticDbPluginIvyDeps(), public mill.define.Target build_.package_$foo$.semanticDbScalaVersion(), public mill.define.Target build_.package_$foo$.semanticDbVersion(), public mill.define.Target build_.package_$foo$.sourceJar(), public mill.define.Target build_.package_$foo$.sources(), public mill.define.Target build_.package_$foo$.transitiveCompileClasspath(), public mill.define.Target build_.package_$foo$.transitiveCompileIvyDeps(), public mill.define.Target build_.package_$foo$.transitiveIvyDeps(), public mill.define.Target build_.package_$foo$.transitiveLocalClasspath(), public mill.define.Target build_.package_$foo$.unmanagedClasspath(), public mill.define.Target build_.package_$foo$.upstreamAssembly(), public mill.define.Target build_.package_$foo$.upstreamAssembly2(), public mill.define.Target build_.package_$foo$.upstreamAssemblyClasspath(), public mill.define.Target build_.package_$foo$.upstreamCompileOutput(), public mill.define.Target build_.package_$foo$.zincAuxiliaryClassFileExtensions(), public mill.define.Target build_.package_$foo$.zincIncrementalCompilation(), public mill.define.Target build_.package_$foo$.zincReportCachedProblems())

@Shri333 That is because Reflect.reflect uses Class#getMethods. If you copy it but make it use `Class#getDeclaredMethods I think it'll limit it to the ones directly declared in that class/object/trait

As for the resolve tests, you'll have to look into your code changes and see why they broke.

[Shri333]

It's the same with getDeclaredMethods unfortunately:

      val tasks = Reflect.reflect(
        cls,
        classOf[Target[_]],
        Function.const(true),
        noParams = true,
        declaredMethods = true
      ).map(_.getName).distinct

  def reflect(
      outer: Class[_],
      inner: Class[_],
      filter: String => Boolean,
      noParams: Boolean,
      declaredMethods: Boolean = false
  ): Seq[java.lang.reflect.Method] = {
    val res = for {
      m <- if (declaredMethods) outer.getDeclaredMethods else outer.getMethods
      n = decode(m.getName)
      if filter(n) &&
        isLegalIdentifier(n) &&
        (!noParams || m.getParameterCount == 0) &&
        (m.getModifiers & Modifier.STATIC) == 0 &&
        inner.isAssignableFrom(m.getReturnType)
    } yield m
    ...
    ...
    ...
  }

Tasks: allIvyDeps, allLocalMainClasses, allSourceFiles, allSources, artifactId, artifactName, artifactNameParts, artifactSuffix, assembly, bspCompileClassesPath, bspCompileClasspath, bspCompiledClassesAndSemanticDbFiles, bspLocalClasspath, bspLocalRunClasspath, bspTransitiveCompileClasspath, bspTransitiveLocalClasspath, compile, compileClasspath, compileIvyDeps, compileResources, compiledClassesAndSemanticDbFiles, docJar, docJarUseArgsFile, docResources, docSources, finalMainClass, finalMainClassOpt, forkArgs, forkEnv, forkWorkingDir, generatedSources, ideaCompileOutput, ivyDeps, jar, javacOptions, javadocOptions, launcher, localClasspath, localCompileClasspath, localRunClasspath, mainClass, mandatoryIvyDeps, mandatoryJavacOptions, manifest, platformSuffix, prependShellScript, resolvedIvyDeps, resolvedRunIvyDeps, resolvedSemanticDbJavaPluginIvyDeps, resources, runClasspath, runIvyDeps, runUseArgsFile, semanticDbData, semanticDbEnablePluginScalacOptions, semanticDbJavaVersion, semanticDbPluginClasspath, semanticDbPluginIvyDeps, semanticDbScalaVersion, semanticDbVersion, sourceJar, sources, transitiveCompileClasspath, transitiveCompileIvyDeps, transitiveIvyDeps, transitiveLocalClasspath, unmanagedClasspath, upstreamAssembly, upstreamAssembly2, upstreamAssemblyClasspath, upstreamCompileOutput, zincAuxiliaryClassFileExtensions, zincIncrementalCompilation, zincReportCachedProblems

[lihaoyi]

Maybe some of them are synthetic trait forwarders? do they have any naming convention or modifiers thst distinguish them?

[Shri333]

Doesn't seem so:

Tasks: public mill.define.Target build.package$foo$.allIvyDeps(), public mill.define.Target build.package$foo$.allLocalMainClasses(), public mill.define.Target build.package$foo$.allSourceFiles(), public mill.define.Target build.package$foo$.allSources(), public mill.define.Target build.package$foo$.artifactId(), public mill.define.Target build.package$foo$.artifactName(), public mill.define.Target build.package$foo$.artifactNameParts(), public mill.define.Target build.package$foo$.artifactSuffix(), public mill.define.Target build.package$foo$.assembly(), public mill.define.Target build.package$foo$.bspCompileClassesPath(), public mill.define.Target build.package$foo$.bspCompileClasspath(), public mill.define.Target build.package$foo$.bspCompiledClassesAndSemanticDbFiles(), public mill.define.Target build.package$foo$.bspLocalClasspath(), public mill.define.Target build.package$foo$.bspLocalRunClasspath(), public mill.define.Target build.package$foo$.bspTransitiveCompileClasspath(), public mill.define.Target build.package$foo$.bspTransitiveLocalClasspath(), public mill.define.Target build.package$foo$.compile(), public mill.define.Target build.package$foo$.compileClasspath(), public mill.define.Target build.package$foo$.compileIvyDeps(), public mill.define.Target build.package$foo$.compileResources(), public mill.define.Target build.package$foo$.compiledClassesAndSemanticDbFiles(), public mill.define.Target build.package$foo$.docJar(), public mill.define.Target build.package$foo$.docJarUseArgsFile(), public mill.define.Target build.package$foo$.docResources(), public mill.define.Target build.package$foo$.docSources(), public mill.define.Target build.package$foo$.finalMainClass(), public mill.define.Target build.package$foo$.finalMainClassOpt(), public mill.define.Target build.package$foo$.forkArgs(), public mill.define.Target build.package$foo$.forkEnv(), public mill.define.Target build.package$foo$.forkWorkingDir(), public mill.define.Target build.package$foo$.generatedSources(), public mill.define.Target build.package$foo$.ideaCompileOutput(), public mill.define.Target build.package$foo$.ivyDeps(), public mill.define.Target build.package$foo$.jar(), public mill.define.Target build.package$foo$.javacOptions(), public mill.define.Target build.package$foo$.javadocOptions(), public mill.define.Target build.package$foo$.launcher(), public mill.define.Target build.package$foo$.localClasspath(), public mill.define.Target build.package$foo$.localCompileClasspath(), public mill.define.Target build.package$foo$.localRunClasspath(), public mill.define.Target build.package$foo$.mainClass(), public mill.define.Target build.package$foo$.mandatoryIvyDeps(), public mill.define.Target build.package$foo$.mandatoryJavacOptions(), public mill.define.Target build.package$foo$.manifest(), public mill.define.Target build.package$foo$.platformSuffix(), public mill.define.Target build.package$foo$.prependShellScript(), public mill.define.Target build.package$foo$.resolvedIvyDeps(), public mill.define.Target build.package$foo$.resolvedRunIvyDeps(), public mill.define.Target build.package$foo$.resolvedSemanticDbJavaPluginIvyDeps(), public mill.define.Target build.package$foo$.resources(), public mill.define.Target build.package$foo$.runClasspath(), public mill.define.Target build.package$foo$.runIvyDeps(), public mill.define.Target build.package$foo$.runUseArgsFile(), public mill.define.Target build.package$foo$.semanticDbData(), public mill.define.Target build.package$foo$.semanticDbEnablePluginScalacOptions(), public mill.define.Target build.package$foo$.semanticDbJavaVersion(), public mill.define.Target build.package$foo$.semanticDbPluginClasspath(), public mill.define.Target build.package$foo$.semanticDbPluginIvyDeps(), public mill.define.Target build.package$foo$.semanticDbScalaVersion(), public mill.define.Target build.package$foo$.semanticDbVersion(), public mill.define.Target build.package$foo$.sourceJar(), public mill.define.Target build.package$foo$.sources(), public mill.define.Target build.package$foo$.transitiveCompileClasspath(), public mill.define.Target build.package$foo$.transitiveCompileIvyDeps(), public mill.define.Target build.package$foo$.transitiveIvyDeps(), public mill.define.Target build.package$foo$.transitiveLocalClasspath(), public mill.define.Target build.package$foo$.unmanagedClasspath(), public mill.define.Target build.package$foo$.upstreamAssembly(), public mill.define.Target build.package$foo$.upstreamAssembly2(), public mill.define.Target build.package$foo$.upstreamAssemblyClasspath(), public mill.define.Target build.package$foo$.upstreamCompileOutput(), public mill.define.Target build.package$foo$.zincAuxiliaryClassFileExtensions(), public mill.define.Target build.package$foo$.zincIncrementalCompilation(), public mill.define.Target build.package$foo$.zincReportCachedProblems()

I think it's a problem with the nature of scala objects itself and the corresponding bytecode that is produced. We may be able to do this with scala reflection:

trait MyTrait1 {
  def bar(): Unit = println("bar")
}
trait MyClass {
  def baz(): Unit = println("baz")
}

object MyObject extends MyClass with MyTrait1 {
  def foo(): Unit = println("foo")
}

typeOf[MyObject.type].members.filter(_.isMethod).toList
// List(method foo, constructor MyObject, method bar, method $init$, method baz, method synchronized, method ##, method !=, method ==, method ne, method eq, method notifyAll, method notify, method clone, method getClass, method hashCode, method toString, method equals, method wait, method wait, method wait, method finalize, method asInstanceOf, method isInstanceOf)
typeOf[MyObject.type].decls.filter(_.isMethod).toList
// List(constructor MyObject, method foo)

[llvee]

@Shri333 Is this complete? If not does it bother you if I decide to attempt this 24 hours from now?

[Shri333]

@llvee this is not complete, but close to completion; there's prob one main thing to fix (displaying tasks) and task resolution

[llvee]

@Shri333 Would you like some help with completing the rest?

[Shri333]

@llvee I have run into a few blockers, but I would like to work on them for a little bit. If I am not able to resolve them, I will mention you on this thread.

[llvee]

@Shri333 Ok, I will continue checking notifications daily. I am up to help in the case that you would like some assistance.