ITypeConverter for custom generics (type constructors)

[Krever]

Hey! I'm trying to use picocli with Scala and one of the main limitations is inability to define converters from scala stdlib. While simple types are ok, the problem arises with generic types (called type constructors in scala).

Scala has its own scala.List, scala.Map and scala.Option. Having to use java variants is a bit suboptimal.

This might be related to https://github.com/remkop/picocli/issues/1804 but Im not sure if interface described there would suffice, so I'm raising a separate ticket.

[remkop]

Is the problem that you want to use Scala collection types for multi-value options and positional parameters? So, in your program you want to annotate fields of type scala.List, scala.Map and scala.Option with picocli @Option and @Parameters?

Picocli internally uses reflection to see if multi-value fields implement the java.util.Collection interface or the java.util.Map interface, and works with these Java interfaces in the parser. Similar with java.util.Optional. This is fairly deeply embedded at the moment and would require some refactoring (and perhaps some additional API) to decouple.

However, I have one idea that should allow you to use picocli-annotated scala.List and scala.Map fields in your Scala program with the current version of picocli: use a @Option-annotated method, whose parameter type is java.util.Collection or java.util.Map, and in the implementation of that method, delegate to fields that use the Scala types. Then, in the business logic of your application, you can just use the Scala-typed fields.

For example (PSEUDO CODE):

@Command
class App {

    scala.List myList;

    scala.Map myMap;

    @Option(names = Array("-f", "--file"), description = Array("The files to process. -fFile1 -fFile2"))
    def setFiles(files: java.util.List) {
        myList.clear()
        myList.addAll(files)
    }

    @Option(names = Array("-D", "--property"), description = Array("The properties. -Dkey1=val1 -Dkey2=val2"))
    def setProperties(properties: java.util.Map) {
        myMap.clear()
        myMap.putAll(properties)
    }

Can you give this a try?

[Krever]

Thanks a lot for the response and for the possible workaround!

Although the described approach would work in theory it won't fly for us in practice. The reason is that we chose picocli primarily because of (sub)command methods and we intend to use those for ~90% of commands (we try to rewrite quite a big cli app from python to scala).

For the sake of anyone who might see this thread in the future, I'm attaching our solution, which is quite good (allows to use any scala type and ensures converter is present at compile time) but is also cumbersome (requires wrapping all parameters) and has significant drawbacks (parsing errors are thrown during command execution, not parsing).

If picocli had some lower level API that would allow us to plug in more directly into the parser, it would be great.

// typeclass responsible for decoding particular type
trait TypeDecoder[T] {
  def decode(str: String): Either[String, T]
}

object TypeDecoder {

  // example instance
  implicit def MyTypeDecoder: TypeDecoder[MyType] = ???

 // generic support for option, same could work for list
  implicit def optionDecoder[T](implicit decoder: TypeDecoder[T]): TypeDecoder[Option[T]] = s => Option(s).traverse(decoder.decode)

}

// wrapper type, captures raw value as string and executes parsing during execution
class P[T](value: String) {
  def get(implicit decoder: TypeDecoder[T], spec: CommandSpec): T = {
    decoder.decode(value).fold(s => throw new ParameterException(spec.commandLine(), s), identity)
  }
}

object P {

  object TypeConverter extends ITypeConverter[P[_]] {
    override def convert(value: String): P[_] = if (value == EmptyMarker) new P(null) else new P(value)
  }

  val EmptyMarker = "??EMPTY"
}

object Main extends StrictLogging {
  def main(args: Array[String]): Unit = {
    new CommandLine(new MyCmd())
      .registerConverter(classOf[utils.P[_]], utils.P.TypeConverter)
      .setDefaultValueProvider((argSpec: Model.ArgSpec) => {
        // required so that P is used even if option/parameter is not specified. Without default value we get `x: P[T] = null`
        if (argSpec.`type`() == classOf[P[_]]) P.EmptyMarker
        else null
      })
  }
}

@Command(name = "myapp")
class MyCmd() {

  @Command(name = "foo")
  def foo(bar: P[Option[MyType]]): Unit = {
    println(bar.get)
  }
}

[remkop]

@Krever Glad you found an efficient workaround.

Are you okay if I close this ticket? To be honest, I don't see myself working on API to support non-java Collection and Map-like data structures.

[Krever]

Hey @remkop, I understand, it's fine to close the issue. I think it might significantly limit the adoption from Scala (having working native collections is rather important). However, at the same time, I see value in focus (on Java/Kotlin) and understand the lack of resources to add this significant change.

Thanks a lot for the responses :)

[bherw]

Thanks to @remkop for the workaround; I was able to use a similar approach to populate a Guava Multimap:

Multimap<UUID, String> examples = MultimapBuilder.linkedHashKeys().linkedHashSetValues().build();

@Option(
        names = "--examples",
        paramLabel = "<uuid>=<string>"
)
public void setExamples(Map<UUID, String> newValues) {
    for (Map.Entry<UUID, String> entry : newValues.entrySet()) {
        examples.put(entry.getKey(), entry.getValue());
    }
}

Unlike the suggestion given above, this code doesn't ever clear the multimap, but this seems to work fine if PicoCLI doesn't need to remove anything--parsing arguments should just be additive.

[Krever]

Hey @remkop, I got back to this problem and thought of an alternative workaround.

What do you think about traversing CommandSpec and injecting custom IParameterConsumer whenever a scala type is present (e.g. collection or Option)?

The problem I currently have with this approach:

• I have CommandLine in my main, from which I can get CommandSpec.
• The CommandSpec seems mutable, so I can substitute ArgSpec with one with a custom consumer
• ArgSpec is not mutable (at least not its parameterConsumer field is not), so I should probably go through ArgSpec.Builder
• I don't see a way to go from ArgSpec to ArgSpec.Builder (to keep all other information that is already there)

I would appreciate any hint or info that this approach won't fly for some reason.

I was also considering modifying the field with reflection, hacky as damn but should work I think.

Alternatively I was also thinking about plugging myself somewhere around ITypeInfo and recognizing scala types in isOptional, isCollection but this seemed much more complicated.

[remkop]

I think there's an OptionSpec.Builder constructor that takes an existing OptionSpec, and similarly for PositionalParamSpec.Builder (constructor that takes a PositionalParamSpec). Is that what you're looking for?

[Krever]

Yes, that's it. Thank you! I haven't looked carefully enough, sorry!