Unacceptable compilation time

[vasily-kirichenko]

First of all, the library is cool and all, but the following code compiles for 24 seconds (!):

import eu.timepit.refined.auto._
import eu.timepit.refined.api.Refined
import eu.timepit.refined.boolean.AnyOf
import eu.timepit.refined.char._
import shapeless._

object Main {
  type MyChar = Char Refined AnyOf[Digit :: Letter :: Whitespace :: HNil]

  val x00: MyChar = 'f'
  val x01: MyChar = 'f'
  val x02: MyChar = 'f'
  val x03: MyChar = 'f'
  val x04: MyChar = 'f'
  val x05: MyChar = 'f'
  val x06: MyChar = 'f'
  val x07: MyChar = 'f'
  val x08: MyChar = 'f'
  val x09: MyChar = 'f'

  val x10: MyChar = 'f'
  val x11: MyChar = 'f'
  val x12: MyChar = 'f'
  val x13: MyChar = 'f'
  val x14: MyChar = 'f'
  val x15: MyChar = 'f'
  val x16: MyChar = 'f'
  val x17: MyChar = 'f'
  val x18: MyChar = 'f'
  val x19: MyChar = 'f'

  val x20: MyChar = 'f'
  val x21: MyChar = 'f'
  val x22: MyChar = 'f'
  val x23: MyChar = 'f'
  val x24: MyChar = 'f'
  val x25: MyChar = 'f'
  val x26: MyChar = 'f'
  val x27: MyChar = 'f'
  val x28: MyChar = 'f'
  val x29: MyChar = 'f'

  val x30: MyChar = 'f'
  val x31: MyChar = 'f'
  val x32: MyChar = 'f'
  val x33: MyChar = 'f'
  val x34: MyChar = 'f'
  val x35: MyChar = 'f'
  val x36: MyChar = 'f'
  val x37: MyChar = 'f'
  val x38: MyChar = 'f'
  val x39: MyChar = 'f'

  val x40: MyChar = 'f'
  val x41: MyChar = 'f'
  val x42: MyChar = 'f'
  val x43: MyChar = 'f'
  val x44: MyChar = 'f'
  val x45: MyChar = 'f'
  val x46: MyChar = 'f'
  val x47: MyChar = 'f'
  val x48: MyChar = 'f'
  val x49: MyChar = 'f'

  val x50: MyChar = 'f'
  val x51: MyChar = 'f'
  val x52: MyChar = 'f'
  val x53: MyChar = 'f'
  val x54: MyChar = 'f'
  val x55: MyChar = 'f'
  val x56: MyChar = 'f'
  val x57: MyChar = 'f'
  val x58: MyChar = 'f'
  val x59: MyChar = 'f'

  val x60: MyChar = 'f'
  val x61: MyChar = 'f'
  val x62: MyChar = 'f'
  val x63: MyChar = 'f'
  val x64: MyChar = 'f'
  val x65: MyChar = 'f'
  val x66: MyChar = 'f'
  val x67: MyChar = 'f'
  val x68: MyChar = 'f'
  val x69: MyChar = 'f'

  val x70: MyChar = 'f'
  val x71: MyChar = 'f'
  val x72: MyChar = 'f'
  val x73: MyChar = 'f'
  val x74: MyChar = 'f'
  val x75: MyChar = 'f'
  val x76: MyChar = 'f'
  val x77: MyChar = 'f'
  val x78: MyChar = 'f'
  val x79: MyChar = 'f'

  val x80: MyChar = 'f'
  val x81: MyChar = 'f'
  val x82: MyChar = 'f'
  val x83: MyChar = 'f'
  val x84: MyChar = 'f'
  val x85: MyChar = 'f'
  val x86: MyChar = 'f'
  val x87: MyChar = 'f'
  val x88: MyChar = 'f'
  val x89: MyChar = 'f'

  val x90: MyChar = 'f'
  val x91: MyChar = 'f'
  val x92: MyChar = 'f'
  val x93: MyChar = 'f'
  val x94: MyChar = 'f'
  val x95: MyChar = 'f'
  val x96: MyChar = 'f'
  val x97: MyChar = 'f'
  val x98: MyChar = 'f'
  val x99: MyChar = 'f'

  val x100: MyChar = 'f'
  val x101: MyChar = 'f'
  val x102: MyChar = 'f'
  val x103: MyChar = 'f'
  val x104: MyChar = 'f'
  val x105: MyChar = 'f'
  val x106: MyChar = 'f'
  val x107: MyChar = 'f'
  val x108: MyChar = 'f'
  val x109: MyChar = 'f'
}

[fthomas]

The reason for long compile times is that each assignment invokes a macro that compiles and evaluates an Expr[Validate[Char, AnyOf[Digit :: Letter :: Whitespace :: HNil]]] which is a costly operation. We could try adding a cache for Validate instances to that macro so that each unique instance is only evaluated once instead of 110 times as in your example.

[vasily-kirichenko]

@fthomas Yes, it sounds good. But it would not solve the problem in general, where you can have hundreds of (different) refined types. What I'm trying to say is that this library should be used with care on large projects.

[fthomas]

That caveat only applies if you have lots of static data that you want to check at compile-time. In my projects the vast majority of the validation refined performs happens at runtime (e.g. via circe-refined, doobie-refined, and other integrations) while there are only a handful of literals that are checked at compile-time. And the runtime stuff has not that much impact on compilation time.

[howyp]

In my experience, the compilation time has only been a problem with complicated predicates or with lots of compile-time validations.

@fthomas I was wondering if you knew if it's the macro that takes time, or the implicit resolution of the Validate instance? I was hoping the changes made to re-organise the Validate instances in 0.9.0 might have made resolution a good bit faster?

[fthomas]

@howyp I've not measured how compilation time changed between 0.8.7 and 0.9.0 due to implicit resolution. That would be interesting to know. What I know is that if the macro is invoked with a Validate instance that we don't precompute this eval call is the biggest contributor to compilation time.