zio / zio-config

Easily use and document any config from anywhere in ZIO apps
https://zio.dev/zio-config
Apache License 2.0
231 stars 112 forks source link

Add ConfigSource#toLayer #666

Open jdegoes opened 2 years ago

jdegoes commented 2 years ago

Currently, wrapping a ConfigSource into a layer will do the wrong thing.

e.g. fromHocon(...).toLayer.

The problem is that the memoization happens in the "outer" ZManaged of the config source. But layers require memoization happen as part of layer construction.

In order to address this, we can add a tolayer method to ConfigSource, which shifts the outer layer into the ZLayer, so that it will behave properly, according to user expectations.

trait ConfigSource {
  def toLayer: ZLayer[Any, Nothing, Has[ConfigSource]] = ???
  ...
}
afsalthaj commented 2 years ago

@jdegoes Could you elaborate a bit more on this? Example, Given we have the following implementation of properties-file ConfigSource (where resource acquisition and release is in terms of ZManaged) what's the implication of using ZLayer?


val x : ZIO[Any, ReadError[K], MyConfig] = read(config from fromPropertiesFile(path))

def propertiesFile(
  filePath: String,
  keyDelimiter: Option[Char] = None,
  valueDelimiter: Option[Char] = None,
  filterKeys: String => Boolean = _ => true
): ConfigSource = {
      val managed: ZManaged[Any, ReadError[K], PropertyTreePath[String] => UIO[PropertyTree[String, String]]] =
        ZManaged
          .make({
            ZIO.effect(
              new FileInputStream(new File(filePath))
            ).flatMap(properties => ZIO.effect {
              val properties = new java.util.Properties()
               properties.load(inputStream)
               properties
            })
          }) { r => ZIO.effectTotal(r.close()) }
          .map { tree =>  (path: PropertyTreePath[K]) => ZIO.succeed(tree.at(path)) }
          .mapError(throwable => ReadError.SourceError(throwable.toString))

      Reader(
        Set(ConfigSourceName(filePath)),
        ZManaged.succeed(managed)
      )
    }

User can also do read(config from fromPropertiesFile(path).memoized) which implies if the same config-source is referred multiple times during the read, the resource management is memoized (or the effect that is required to retrieve the ConfigSource is memoized)

afsalthaj commented 2 years ago

@jdegoes As of now what I have as toLayer is this:

https://github.com/zio/zio-config/blob/e903eea859f5c8621db52b80f859b63f25ecf504/core/shared/src/main/scala/zio/config/ConfigSourceModule.scala#L71

    /**
     * With `strictlyOnce`, regardless of the number of times `read`
     * is invoked, `ConfigSource` is evaluated
     * strictly once.
     *
     * It returns an Effect, because by the time ConfigSource is retrieved,
     * an effect is performed (which may involve a resource acquisition and release)
     *
     * {{{
     *   val sourceZIO = ConfigSource.fromPropertiesFile(...).strictlyOnce
     *
     *   for {
     *     src     <- sourceZIO
     *     result1 <- read(config from src)
     *     result2 <- read(config from src)
     *   } yield (result1, result2)
     *
     * }}}
     *
     * In this case, the propertiesFile is read only once.
     *
     * vs
     *
     * {{{
     *   val source: ConfigSource =
     *     ConfigSource.fromPropertiesFile(...).memoize
     *
     *   for {
     *     result1 <- read(config from source)
     *     result2 <- read(config from source)
     *   } yield (result1, result2)
     *
     * }}}
     *
     * In this case, the propertiesFile is read once per each read, i.e, twice.
     */
    def strictlyOnce: ZIO[Any, ReadError[K], ConfigSource] =
      (self match {
        case ConfigSource.OrElse(self, that) =>
          self.strictlyOnce.orElse(that.strictlyOnce)

        case ConfigSource.Reader(names, access) =>
          val strictAccess = access.flatMap(identity).use(value => ZIO.succeed(value))
          strictAccess.map(reader => Reader(names, ZManaged.succeed(ZManaged.succeed(reader))))
      })

    /**
     * A Layer is assumed to be "memoized" by default, i.e the construction
     * of ConfigSource layer is done strictly once regardless of number times the read is invoked.
     */
    def toLayer: ZLayer[Any, ReadError[K], Has[ConfigSource]] =
      strictlyOnce.toLayer