Is fileType = csv supported?

[zoraida]

I have tried to run ExampleCustomer.conf configuration and after getting the exception: 18/01/14 20:27:13 WARN apps.DQMasterBatch$: **** Exception in thread "main" it.agilelab.bigdata.DataQuality.exceptions.IllegalParameterException: Unknown parameters = csv at it.agilelab.bigdata.DataQuality.configs.ConfigReader$$anonfun$getSourcesById$1.apply(ConfigReader.scala:81) at it.agilelab.bigdata.DataQuality.configs.ConfigReader$$anonfun$getSourcesById$1.apply(ConfigReader.scala:61)

I run into the code and it looks like only "fixed" fileType is supported:

        val schema    = generalConfig.getString("fileType") match {
          case "fixed"   => {
            if      (Try(generalConfig.getObjectList("schema")).isSuccess) getFixedStructSchema(generalConfig)
            else if (Try(generalConfig.getStringList("fieldLengths")).isSuccess) getFixedSchema(generalConfig)
            else {
              val allKeys = generalConfig.entrySet().map(_.getKey)
              throw new IllegalParameterException("\n CONFIG: "+allKeys.mkString(" - "))
            }
          }
          case x => throw new IllegalParameterException(x)
        }

Is there any other more updated branch than master? Thanks!

[zoraida]

Could you at least upload an example of how to specify the schema for fixed fileType?

[GianRock]

Hi @zoraida we are aware of that issue, indeed very soon we will update the project with a new release that not only will fix that issue but will also introduce new features.

[zoraida]

Hi! That sounds great! Thanks!

De: GianRock notifications@github.com Responder a: agile-lab-dev/DataQuality reply@reply.github.com Fecha: lunes, 22 de enero de 2018, 11:47 Para: agile-lab-dev/DataQuality DataQuality@noreply.github.com CC: ZORAIDA HIDALGO SANCHEZ zoraida.hidalgosanchez@telefonica.com, Mention mention@noreply.github.com Asunto: Re: [agile-lab-dev/DataQuality] Is fileType = csv supported? (#1)

Hi @zoraidahttps://github.com/zoraida we are aware of that issue, indeed very soon we will update the project with a new release that not only will fix that issue but will also introduce new features.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/agile-lab-dev/DataQuality/issues/1#issuecomment-359387515, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACWQzoJOTkkTRY7g8xCTmXUwxuAd_TWzks5tNGdGgaJpZM4Rdt1u. ---INFORMACION IMPORTANTE ITUSER---

Este correo se ha recibido en tu dirección de correo @TID.ES

Por favor contacta con el Origen y realiza las gestiones oportunas para cambiarlo a tu buzón @telefonica.com

---

---

Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción.

The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it.

Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição

[zoraida]

That sounds great! Thanks!

[zoraida]

Hi! any update on this?

[GianRock]

@zoraida did you try the new relase?