Path can not be reached: schema.json

[anilkulkarni87]

I have downloaded the jar file and imported the Schemaconverter and the below libraries import org.apache.spark.sql.types. import play.api.libs.json. import scala.annotation.tailrec import scala.io.Source

Trying to load a schema using the Windows command line val schemas = SchemaConverter.convert("schema.json")

Error: java.lang.IllegalArgumentException: requirement failed: Path can not be reached: schema.json at scala.Predef$.require(Predef.scala:224) at org.zalando.spark.jsonschema.SchemaConverter$.loadSchemaJson(SchemaConverter.scala:89) at org.zalando.spark.jsonschema.SchemaConverter$.convert(SchemaConverter.scala:45) ... 52 elided

What path should be passed for the filepath? Since its a jar file I do not have a resources folder in ym project yet.

[hesserp]

Hi, currently the 'convert'-function only works relative to the resources-folder. But there is the 'convertContent'-function. For this you can just load the schema file as a string and use this as input.

e.g:

val fileContents = Source.fromFile("/path/to/schema.json").getLines.mkString
val schema = SchemaConverter.convertContent(fileContents)

[anilkulkarni87]

@hesserp I tried the like the above example. But i am getting an exception as below. Not sure about the cause fo this exception.

java.lang.NoSuchMethodError: scala.collection.immutable.$colon$colon.hd$1()Ljava/lang/Object; at play.api.libs.json.jackson.JsValueDeserializer.deserialize(JacksonJson.scala:162) at play.api.libs.json.jackson.JsValueDeserializer.deserialize(JacksonJson.scala:126) at play.api.libs.json.jackson.JsValueDeserializer.deserialize(JacksonJson.scala:121) at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:3708) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2005) at play.api.libs.json.jackson.JacksonJson$.parseJsValue(JacksonJson.scala:233) at play.api.libs.json.StaticBinding$.parseJsValue(StaticBinding.scala:15) at play.api.libs.json.Json$.parse(Json.scala:168) at org.zalando.spark.jsonschema.SchemaConverter$.parseSchemaJson(SchemaConverter.scala:85) at org.zalando.spark.jsonschema.SchemaConverter$.convertContent(SchemaConverter.scala:43) ... 52 elided

[hesserp]

That looks as if the json-parser from play is failing. So the following code should give the same error, right?

val fileContents = Source.fromFile("/path/to/schema.json").getLines.mkString
Json.parse(fileContents)

So either there is something wrong with your schema.json or with the parser from play. You could try, if you can run it with some simple test schema.json, e.g. there are some test jsons in this project (https://github.com/zalando-incubator/spark-json-schema/tree/master/src/test/resources). There are some json validators out there, so you could try one of them to check for errors in your schema.json.

[anilkulkarni87]

There are no errors while i validate my json schema. Also i tried the same with one of the test json schemas provided https://github.com/zalando-incubator/spark-json-schema/blob/master/src/test/resources/testJsonSchema.json

I still get the same error

scala> Json.parse(fileContents) java.lang.NoSuchMethodError: scala.collection.immutable.$colon$colon.hd$1()Ljava/lang/Object; at play.api.libs.json.jackson.JsValueDeserializer.deserialize(JacksonJson.scala:162) at play.api.libs.json.jackson.JsValueDeserializer.deserialize(JacksonJson.scala:126) at play.api.libs.json.jackson.JsValueDeserializer.deserialize(JacksonJson.scala:121) at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:3708) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2005) at play.api.libs.json.jackson.JacksonJson$.parseJsValue(JacksonJson.scala:233) at play.api.libs.json.StaticBinding$.parseJsValue(StaticBinding.scala:15) at play.api.libs.json.Json$.parse(Json.scala:168) ... 54 elided

scala> val schema = SchemaConverter.convertContent(fileContents) java.lang.NoSuchMethodError: scala.collection.immutable.$colon$colon.hd$1()Ljava/lang/Object; at play.api.libs.json.jackson.JsValueDeserializer.deserialize(JacksonJson.scala:162) at play.api.libs.json.jackson.JsValueDeserializer.deserialize(JacksonJson.scala:126) at play.api.libs.json.jackson.JsValueDeserializer.deserialize(JacksonJson.scala:121) at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:3708) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2005) at play.api.libs.json.jackson.JacksonJson$.parseJsValue(JacksonJson.scala:233) at play.api.libs.json.StaticBinding$.parseJsValue(StaticBinding.scala:15) at play.api.libs.json.Json$.parse(Json.scala:168) at org.zalando.spark.jsonschema.SchemaConverter$.parseSchemaJson(SchemaConverter.scala:85) at org.zalando.spark.jsonschema.SchemaConverter$.convertContent(SchemaConverter.scala:43) ... 54 elided

[hesserp]

As the line Json.parse(fileContents) does not work, your problem is with the play json library. The problem is not about the input data as the test json is also not working for you. You need to be able to parse jsons with the play json parser in order to use this converter here, but fixing the play json parser is out of scope for this project.

Sorry as this is a quite unsatisfactory reply.

[anilkulkarni87]

@hesserp No problems. But just to understand the cause of this Are you able to use this with the latest version of Spark?

Could this be one of the reason? http://stackoverflow.com/questions/27453552/nosuchmethoderror-while-running-scalatest

[hesserp]

There could be some version conflicts, but first the Json.parse() from Play must work or the rest cannot succeed as well. This is even independent from Spark. For using Json.parse() you only need to import play.api.libs.json._ (and probably scala.io.Source for reading some file)