search

zalando-incubator / spark-json-schema

JSON schema parser for Apache Spark
MIT License
81 stars 43 forks source link

ClassNotFoundException: scala.Product$class #52

Open srijitcn opened 2 years ago

srijitcn commented 2 years ago

Hi, I am trying to use this library to convert json-schema to Spark Structs. Using scala version 2.12.14

import org.zalando.spark.jsonschema.SchemaConverter
import scala.io.Source

val schemaUrl = "https://raw.githubusercontent.com/CMSgov/price-transparency-guide/master/schemas/allowed-amounts/allowed-amounts.json"
val html = Source.fromURL(schemaUrl)
val inputSchema = html.mkString
println(inputSchema)
val sparkSchema = SchemaConverter.convertContent(inputSchema)

I get the below exception

NoClassDefFoundError: scala/Product$class
Caused by: ClassNotFoundException: scala.Product$class
    at play.api.libs.json.JsObject.<init>(JsValue.scala:76)
    at play.api.libs.json.JsObject$.apply(JsValue.scala:155)
    at org.zalando.spark.jsonschema.SchemaConverter$.<init>(SchemaConverter.scala:44)
    at org.zalando.spark.jsonschema.SchemaConverter$.<clinit>(SchemaConverter.scala)
    at $line5694e0f58f7a402c8eec91ff13bcf9ab29.$read$$iw$$iw$$iw$$iw$$iw$$iw.<init>(command-2203313569958050:1)
    at $line5694e0f58f7a402c8eec91ff13bcf9ab29.$read$$iw$$iw$$iw$$iw$$iw.<init>(command-2203313569958050:45)
    at $line5694e0f58f7a402c8eec91ff13bcf9ab29.$read$$iw$$iw$$iw$$iw.<init>(command-2203313569958050:47)
    at $line5694e0f58f7a402c8eec91ff13bcf9ab29.$read$$iw$$iw$$iw.<init>(command-2203313569958050:49)

Any idea what I am doing wrong?

simenojensen commented 1 year ago

scala version and spark-json-schema version must use same version of scala