Closed hntd187 closed 6 years ago
Exception in thread "main" java.lang.ClassCastException: java.lang.Integer cannot be cast to scala.collection.Seq
at io.crossref.Publication.<init>(JsonObjects.scala:123)
at io.crossref.JsonImplicits$$anon$1.d4(JsonImplicits.scala:7)
at io.crossref.JsonImplicits$$anon$1.d3(JsonImplicits.scala:7)
at io.crossref.JsonImplicits$$anon$1.d1(JsonImplicits.scala:7)
at io.crossref.JsonImplicits$$anon$1.d0(JsonImplicits.scala:7)
at io.crossref.JsonImplicits$$anon$1.decodeValue(JsonImplicits.scala:7)
at io.crossref.JsonImplicits$$anon$1.decodeValue(JsonImplicits.scala:7)
at com.github.plokhotnyuk.jsoniter_scala.core.JsonReader.read(JsonReader.scala:529)
at com.github.plokhotnyuk.jsoniter_scala.core.package$.readFromArray(package.scala:101)
at io.crossref.Testing$.delayedEndpoint$io$crossref$Testing$1(CR.scala:67)
at io.crossref.Testing$delayedInit$body.apply(CR.scala:51)
at scala.Function0.apply$mcV$sp(Function0.scala:34)
at scala.Function0.apply$mcV$sp$(Function0.scala:34)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App.$anonfun$main$1$adapted(App.scala:76)
at scala.collection.immutable.List.foreach(List.scala:389)
at scala.App.main(App.scala:76)
at scala.App.main$(App.scala:74)
at io.crossref.Testing$.main(CR.scala:51)
at io.crossref.Testing.main(CR.scala)
So the line it thinks are bad is my reference: Seq[Reference] = Seq.empty[Reference]
Hi, Stephen!
Thanks for reporting!
I have reproduced that exception in the following test: https://github.com/plokhotnyuk/jsoniter-scala/commit/37b9f29a0db2d2cfe9f3e5de67aa1cd93fb5b42f#diff-7f1e691cb8eee9823fc1e4df91e30c44R9
Also, I see that the generated codec, that you have reported above doesn't throw that error, instead it throws another:
com.github.plokhotnyuk.jsoniter_scala.core.JsonParseException: unexpected field "journal-issue", offset: 0x00000b37, buf:
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+----------+-------------------------------------------------+------------------+
| 00000b10 | 6e 63 65 73 2d 63 6f 75 6e 74 22 3a 20 30 2c 0a | nces-count": 0,. |
| 00000b20 | 20 20 20 20 20 20 20 20 22 6a 6f 75 72 6e 61 6c | "journal |
| 00000b30 | 2d 69 73 73 75 65 22 3a 20 7b 0a 20 20 20 20 20 | -issue": {. |
| 00000b40 | 20 20 20 20 20 22 69 73 73 75 65 22 3a 20 22 31 | "issue": "1 |
| 00000b50 | 2d 32 22 0a 20 20 20 20 20 20 20 20 7d 2c 0a 20 | -2". },. |
+----------+-------------------------------------------------+------------------+
at com.github.plokhotnyuk.jsoniter_scala.core.JsonReader.decodeError(JsonReader.scala:646)
at com.github.plokhotnyuk.jsoniter_scala.core.JsonReader.unexpectedKeyError(JsonReader.scala:75)
at io.crossref.JsonCodecThings.d4(JsonImplicits.scala:2023)
at io.crossref.JsonCodecThings.d3(JsonImplicits.scala:2102)
at io.crossref.JsonCodecThings.d1(JsonImplicits.scala:2191)
at io.crossref.JsonCodecThings.d0(JsonImplicits.scala:2287)
at io.crossref.JsonCodecThings.decodeValue(JsonImplicits.scala:14)
at io.crossref.JsonCodecThings.decodeValue(JsonImplicits.scala:12)
at com.github.plokhotnyuk.jsoniter_scala.core.JsonReader.read(JsonReader.scala:544)
at com.github.plokhotnyuk.jsoniter_scala.core.package$.readFromStream(package.scala:34)
at io.crossref.CrossrefSpec.$anonfun$new$4(CrossrefSpec.scala:16)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
...
And the same error for unexpected field I can get when turning off skipUnexpectedFields
configuration option for the macro call:
val codec: JsonValueCodec[ItemResponse] =
JsonCodecMaker.make[ItemResponse](CodecMakerConfig(skipUnexpectedFields = false))
I have reproduced that java.lang.ClassCastException
in the isolated call of the constructor: https://github.com/plokhotnyuk/jsoniter-scala/commit/338d6cdba92662c187c1206e9f1c6809e21819bd
So it looks more like a bug of the scalac (I've opened an issue for that https://github.com/scala/bug/issues/10825 ), but I will investigate further.
I've mitigated that compiler bug by renaming of reference-count
field to referenceCount
with using of the @named
annotation: https://github.com/plokhotnyuk/jsoniter-scala/commit/e0fabd07db12934492153ca265a684a084d5b7ba
Now tests are failing due unexpected array JSON value, while a JSON object is expected as stated in the case class (relation: Option[Map[String, Relation]],
):
com.github.plokhotnyuk.jsoniter_scala.core.JsonParseException: expected '{', offset: 0x0000788a, buf:
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+----------+-------------------------------------------------+------------------+
| 00007860 | 0a 20 20 20 20 20 20 20 20 22 72 65 6c 61 74 69 | . "relati |
| 00007870 | 6f 6e 22 3a 20 7b 0a 20 20 20 20 20 20 20 20 20 | on": {. |
| 00007880 | 20 22 63 69 74 65 73 22 3a 20 5b 5d 0a 20 20 20 | "cites": []. |
| 00007890 | 20 20 20 20 20 7d 2c 0a 20 20 20 20 20 20 20 20 | },. |
| 000078a0 | 22 49 53 53 4e 22 3a 20 5b 0a 20 20 20 20 20 20 | "ISSN": [. |
+----------+-------------------------------------------------+------------------+
at com.github.plokhotnyuk.jsoniter_scala.core.JsonReader.decodeError(JsonReader.scala:646)
at com.github.plokhotnyuk.jsoniter_scala.core.JsonReader.com$github$plokhotnyuk$jsoniter_scala$core$JsonReader$$tokenError(JsonReader.scala:632)
at com.github.plokhotnyuk.jsoniter_scala.core.JsonReader.readNullOrTokenError(JsonReader.scala:475)
at io.crossref.JsonCodecThings.d44(JsonImplicits.scala:1628)
at io.crossref.JsonCodecThings.d43(JsonImplicits.scala:1637)
at io.crossref.JsonCodecThings.d42(JsonImplicits.scala:1651)
at io.crossref.JsonCodecThings.d4(JsonImplicits.scala:2211)
at io.crossref.JsonCodecThings.d3(JsonImplicits.scala:2256)
at io.crossref.JsonCodecThings.d1(JsonImplicits.scala:2358)
at io.crossref.JsonCodecThings.d0(JsonImplicits.scala:2461)
at io.crossref.JsonCodecThings.decodeValue(JsonImplicits.scala:14)
Changed the issue type to question
(not a bug
), because it is a Scala compiler error and can be easy mitigated by renaming of affected fields with using of the @named
annotation or a field name mapping function for the configuration of the macro call.
Another w/a for the scalac bug is to move a definition of the field with encoded chars (reference-count
in our case) to be after the field that is affected by the exception (after reference
field): https://github.com/plokhotnyuk/jsoniter-scala/commit/eb44f77ad2822f4557223ef733c1b02b97f5e75c#diff-622f98fd354a801ea63286f1c423810bR127
@plokhotnyuk thank you very much for the super fast turn around I appreciate it!
So I think Scala 2.12.6 is slated to come out very soon from the macro regression, so it would seem the compiler bug won't be fixed for sometime. I don't even know if it'll get into 2.12.7. So what do you want to do with this issue? Do you want me to leave it open as an FYI for maybe someone else coming along who has this problem or just close it? The work around solved what I was trying to do so at least there is a solution for the time being.
Other than that if you wanted some real world benchmarks of your library, I wrote some to see how the new JSON parser held up against my old parser, which was Json4s using Jackson.
[info] Benchmark Mode Cnt Score Error Units
[info] JsonBenchmarks.inter_read avgt 20 533.260 ± 44.943 ms/op (x22 Faster)
[info] JsonBenchmarks.inter_write avgt 20 400.184 ± 1.728 ms/op (x14 Faster)
[info] JsonBenchmarks.json4s_read avgt 20 11897.006 ± 171.331 ms/op
[info] JsonBenchmarks.json4s_write avgt 20 5694.458 ± 139.742 ms/op
This is JDK8 and Scala 2.12.4. Some background, I have 300mb-ish dumps of JSON I collect from Crossref, which comprise about 100k publications each. I'd say they cover a pretty broad sample of all kinds of different types and structures that a library would have to cover. I was just interested in the raw time taken to parse and write JSON to and from a case class. As you can see, I think the library does pretty well 🥇 You can find my (probably wrong or bad) benchmark code here if you are interested.
Hi, Stephen!
Thanks for your feedback! Feel free to close the issue if you got an answer that satisfied your needs.
I'll add the Known issues
section to the README with description of this case found by you including a link to the compiler bug.
Also, I'm going to add the FAQ
section where I will gather Q & A about common and special cases, security and performance hints, etc.
I believe that jsoniter-scala will save lot of your time & money for JSON processing. It shines especially when you do not need to hold all JSON data in memory: 1) it can efficiently skip all not needed values, just do not add them to your case classes and turn on skipping of unknown fields in the CodecMakerConfig 2) it has a couple of methods to parse streamed JSON values or JSON array values: https://github.com/plokhotnyuk/jsoniter-scala/blob/17c0ccbed114ba3a35989fe1ff8ec553feda0c80/core/src/main/scala/com/github/plokhotnyuk/jsoniter_scala/core/package.scala#L37-L83 3) custom codecs can be written to provide the most efficient processing.
BTW, in your benchmarks jsoniter-scala do decoding/encoding of UTF-8 bytes, while other library works with UTF-16 encoded strings instead. Also, a manual call of the blackhole consume is not required if you return all results from the benchmark method (check generated sources in target/scala_x.x/src_managed
).
Hi, I'm trying to use this library to derive encoders for the crossref API.
Exception in thread "main" java.lang.ClassCastException: java.lang.Integer cannot be cast to scala.collection.Seq
is the error I get.
The JSON coming back is
and my case classes
Finally here is the generated parser
I really have no idea how to debug this, so I'm just dumping what I know I have. Is there a better way to debug this? My case classes were derived from https://github.com/CrossRef/rest-api-doc/blob/master/api_format.md