json / jsonutils cannot unserialize table on js target

[enthus1ast]

What happened?

Unserializing a table on the js target fails. The c backend works. Just run the code once on the c target, to generate the data. Then run it on the js target to see the error.

import json
import tables
import std/jsonutils

when not defined(js):
  import os
  var hs = initTable[int, int](5)
  hs[0] = 1
  echo hs
  echo toJson(hs)
  writeFile(getAppDir() / "tab.json", $(toJson(hs)))

when defined(js):
  const hss = staticRead("tab.json")
  echo hss
  echo hss.parseJson()
  echo hss.parseJson().jsonTo(Table[int, int])

# 1:
# nim c -r foo.nim
#
# 2:
# nim js -r foo.nim
#
# Error: Error: unhandled exception: Incorrect JSON kind. Wanted '{JInt}' in '' but got 'JString'. [JsonKindError]

{"data":[{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":-5940405725068231575,"key":0,"val":1},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0}],"counter":1}
{"data":[{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":-5940405725068232000,"key":0,"val":1},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0},{"hcode":0,"key":0,"val":0}],"counter":1}
c:\Users\david\projects\nimFulltextsearch\hashdump.js:5001
    throw new Error(cbuf_33556665);
    ^

Error: Error: unhandled exception: Incorrect JSON kind. Wanted '{JInt}' in '' but got 'JString'. [JsonKindError]
Traceback (most recent call last)
c:\Users\david\projects\nimFulltextsearch\hashdump.nim(84) at module hashdump
C:\Users\david\.choosenim\toolchains\nim-#devel\lib\std\jsonutils.nim(299) at jsonTo.jsonTo
C:\Users\david\.choosenim\toolchains\nim-#devel\lib\std\jsonutils.nim(162) at fromJson.fromJson
C:\Users\david\.choosenim\toolchains\nim-#devel\lib\std\jsonutils.nim(263) at fromJson.fromJson
C:\Users\david\.choosenim\toolchains\nim-#devel\lib\std\jsonutils.nim(162) at fromJson.fromJson
C:\Users\david\.choosenim\toolchains\nim-#devel\lib\std\jsonutils.nim(231) at fromJson.fromJson
C:\Users\david\.choosenim\toolchains\nim-#devel\lib\pure\json.nim(1365) at to.to
C:\Users\david\.choosenim\toolchains\nim-#devel\lib\pure\json.nim(1060) at initFromJson.initFromJson

    at unhandledException (c:\Users\david\projects\nimFulltextsearch\hashdump.js:5001:11)
    at raiseException (c:\Users\david\projects\nimFulltextsearch\hashdump.js:430:5)
    at initFromJson_436208047 (c:\Users\david\projects\nimFulltextsearch\hashdump.js:7704:5)
    at to_436208040 (c:\Users\david\projects\nimFulltextsearch\hashdump.js:7724:5)
    at fromJson_436208027 (c:\Users\david\projects\nimFulltextsearch\hashdump.js:7740:36)
    at fromJson_436207920 (c:\Users\david\projects\nimFulltextsearch\hashdump.js:7929:5)
    at fromJson_436207753 (c:\Users\david\projects\nimFulltextsearch\hashdump.js:8047:13)
    at fromJson_436207634 (c:\Users\david\projects\nimFulltextsearch\hashdump.js:8088:5)
    at jsonTo_436207628 (c:\Users\david\projects\nimFulltextsearch\hashdump.js:8158:5)
    at Object.<anonymous> (c:\Users\david\projects\nimFulltextsearch\hashdump.js:8171:25)
Error: execution of an external program failed: '"C:\Program Files\nodejs\node.exe" --unhandled-rejections=strict c:\Users\david\projects\nimFulltextsearch\hashdump.js'

Nim Version

PS C:\Users\david> nim -v
Nim Compiler Version 1.7.1 [Windows: amd64]
Compiled at 2022-08-18
Copyright (c) 2006-2022 by Andreas Rumpf

active boot switches: -d:release

Current Standard Output Logs

No response

Expected Standard Output Logs

No response

Possible Solution

No response

Additional Information

No response

[ire4ever1190]

Looks like the problem is from the hcode num being too large so its parsed as a raw num (which acts like a string)

json.nim has the logic for handling raw nums but doesn't pick up that the number could be too large since Hash is defined as just int and so doesn't believe it could to too large

solution might be making Hash a distinct type so it can be overloaded properly in json.

But there is also a workaround for the example you gave where you properly convert it to JSON so that it isn't sending hashes (not fully tested but something like this should work)

proc toJsonHook*[T](table: Table[int, T]): JsonNode =
  result = newJObject()
  for k, v in table: result[$k] = %v

proc fromJsonHook*[V](t: var Table[int, V], jsonNode: JsonNode) =
  clear(t)
  for k, v in jsonNode:
    t[k.parseInt()] = jsonTo(v, V)