search

nim-lang / RFCs

A repository for your Nim proposals.
135 stars 26 forks source link

JsonNode as the standard Nim runtime type (i.e. remove JSON connotation) #80

Open mratsim opened 5 years ago

mratsim commented 5 years ago

-## Context

Recurrent need for dynamic types

Very often, people coming to Nim ask how to represent dynamic types and the most common answer is check out how it's done in the JSON module:

and plenty of examples on IRC as well

2018-12-13_12-10-40

Adding feature requests as well:

Tooling and common representation for non-binary serialisation/deserialisation.

Having JsonNode as a common representation would allow library writer to focus only one a single area when serialising and deserialising and rely on the ecosystem for the rest.

This would also encourage people to develop runtime "JsonNode" AST manipulation and visualisation packages and have everyone benefits from it.

Non-binary formats benefitting

Alternative

The typeinfo module: https://nim-lang.org/docs/typeinfo.html

zah commented 5 years ago

Serialization libraries allocating intermediate dynamic structures are sub-optimal. Furthermore, not every data format maps cleanly to the Json data model.

alehander92 commented 5 years ago

I like the idea in general: but separated in 2 parts:

fully dynamic abstraction (similar to JsObject) variant-like abstraction (similar to JsonNode).

@zah It's true that formats differ sometimes a bit, but on the other hand it's very annoying to interop between YamlValue and JsonValue which are obviously similar (e.g. YamlInt(2) and JsonInt(4))

maybe having a more general variant type as a case object and format-specific variants which disable some of the cases might be nice, but this would require case object analysis everywhere and is not really compatible with Nim's type system

deansher commented 5 years ago

This is similar to a very well explored technique in Haskell called (confusingly for us!) generics. The idea is that for any given original type, the compiler can be asked to construct at compile time a "representation type". The representation type is composed from a modest number of general-purpose types, in a way that enables writing fairly convenient runtime code that is independent of the original type.

The specifics of how this solution evolved in Haskell have been heavily driven by Haskell's strengths, weaknesses, and common idioms. So we shouldn't expect to mimic it literally. But I expect we'd benefit from trying to reproduce its strengths.

Another way of coming at this idea would be to say that Nim's existing Any type is a primitive, unsafe, inconvenient counterpart to Haskell generics. One way of gaining these benefits in Nim might be to wrap a higher-level, safe, convenient API around Any.

Araq commented 5 years ago

Nim's JsonNode is quite bad though, my packedjson offers a more efficient implementation. JSON does not support DateTime though which is really important.

On the other hand, an "extensible" design tends to suffer from the expression problem. Either the datatypes are fixed allowing for an unlimited number of operations you can do with them or the datatypes are open for a limited number of possible operations.

bluenote10 commented 5 years ago

Chiming in, because I also made some experiments with dynamic typing in Nim. My main use cases in that direction were almost covered by the Any type of typeinfo, but without its limitation:

The client needs to ensure that the wrapper does not live longer than x!

I wanted to pass around the Any freely, i.e., I needed an implementation that actually stores the underlying element. I came up with this, however I never verified if it makes sense / is actually legal to do ;). If so, this could be turned into a small lib.

@sillibird With a simplified implementation your example would look like:

import typetraits
import strformat

type
  AnyVal* = ref object of RootObj

  AnyValTyped[T] = ref object of AnyVal
    x: T

method `$`*(a: AnyVal): string {.base.} =
  assert false

method `$`*[T](a: AnyValTyped[T]): string =
  when compiles($a.x):
    &"AnyVal[{name(T)}]({$a.x})"
  else:
    &"AnyVal[{name(T)}]({repr(a.x)})"

proc ofType*(a: AnyVal, T: typedesc): bool =
  a of AnyValTyped[T]

proc toAnyVal*[T](x: T): AnyVal =
  AnyValTyped[T](x: x)

proc to*(a: AnyVal, T: typedesc): T =
  doAssert a.ofType(T), &"Expected type {name(T)} but got {a}"
  # Previously I was using a serious hack...
  # copyMem(result.addr, anyval.getAddr(), sizeOf(T))
  # Maybe this is more sane:
  let aTyped = cast[AnyValTyped[T]](a)
  return aTyped.x

when isMainModule:

  type
    # Storing typeinfo's Any in an object is tricky, because
    # you would have to store the underlying element as well.
    Response = object
      data: AnyVal

  proc f(r: Response) =
    echo r
    if r.data.ofType(int):
      let i = r.data.to(int)
      echo "int: ", i
    elif r.data.ofType(string):
      let s = r.data.to(string)
      echo "string: ", s

  f(Response(data: 1.toAnyVal))
  f(Response(data: "s".toAnyVal))
treeform commented 3 years ago

I don't think JsonNode can serve as a dynamic type that works everywhere. Everyone would just want different things from it. I think we should focus on: