Question about Option support

[al6x]

Hi, wondering why Bson doesn't support auto-conversion of Option?

It code below the bson -> json -> object conversion works, but the bson -> object does not. Wonder why?

type
  User* = object
    name*: string
    age*:  Option[int]

let jim = User(name: "Jim", age: 30.some)

echo jim.to_bson.to(User)
echo jim.to_bson.to_json.to(User)

Produces

(name: "Jim", age: None[int])
(name: "Jim", age: Some(30))

Full example

import anonimongo, options, json

proc to_bson*[T](v: Option[T]): BsonBase =
  if v.is_some: v.get.to_bson else: bson_null()

proc to_bson[T: tuple | object](o: T): BsonDocument =
  result = new_bson()
  for k, v in o.field_pairs:
    result[k] = v.to_bson

proc toJson(b: BsonDocument): JsonNode

proc convertElem(v: BsonBase): JsonNode =
  case v.kind
  of bkInt32, bkInt64: result = newJInt v.ofInt
  of bkString: result = newJString v.ofString
  of bkBinary: result = newJString v.ofBinary.stringbytes
  of bkBool: result = newJBool v.ofBool
  of bkDouble: result = newJFloat v.ofDouble
  of bkEmbed: result = v.ofEmbedded.toJson
  of bkNull: result = newJNull()
  of bkTime: result = newJString $v.ofTime
  of bkArray:
    var jarray = newJArray()
    for elem in v.ofArray:
      jarray.add elem.convertElem
    result = jarray
  else:
    discard

proc toJson(b: BsonDocument): JsonNode =
  result = newJObject()
  for k, v in b:
    result[k] = v.convertElem

type
  User* = object
    name*: string
    age*:  Option[int]

let jim = User(name: "Jim", age: 30.some)

echo jim.to_bson.to(User) # Wrong
echo jim.to_bson.to_json.to(User) # Correct

[mashingan]

Due to additional layers of type information it holds, Option specifically and/or generic generally isn't supported. The current workaround is mentioned here, at point #9.

While it can be added but I'm still not sure on how to do it on top of current implementation. (And yeah, the current conversion implementation quite a mess I admit).
What I can think the solution is whether to simply take out all additional layers of the generic types, or recursively convert each generic. With the former, it's unreliable while the later would need insight, tests and explorations whether it's doable. Either way, both need a good plan on how to implement.

Since the start, I planned to have custom conversion mechanism for users to implement, users can add to{Typename} proc or converter as mentioned in this part working with Bson, point #2.

With custom implementation, hopefully it's enough.

The reason why bson -> json -> object is able to get the Option[T] is because json module supports the generic/option conversions.

[al6x]

Thanks for quick reply!

Since the start, I planned to have custom conversion mechanism for users to implement, users can add to{Typename} proc or converter as mentioned in this part working with Bson, point #2.

Hmm, I would say that huge advantage of MongoDB over other databases is that it works with native objects, and support flexible data scheme out of the box. Need to write custom conversion don't fit well with MongoDB...

Anyway, there's still an option to use automatic bson -> json -> object bridge, it should be good enough for my project :)

[mashingan]

This example of optional int is flexible enough to accept bson of int or string, and even nil

from strutils import parseInt
from options import some, none, Option, get, isNone
import anonimongo/core/bson

type
  CanAcceptStringInt = Option[int]
  Myobj = object
    value {.bsonExport.}: CanAcceptStringInt

let withStrVal = bson {
  value: "5",
}
let withIntVal = bson {
  value: 10,
}
let withNilVal = bson {
  value: nil,
}

proc ofCanAcceptStringInt(b: BsonBase): Option[int] =
  if b.kind == bkString:
    var valstr = b.ofString
    result = try: some(parseInt(valstr)) except: none[int]()
  elif b.kind == bkInt32:
    result = some b.ofInt
  elif b.kind == bkNull:
    result = none[int]()
  else:
    result = none[int]()

var mob: MyObj
mob = withStrVal.to Myobj
doAssert mob.value.get == 5
mob = withIntVal.to Myobj
doAssert mob.value.get == 10
mob = withNilVal.to Myobj
doAssert mob.value.isNone

Using json bridging will fail for this flexibility. In case of there's no fix object schema for the Bson object, it's better to use the Bson itself directly, it can accept new fields, delete old fields, replace the value of a field with different type (from bson 5 to bson "5" or even to true or false), etc.
The native types already defined as converter so no need to type toBson like for example:

var tosendBson = bson {
  valueInt: 5,
}

tosendBson["valueInt"] = 10 # replace directly the 5 to 10 without adding toBson

# the opposite is same too
var five: int
# for example we got the tosendBson is from mongodb
five = tosendBson["valueInt"] - 5 # this immediately got 10 - 5
doAssert five == 5

Check the bson tests to see how flexible it is.