Schemaless serialization?

[taoeffect]

Is it possible to have a feature that auto-generates a schema from a plain-old-javascript-object? Or does Borsch not support that?

[marcus-pousette]

Do you mean something like a 'runtime' thing

const obj = { x: 123 } 

const schema = getSchema(obj)

const bytes = serialize(obj, schema)

deserialize(bytes, schema) 

This would perhaps be doable, the tricky thing is that you need to determine what type 'x' would be, can it be negative, is it a floating point number? What precision? There a lot of uncertainties implied.

A perhaps more structured, but partly automatic approach would be perhaps:

class MyThing {

    @field({type: 'u32'})
    x: number

    @auto // or something
    string: string
}

or something like

@auto
class MyThing 
...

But perhaps it would be more interesting to go to the root cause of the inconveniences with this lib that makes you want this to be automated, do you have many different kinds of object types? Where fields are unknown?

[taoeffect]

the tricky thing is that you need to determine what type 'x' would be

Well, numbers in JavaScript are doubles so that's what you'd probably use.

However, I'm realizing that Borsh may be unusable for situations where you don't know in advance what type of struct you're expecting, is that right?

[marcus-pousette]

Even though the number will be double precision floating point format, when you are sending the data to a server and serializing/deserializing it will be quicker if you in advance know the number for example only going to be 0-255, then you can just use u8 and have the number to only take 8 bits instead of 64.

Yes, that is true. But there ways to work around this, if some parts of your data are known,

e.g.

class MyData {

     @field({type: 'u8'}))
     number: number

     @field({type: 'string'})
     private  jsonString: string // data that have unknown format

     private _obj: any

     constructor(number: number, obj: any){
         this.number = number;
         this._obj = obj;
         this.jsonString = JSON.stringify(this._obj)
     }
     get obj() 
     { 
          return  this._obj = JSON.parse(jsonString)
     }
}

Another example, if you advance know some fields, but eventually, more fields will need to be added, then you can prefix with a version/variant, so that in the future, you can create an updated structure, where both the old and the new structure can co-exist.

@variant(0)
class MyDataV0
....

@variant(1)
class MyDataV1
....

If you want more flexibility than this I would try instead just to use JSON or Protobuf (where the additional encoding allows for more flexibility).

[taoeffect]

Thanks for your help @marcus-pousette and for the clarifications! We are already using JSON but were considering switching to Borsh...

If you want more flexibility than this I would try instead just to use JSON or Protobuf (where the additional encoding allows for more flexibility).

Yeah we're probably going to go with JSON and just require that other languages follow the same rules that browsers use for handling duplicate keys (using the last key).

EDIT: context about my questions related to JSON and why I was considering Borsh: https://news.ycombinator.com/item?id=36822629