Golang Borsh implementation | Bounty $3000 in NEAR

[ailisp]

Description

A Golang library that allows to generically serialize and deserialize Borsh

Context

Borsh is binary serializer that NEAR uses. Golang developers may want to use it to interact with NEAR. Borsh has a simple specification outlined at https://borsh.io

Also Rust and JS implementations can be found here: https://github.com/near/borsh Java implementation can be found here: https://github.com/near/borshj Ideally Golang implementation should follow interface similar to Rust, where an existing type can be just serialized / deserialized without any extra code.

Acceptance Criteria

• [ ] A go-borsh library that other projects can include to use Borsh

• [ ] Any type can be serialized with obj.borshSerialize() and obj.borshDeserialize(bytes) (or similar), user should be able to skip some fields, such as:

  type Tree struct {
  Value int   `borsh_skip:true`
  Left  *Tree
  Right *Tree
  }

• [ ] Test coverage for the spec similar to Rust tests - https://github.com/near/borsh/tree/master/borsh-rs/borsh/tests

• [ ] Fuzz testing to make sure there are no edge cases

• [ ] Add a readme describe usage, a good example is https://github.com/near/borshj/blob/master/README.md

Bounty

$3,000 in NEAR

[leofisG]

Hi, I am interested in working on this if no one has started. Thanks. ( Update: looks like there is already someone assigned to work on this

[frol]

@524119574 I have not heard of anyone working on it, so please, go ahead!

[leofisG]

I'd like to attempt implementing this. But I do have some questions before starting:

1. GoLang doesn't have a built-in type for option type nor generic what are your thoughts on best representing the option type in GoLang? I am thinking maybe I can do something like:

   type Optional struct {
   Val interface{}
   IsPresent bool
   } 

   Any other thoughts/recommendation?

2. Does the borshj libraries support map or set? I was trying to follow the code path for the Go implementation, but I can't seem to find the implementation nor tests.
3. What will be a good way to do fuzz testing in Golang. I am a bit unfamiliar with this, it will be great if you could give me some pointer on this front.
4. Also, unlike rust, go doesn't have enum type, how should I go about implementing this?

Thank you and I hope the questions make sense. And looking forward to hearing from you.

[ailisp]

For 1 I think your solution works well and semantically correct, but introduced a interface{} type which is slower than original specific type. What do you think of simply a pointer of original (when nil means not present).

For 2. borshj seems not, but for original borsh-rs it does, and I think it's better to at least support map in borsh-go because it's a golang primitive.

3. What will be a good way to do fuzz testing in Golang. I am a bit unfamiliar with this, it will be great if you could give me some pointer on this front.

https://github.com/google/gofuzz? Usage looks straightfoward in readme, although honestly I haven't used before too

4. Also, unlike rust, go doesn't have enum type, how should I go about implementing this?

The purpose of enum in rust is provide a union type. In golang enum is c-like, so it's adequate to serialize int

type Direction int

const (
    North Direction = iota
    East
    South
    West
)

func (d Direction) String() string {
    return [...]string{"North", "East", "South", "West"}[d]
}

var d Direction
d = South
d.serialize() // should work 

[matiasinsaurralde]

Would a cgo wrapper work for you?

[StEvUgnIn]

I may start design a PlantUML diagram. When done, I'll start implementing.

[ouromoros]

Hi, it seems that currently no one is working on it. I think I'll give it a try.

[ouromoros]

About half the code is done :) Can I get assigned? @ailisp

[ouromoros]

https://github.com/ouromoros/borsh-go

Work schedule:

• [x] Implement non-native types (Enum, Set)
• [x] Support for struct tags for borsh_skip
• [x] More comprehensive tests
• [x] Documentation
• [x] Fuzz test

[frol]

@ouromoros Great job! I think it is worth explicitly mentioning on the README that more usage examples can be seen in the borsh_test.go file.

@evgenykuzyakov Could you help to review this Borsh implementation in Go? Do we have some set of binary blobs that we can use to test implementation compatibility?

[ouromoros]

@frol Sure, I've added that mention.

I think I'll also find time to write some in-comment docs.

[ailisp]

@ouromoros I checked your implementation and looks awesome! The test coverage is more than great. @nearmax imo this satisfies our accepting criteria

[ouromoros]

@ailisp Thanks! I wonder if my next step would be: 1. move the project under near 2. collect my bounty :)

[frol]

@ouromoros

1. yes, let's arrange that, please, add me to the repo owners, so I can transfer it
2. I am going to remind @nearmax, sorry for the delay

[ouromoros]

@frol Sure, seems like Github doesn't have an add owner option. So I guess I'll just transfer it to you first.

I'm willing to wait as long as NEAR doesn't 🚀 🚀 🚀 to the 🌕 before I get paid ;)

[frol]

@ouromoros Great! I have already transferred it to https://github.com/near/borsh-go :rocket: