datastore: load and write string representation of int enum

[scorsi]

Is your feature request related to a problem? Please describe. I have issues loading enums from/saving enums to datastore.

Describe the solution you'd like I would be great to use the String method if it exists on enums ? Or give a property like datastore:",string" to specify we want the string repr. Maybe there is a solution I didn't found ?

Describe alternatives you've considered I make it working by changing my enum to string but loose good (automatic) enum checks on my API.

Additional context For example:

//go:generate enum Role -text -nosql -transform=kebab

type Role int

const (
    Admin Role = iota
    Standard
)

type User struct {
    Name       string
    Role        Role
}

This will save 1 for admin and 2 for standard. The issue is that I have Dataflow, and other stuffs, working with admin or standard string representation instead of the int value.

[crwilcox]

Can you convert the enum value to the string in before using it in the other instances?

role := Role(2)

[scorsi]

Sorry for the delay I missed the notification. A lot of other systems are reading/writing enum as strings, only my Go service can't, I won't change a lot of systems and risk a system outage :)

Enums in Go are conventionnaly coded as int to lightweight the memory usage (I suppose) and read/written as string from api request/database.

[scorsi]

I am making a PR to resolve that

[crwilcox]

If you need to store and hydrate from a string version, have you tried using the PropertyLoaderSaver interface. Before adding additional support I'd like to understand if this is style, something that couldn't be done, etc. :)

[scorsi]

Hello,

Sorry for the delay to respond...

Judge by yourself:

//go:generate enum MyEnum -text -nosql -transform=kebab

package main

import (
    "cloud.google.com/go/datastore"
    "context"
    "fmt"
    "time"
)

type MyEnum int

const (
    MyEnumA MyEnum = iota
    MyEnumB
    MyEnumC
)

type MyStruct struct {
    MyEnum MyEnum `datastore:"my_enum"`
    MyInt  int    `datastore:"my_int"`
}

func (x *MyStruct) Load(ps []datastore.Property) error {
    var err error
    for _, p := range ps {
        switch p.Name {
        case "my_enum":
            if s, ok := p.Value.(string); !ok {
                err = fmt.Errorf("error")
                break
            } else {
                if err_ := x.MyEnum.UnmarshalText([]byte(s)); err_ != nil {
                    err = err_
                    break
                }
            }
        case "my_int":
            if i, ok := p.Value.(int64); !ok {
                err = fmt.Errorf("error")
                break
            } else {
                x.MyInt = int(i)
            }
        }
    }
    return err
}

func (x *MyStruct) Save() ([]datastore.Property, error) {
    return []datastore.Property{
        {
            Name:
            "my_enum",
            Value: x.MyEnum.String(),
        },
        {
            Name:
            "my_int",
            Value: int64(x.MyInt),
        },
    }, nil
}

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
    defer cancel()

    dsClient, err := datastore.NewClient(ctx, "my-project")
    if err != nil {
        panic(err)
    }
    defer func(dsClient *datastore.Client) {
        err := dsClient.Close()
        if err != nil {
            panic(err)
        }
    }(dsClient)

    key := datastore.NameKey("MyStruct", "stringID", nil)

    structToWrite := &MyStruct{
        MyEnum: MyEnumB,
        MyInt:  42,
    }

    if _, err := dsClient.Put(ctx, key, structToWrite); err != nil {
        panic(err)
    }

    structToRead := new(MyStruct)

    if err := dsClient.Get(ctx, key, structToRead); err != nil {
        panic(err)
    }

    fmt.Printf("%#v %#v\n", structToRead, structToWrite)
}

• I can't use datastore.LoadStruct because of incompatible type (string in datastore vs int in code).
• I could use datastore.SaveStruct but need to remove the old my_enum field and change it with string type.
• I could mark MyStruct.MyEnum with datastore:"-" but got an error with datastore.LoadStruct because unknown field my_enum...

This code is the only one I succeed to make it work. This is not conceivable to manually write the load/save methods... In addition, here to simplify the code, I didn't check if all fields are correctly set/save so the code should contains much more boilerplate.

[scorsi]

I finally found a way to lightweight a little bit the code by using LoadStruct and SaveStruct:

type MyStruct struct {
    MyEnum MyEnum `datastore:"-"` // we exclude for LoadStruct/SaveStruct
    MyInt  int    `datastore:"my_int"`
}

func (x *MyStruct) Load(ps []datastore.Property) error {
    var newPs []datastore.Property
    var myEnumP datastore.Property
    for _, p := range ps {
        if p.Name != "my_enum" {
            newPs = append(newPs, p)
        } else {
            myEnumP = p
        }
    }
    if err := datastore.LoadStruct(x, newPs); err != nil {
        return err
    }
    if s, ok := myEnumP.Value.(string); !ok {
        return fmt.Errorf("error")
    } else {
        if err := x.MyEnum.UnmarshalText([]byte(s)); err != nil {
            return err
        }
    }
    return nil
}

func (x *MyStruct) Save() ([]datastore.Property, error) {
    props, err := datastore.SaveStruct(x)
    if err != nil {
        return nil, err
    }
    props = append(props, datastore.Property{
        Name:  "my_enum",
        Value: x.MyEnum.String(),
    })
    return props, nil
}

It's still to much code for just this but it works.

[scorsi]

@crwilcox did you take the time to look at the snippet before ? :)

[scorsi]

@crwilcox did you find a solution for my problem ? It is "simple" when the element is at the root of the type but when we work with embedded structs in arrays... It's starting to be very difficult to handle it.

[zamicol]

We're trying to do PropertyLoadSaver on a single type (which we use as an enum). The enum is an int, we want to save a string in the database.

Enums don't exists in Go as they do in other languages and the int value may change in the future, whereas a short string will remain constant which we can easily marshal/unmarshal into its enum type. A string is also more compatible across systems. And finally, the datastore int type uses 8 bytes, a string of 7 character is the same size in the database. There's no space advantage in using int over a 7 character string.

PropertyLoadSaver's Load works fine on a single field. Save will not save a single field. It will only save a struct. On a type instead of a struct, Save() saves an embedded entity into the field in the database instead of simply a single property.

In summary, although Load() works well for PropertyLoadSaver on individual custom types, Save() appears broken.

For example, Use.Save() will save an embedded {} in the database. There's no way to save "use" as a string in the database on a type. However, Use.Load() works as expected.

type Use int64

func (u *Use) Save() (ps []datastore.Property, err error) {
    return nil, nil
}

func (u *Use) Load(p []datastore.Property) error {
    s, ok := p[0].Value.(string)
    if ok {
        *u = Use.Parse(s)
    }
    return nil
}

Or alternatively, if storing enums as ints in the database, this works:

func (u *Use) Load(p []datastore.Property) error {
    s, ok := p[0].Value.(int)
    if ok {
        *u = Use.(i)
    }
    return nil
}

[meredithslota]

Reclassifying this as a feature request — we will have to consider it against other proposed features/solutions. Thanks for filing!

[alon-ne]

Hey @meredithslota, I also came across the same exact situation, upvote 👍 for the feature request, thanks!