GORM v2 is on going

[jinzhu]

Hello All,

GORM v2 is under active development (https://github.com/jinzhu/gorm/tree/v2_dev), going to release in the next two months.

Before that, I am NOT going to merge any pull requests based on the master branch.

V2 will be overwritten from scratch with similar public API, it would focus on performance, improve usability and fix fragile designs. We will provide a migration guide to help users migrate before the release.

With the new architecture, it opens the possibility to have a better migration tool, a static code generator that generates type-safe code with all the features of GORM, visualizes models and its relationships to help a new team member quickly understand your project, and even support Redis & MongoDB...

Your code review or suggestions would be much appreciated, please comment to this thread, thank you.

[donutloop]

@jinzhu Could you please add support for generation of migration files like djangos makemigrations? Django's orm is pretty stable, very useful and has lot of good features.

Ref: https://docs.djangoproject.com/en/3.0/ref/django-admin/#django-admin-makemigrations

Example Django application with migrations:

https://github.com/stadtulm/cykel/tree/master/bikesharing/migrations

[MurtadhaS]

• Fix the foreign key relations (that work in a very convoluted way) and in Postgres only.
• Write a sensible comprehensible documentation that describes your library in the right way.

[patrikeh]

Preloads using joins for PostgreSQL (I don't know how that's implemented in other dialects), for me that is by far the most important feature absent in the current version.

As it currently stands, preloads literally stop working once the number of parameters exceed a certain amount. Before then preloads are prohibitively slow partly due to the extra roundtrip per preload, but more importantly due to the missed opportunity for query optimization.

[rjeczalik]

Has any work on static code generator started? I'd be interested in contributing.

[jinzhu]

Hi @rjeczalik haven't started it, just created some draft notes, was going to start it after the release of v2, but if you are interested, I can create a project for the generator under GORM's organization. ( yes, we will have an organization ;) )

Here is the draft notes (feedback is welcome)

Generate static code based on relationships parsed by https://github.com/jinzhu/gorm/tree/v2_dev/schema

generated code looks like

package models // user defiend

type DB struct {
  DB *gorm.DB
  User User
}

type User struct {
  ID       IntField
  Name     StringField
  Languages HasManyRelation
}

var user = User{
  ID: IntField{
    // xxx
  },
  Name: StringField{
    // xxx
  }
  Languages: HasManyRelation{
    // xxx
  }
}

func New(gorm.Dialector, gorm.Config) *DB {
  return &DB{
    DB: gormDB,
    User: user,
  }
}

Usage API

initalize db

db := models.NewDB(sqlite.Open('xxx'), gorm.Config{
})

with context

db = db.WithContext(ctx)

API

// find
db.User.Select(db.User.Name, db.User.Age).Find() (users []*yourapp.User, err error)

// first
db.User.Select(db.User.Name, db.User.Age).First() (*yourapp.User, error)

// last
db.User.Select(db.User.Name, db.User.Age).Last() (*yourapp.User, error)

// last
db.User.Where(db.User.Name.Like("%jinzhu"), db.User.Age.Eq(10)).Last() (*yourapp.User, error)

// preload
db.User.Preload(db.User.Languages.Where(
  db.Languages.Code.Eq("zh-CN")
)).Select(db.User.Name, db.User.Age).Last() (*yourapp.User, error)

// update
db.User.Update(user)

// Relations
db.User.Languages.Model(user).Add([]*yourapp.Languages{})
db.User.Languages.Model(user).Replace(]*yourapp.Languages{})
db.User.Languages.Model(user).Delete([]*yourapp.Languages{})

[rjeczalik]

@jinzhu Not quite sure what e.g. IntField is, is this a pseudocode for wrapper types like sql.NullInt64 or null.Int64?

There's already a functional generator for MySQL - https://github.com/smallnest/gen. What I wanted to achieve is db-agnostic generator with the ability of generating fields for associations (preloading), configurable via struct tags.

[madeofstars0]

@jinzhu Could you please add support for generation of migration files like djangos makemigrations? Django's orm is pretty stable, very useful and has lot of good features.

Maybe instead of coming up with another migration toolkit, have documentation and examples (or an integration) for using pressly/goose for migrations. There are a few other libraries, but goose can be fully integrated or it can be standalone with sql files for the migrations. Couple that with a generator that follows a convention or specified template for the migrations (generator like in rails, where you generate a new migration file with a rails command)

/cc @donutloop

[jinzhu]

@rjeczalik sorry, haven't made things clear, this tool WON'T generate models definition from database

You need to defined it by yourself, for example, you already have your user package

package user

type User struct {
  gorm.Model
  Name         string
  Language     Language
  LanguageCode string
}

type Language struct {
  Code string `gorm:primarykey`
}

With this tool, it will generate another package named db:

package db

type DB struct {
  DB *gorm.DB
  User User
}

type User struct {
  ID        IntField
  Name      StringField
  Languages HasManyRelation
}

func (user User) First() (user.User, error) {
  // xxx
}

var user = User{
  ID: IntField{
    DBName: "id",
  },
  Name: StringField{
    DBName: "name",
  }
  Languages: HasManyRelation{
    ReferenceTable: "languages",
    ForeignKey: "code",
  }
}

func New(gorm.Dialector, gorm.Config) *DB {
  return &DB{
    DB: gormDB,
    User: user,
  }
}

In your application, if you want to use the generated code like:

import 'yourapp/db'

var db = db.New(gorm.dialector, gorm.Config{})

func handler(w http.ResponseWriter, r *http.Request) {
  user, err := db.User.First()
  fmt.Println(user.Name) // string
  json.NewEncoder(w).Encode(user)
}

func main() {
  http.HandleFunc("/", handler)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

[dobegor]

Please review some crucial fixes, we even had to use our fork to mitigate these problems:

https://github.com/jinzhu/gorm/pull/2737 -- erros on nested preloads get ignored, this is a critical issue.

Also please support context for cancellation and tracing support.

Our fork with context support and fix for ignored nested preload errors: https://github.com/readdle/gorm/tree/nested-preload-fix

[rjeczalik]

With this tool, it will generate another package named db:

@jinzhu Let me know if I understand it correctly - you want to have a generator which will generate objects with API similar to what Active Record in RoR is offering, correct?

[jinzhu]

@rjeczalik yes, generate type-safe code with similar API like that, suggestions?

[edwardsb]

allow https://golang.org/pkg/context/ to be passed to the underlying SQL driver.

[jinzhu]

@edwardsb the API will work like this:

tx := db.WithContext(ctx)
tx.First(&user)
tx.Update(&user)

db.WithContext(ctx).First(&user)

(similar to https://golang.org/pkg/net/http/#Request.WithContext)

[chowyu08]

could remove cgo depend for sqlites

[eslizn]

dynamic model support

[jinzhu]

Fix the foreign key relations (that work in a very convoluted way) and in Postgres only.

@MurtadhaS just finished that ;)

[ianarco]

The most important feature for my team would be a more advanced migration tool. I'm really missing Django's migration framework.

Integration into Goose would be fine, as long as the up/down SQL is generated by Gorm.

The main thing is that the migration tool is able to determine automatically what has changed between versions.

[dineshsprabu]

Postgres ErrorCodes support please. https://www.postgresql.org/docs/10/errcodes-appendix.html

[heww]

Savepoint (nested transaction) support in transaction please.

[ianarco]

Also better handling of Postgres ENUM types would be great.

[MurtadhaS]

Another thing. Handle GUID internally.

[DoubleDi]

Hi. Please check out my PR #2921

[fr3fou]

Have a clear-well-defined API.

In my experience with gorm I found that there are multiple ways to achieve the same things which bothered me a lot. I could provide some examples in the next day or so

[vzool]

@jinzhu Nice to see v2 rising. So, please don't forget this #1436 ^_^ Good luck

[jinzhu]

@vzool will make it works with API like db.Joins("Profile").Find(&users)

[profbiss]

There is no api for subqueries.

[ipfans]

Any release schedules of v2? Maybe a TODO list will be very helpful.

Please notices on https://github.com/jinzhu/gorm/issues/2871

[ks129]

Relations is one thing that I hope to see simplified in new release. Currently you have to manage foreign keys yourself, but I hope in v2 this is automatically managed.

[helmus]

@jinzhu any plans to support amazon xray context ? Is this something you deem possible, could use help on ? https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-go-sqlclients.html

[jackskj]

@jinzhu Why are you going the code generation route? I believe that lot of the functionality that generated code provides could be accomplished through reflecting the user struct.

[fr3fou]

@jinzhu Why are you going the code generation route? I believe that lot of the functionality that generated code provides could be accomplished through reflecting the user struct.

performance + typesafety

[inliquid]

performance

When it comes to database transactions this is absolutely negligible.

[helmus]

Type safety is the primary concern that is solved with code generation though.

On Sun, Apr 5, 2020 at 12:53 PM inliquid notifications@github.com wrote:

performance When it comes to database transactions this is absolutely negligible.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jinzhu/gorm/issues/2886#issuecomment-609472683, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH6QHYROASAJQWFVAPGL7TRLDOSNANCNFSM4KV6IZOQ .

[fr3fou]

performance

When it comes to database transactions this is absolutely negligible.

reflection has a performance overhead, but my personal issue is definitely the typesafety

[jackskj]

If type safety is your main concern and a motivation behind this feature, I believe a more sturdy type safety would be provided by generating code based on reverse engineering you sql schema, not by reading your go structs.

[fr3fou]

If type safety is your main concern and a motivation behind this feature, I believe a more sturdy type safety would be provided by generating code based on reverse engineering you sql schema, not by reading your go structs.

sql schemas are more tedious to write tbh compare this:

CREATE TABLE IF NOT EXISTS users (
  id varchar(36) NOT NULL PRIMARY KEY,
  name varchar(255) NOT NULL,
  email varchar(255) NOT NULL,
  password varchar(255) NOT NULL,
  avatar varchar(255) NOT NULL,
  points double NOT NULL,
  role varchar(255) NOT NULL,
  verified tinyint(1) NOT NULL,
  UNIQUE KEY email (email)
-- as well as every relation in other tables
);

with

type User struct {
    ID         uuid.UUID `json:"id,omitempty" db:"id,omitempty"`
    Name       string    `json:"name" db:"name"`
    Email      string    `json:"email" db:"email"`
    Password   string    `json:"-" db:"password"`
    Avatar     string    `json:"avatar" db:"avatar"`
    Points     float64   `json:"points" db:"points"`
    Role       Role      `json:"role" db:"role"`
    Orders     []Order   `json:"orders" db:"-"`
    IsVerified bool      `json:"is_verified" db:"verified"`
}

[jackskj]

I am not sure if this point relates to the code generation feature. SQL can get tedious, but I am implying that your database should be source of truth to your structs. i believe your comment has it the other way around.

[somebodytohave]

v2 How is it going. About how long to release v2 ？ @jinzhu

[jimlambrt]

• Will there be a migration path from v1?
• Is there any backward compatibility or will devs just have to port their code base to v2?

[ifnotak]

@jimlambrt both your questions are answered in the first post.

• Will there be a migration path from v1?
• Is there any backward compatibility or will devs just have to port their code base to v2?

V2 will be overwritten from scratch with similar public API, it would focus on performance, improve usability and fix fragile designs. We will provide a migration guide to help users migrate before the release.

[waichee]

@jinzhu just checking if there's an estimate date for when v2 will be available?

[ilovesusu]

是否会在v2中添加读写分离的支持

[sjmtan]

Regarding drivers + dialects - I assume v2 will be as interoperable between different drivers? I ask this because jackc/pgx appears to be the preferred replacement for lib/pq, and I was having a hard time migrating Copy In functionality over.

I then saw an issue in this repo slated for the v2 milestone that will support copy in. Would love to know if that's still slated for v2 since that was closed.

[Linzdigr]

@jinzhu Is there a support for generating and querying additionnal fields on join table (Many2Many relation) ?

[birdycn]

@jinzhu是否支持在联接表（Many2Many关系）上生成和查询附加字段？

I'm looking forward to this feature.

Example:manytomany need a tenant_id.

[Jtaylorapps]

@Linzdigr I'm attempting to do the same right now in v1. So far it looks like it might work as long as you create the join table yourself before you create the tables involved in the M2M relationship. Either way, I haven't confirmed it's working yet but it would be excellent to hear from @jinzhu on this. Sometimes junction tables are the only place a piece of data could reasonably be placed.

EDIT: Go-RM does already support this see https://github.com/jinzhu/gorm/issues/719

[rkuska]

Would it be possible to change the logger interface to every method accepting a context.Context? Many people use context to carry (for example) a request bound parameters, for example RequestID. This way we could extract these fields from context during logging.

[linbaozhong]

db.Create(&user) 并没有排除零值字段呀!

[jinzhu]

@rkuska added that, thank you for your suggestion.

[ks129]

Any news about the release date?