zio / zio-quill

Compile-time Language Integrated Queries for Scala
https://zio.dev/zio-quill
Apache License 2.0
2.15k stars 346 forks source link

Batch insert using finagle-postgres is not working properly #1251

Open davoclavo opened 5 years ago

davoclavo commented 5 years ago

Version: quill-finagle-postgres:2.5.4

Module: quill-finagle-postgres

Database: postgres

Expected behavior

Similar to #747, we are getting the wrong behavior when doing batch inserts using finagle-postgres

Batch insert actions should be executed as a single query.

def insert(circles: List[Circle]) = quote {
  liftQuery(circles).foreach(c => query[Circle].insert(c))
}

ctx.run(insert(List(Circle(1.1F), Circle(1.2F)))) 
INSERT INTO Circle (radius) VALUES (1.1), (1.2)

Actual behavior

Multiple sequential insert queries.

INSERT INTO Circle (radius) VALUES (1.1)
INSERT INTO Circle (radius) VALUES (1.2)

Also, because the batchAction actually sends each insert individually and in parallel (not sequentially), IDs/timestamps of inserted items are not deterministically incremental.

Steps to reproduce the behavior

Execute a batch insert, such as the one in the Expected Behavior section.

Workaround

Writing insert queries using infix.

Will try to write a proper script to reproduce this issue, experiments using barebones finagle-postgres and hopefully a pull request to fix this issue.

Using finagle-postgres directly:

import com.twitter.finagle.Postgres

val client = Postgres.Client()
  .withCredentials("postgres", None)
  .database("test_db")
  .withSessionPool.maxSize(1)
  .withBinaryResults(true)
  .withBinaryParams(true)
  .newRichClient("localhost:5432")

val createTableQuery = """
  |CREATE TABLE test_table (
  | str_field VARCHAR(40),
  | int_field INT,
  | double_field DOUBLE PRECISION,
  | timestamp_field TIMESTAMP WITH TIME ZONE,
  | bool_field BOOLEAN
  |)
  """.stripMargin

client.executeUpdate(createTableQuery)

val insertQuery = """
  |INSERT INTO test_table VALUES
  | ('hello', 1234, 10.5, '2015-01-08 11:55:12-0800', TRUE),
  | ('hello', 5557, -4.51, '2015-01-08 12:55:12-0800', TRUE),
  | ('hello', 7787, -42.51, '2013-12-24 07:01:00-0800', FALSE),
  | ('goodbye', 4567, 15.8, '2015-01-09 16:55:12+0500', FALSE)
  """.stripMargin

client.executeUpdate(insertQuery)

@getquill/maintainers

NikitaMishin commented 4 years ago

same issue with quill-async for postgre.