marcboeker / go-duckdb

go-duckdb provides a database/sql driver for the DuckDB database engine.
MIT License
646 stars 97 forks source link

Best practices when using Appender API #258

Open charlespnh opened 1 month ago

charlespnh commented 1 month ago

I have an use case to handle traffic data and since it'll be write-heavy the Appender API is suitable for this. The traffic data will be parsed and inserted into 3 tables, so I'll have to create 3 appenders. I have a few questions regarding the best practices of using the Appender API in my particular cases:

  1. Should each of this appender be created with a different connection, or can they share the same connection?
  2. I'm storing these appenders as part of the application's lifetime, is this what I should be doing? Or is the appender only supposed to be short-lived in a function and the connection should be released back to db managed connection pool? It's a write-heavy use case so that's why I chose to go with the former, and the latter might incur overhead with creating a new connection and appender every time new traffic data arrives...

Thanks!