JuliaDatabases / ODBC.jl

An ODBC interface for the Julia programming language
https://odbc.juliadatabases.org/stable
Other
106 stars 63 forks source link

Insert only if Unique. #332

Open robertmkjeffrey opened 2 years ago

robertmkjeffrey commented 2 years ago

Hi all,

I am using ODBC.load to push updated data into a database. This unfortunately fails if the dataframe contains elements already present in the database.

It would be great is ODBC.load had a "skip existing" option which ignored rows for which the primary key already exists. This would likely require passing the unique column names to the function as well.

Example of the error:

ERROR: LoadError: 40002: [MonetDB][ODBC Driver 11.43.5]INSERT INTO: PRIMARY KEY constraint 'weather_history.weather_history_station_id_date_pkey' violated
Stacktrace:
 [1] error(s::String)
   @ Base .\error.jl:33
 [2] execute(stmt::ODBC.API.Handle)
   @ ODBC.API C:\Users\rjeffrey003\.julia\packages\ODBC\qhwMX\src\API.jl:114
 [3] execute(stmt::ODBC.Statement, params::Tables.Row{DataFrameRow{DataFrame, DataFrames.Index}}; debug::Bool, kw::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ ODBC C:\Users\rjeffrey003\.julia\packages\ODBC\qhwMX\src\dbinterface.jl:164
 [4] (::ODBC.var"#46#47"{Bool, Int64, ODBC.Connection})()
   @ ODBC C:\Users\rjeffrey003\.julia\packages\ODBC\qhwMX\src\load.jl:102
 [5] transaction(f::ODBC.var"#46#47"{Bool, Int64, ODBC.Connection}, conn::ODBC.Connection)
   @ ODBC C:\Users\rjeffrey003\.julia\packages\ODBC\qhwMX\src\load.jl:112
 [6] load(itr::DataFrame, conn::ODBC.Connection, name::String; append::Bool, quoteidentifiers::Bool, debug::Bool, limit::Int64, kw::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ ODBC C:\Users\rjeffrey003\.julia\packages\ODBC\qhwMX\src\load.jl:96
 [7] load(itr::DataFrame, conn::ODBC.Connection, name::String)
   @ ODBC C:\Users\rjeffrey003\.julia\packages\ODBC\qhwMX\src\load.jl:76
 [8] top-level scope
   @ c:\Users\rjeffrey003\Documents\Work\1 - Weather Data\main.jl:211
in expression starting at c:\Users\rjeffrey003\Documents\Work\1 - Weather Data\main.jl:211