dimitri / pgloader

Migrate to PostgreSQL in a single command!
http://pgloader.io
Other
5.45k stars 548 forks source link

default target column to current date time #1496

Open d3r3ktm opened 1 year ago

d3r3ktm commented 1 year ago

No

did a search but could not find any information on how to set the target column (timestamp not null) to current date time

pgloader -D /apps/edw/derek -L load.log csv.load

--
-- EDIT THIS FILE TO MATCH YOUR BUG REPORT
--

LOAD CSV
   FROM 'daily_mktg_source-2023-05-24.csv'
      HAVING FIELDS
      (
       traffic_date [date format 'YYYY-MM-DD HH24-MI-SS'],
       form_id,
       form_type,
       mktg_source
      )
   INTO postgresql://user:password@aws_host/edwdb
      TARGET TABLE derek_sda_mktg_source
      TARGET COLUMNS
      (
         traffic_date,
         form_id,
         form_type,
         mktg_source,
         refresh_date
      )
   WITH truncate,
        fields optionally enclosed by '"',
        fields terminated by ',',
        on error resume next;
2023-05-26T10:39:07.042000+08:00 LOG pgloader version "3.6.3~devel"
; in: LAMBDA (PGLOADER.SOURCES::ROW)
;     (VECTOR PGLOADER.USER-SYMBOLS::TRAFFIC_DATE PGLOADER.USER-SYMBOLS::FORM_ID
;             PGLOADER.USER-SYMBOLS::FORM_TYPE PGLOADER.USER-SYMBOLS::MKTG_SOURCE
;             PGLOADER.USER-SYMBOLS::REFRESH_DATE)
; --> MAKE-ARRAY MAKE-ARRAY LOCALLY
; ==>
;   (VECTOR PGLOADER.USER-SYMBOLS::TRAFFIC_DATE PGLOADER.USER-SYMBOLS::FORM_ID
;           PGLOADER.USER-SYMBOLS::FORM_TYPE PGLOADER.USER-SYMBOLS::MKTG_SOURCE
;           PGLOADER.USER-SYMBOLS::REFRESH_DATE)
;
; caught WARNING:
;   undefined variable: PGLOADER.USER-SYMBOLS::REFRESH_DATE
;
; compilation unit finished
;   Undefined variable:
;     PGLOADER.USER-SYMBOLS::REFRESH_DATE
;   caught 1 WARNING condition
2023-05-26T10:39:07.350000+08:00 ERROR Could not read input: The variable PGLOADER.USER-SYMBOLS::REFRESH_DATE is unbound.
2023-05-26T10:39:07.351000+08:00 ERROR Could not read input: The variable PGLOADER.USER-SYMBOLS::REFRESH_DATE is unbound.
2023-05-26T10:39:07.351000+08:00 ERROR Could not read input: The variable PGLOADER.USER-SYMBOLS::REFRESH_DATE is unbound.
2023-05-26T10:39:07.351000+08:00 ERROR Could not read input: The variable PGLOADER.USER-SYMBOLS::REFRESH_DATE is unbound.
2023-05-26T10:39:07.351000+08:00 ERROR Could not read input: The variable PGLOADER.USER-SYMBOLS::REFRESH_DATE is unbound.
2023-05-26T10:39:07.351000+08:00 LOG report summary reset
                      table name     errors       rows      bytes      total time
--------------------------------  ---------  ---------  ---------  --------------
                           fetch          0          0                     0.013s
--------------------------------  ---------  ---------  ---------  --------------
"public"."derek_sda_mktg_source"          5          0                     0.029s
--------------------------------  ---------  ---------  ---------  --------------
                 Files Processed          0          1                     0.019s
         COPY Threads Completion          0          2                     0.029s
--------------------------------  ---------  ---------  ---------  --------------
               Total import time          5          0                     0.048s
2023-05-24 00:00:00,1661249507289075712,DO,www.google.com
2023-05-24 00:00:00,8005001719712,INQ,1240043
2023-05-24 00:00:00,8005001719287,INQ,chat.bo-e.com
2023-05-24 00:00:00,8005001719039,INQ,1410219
2023-05-24 00:00:00,8005001722289,INQ,1001008

I am using pgloader for the first time, please pardon me. I am trying to load a csv file which has 4 columns into a table that has 5 columns. For the additional column (refresh_date) in the table, I want to load it with current date time. I have tried appending 'date' to it, 'refresh_date date' but there are syntax errors.

May I know what options should i use to load the additional column with current date time?