tehnorm
commented
2 years ago Also seeing this issue as well. Syncing from Smart Sheet to Postgres
Smart Sheet Row
| Job No | Name | Estimator | Entry Completed | Take Off Excel | CS Sheet Excel | CS Sheet Sub Sheet Name | Multiple Rows | Aggregate Project | Non-standard takeoff? | Take Off Type | Region | Specific Bldg Type | Structural Eng | Architect | Unit Count | Gross SQFT | Porch SQFT | Below Podium Wall Height | L1 BC Wall Hgt | L2 BC Wall Hgt | L3 BC Wall Hgt | L4 BC Wall Hgt | L5 BC Wall Hgt | Tall @ Parallel | GSF Walls below Podium | GSF L1 Walls | GSF L2 Walls | GSF L3 Walls | GSF L4 Walls | GSF L5 Walls | GSF L1 Floors | GSF L2 Floors | GSF L3 Floors | GSF L4 Floors | GSF L5 Floors | Roof GSF | Labor Incld? | Walls Incld? | Panels? | Floors Incld? | Roof Incld? | Hardware Incld? | Win & Doors Incld? | Building Wrap Incld? |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GA20-0440-0 | Modera | Name | Yes | https://box.com/s/ | https://app.box.com/file/ | 2-1-21 Final Sale | Full | South East | Type III – Apartments - 5 Levels | Ellinwood & Machado | Cooper Carry | 194 | 282046 | 11390 | 0 | 9 | 9 | 9 | 9 | 11 | 0 | 45800 | 53146 | 57818 | 57805 | 56087 | 0 | 45293 | 52821 | 57194 | 57189 | 57231 | TRUE | TRUE | TRUE | TRUE | TRUE | TRUE | TRUE |
_airbyte_data payload
{
"Name": "GA20-0440-0",
"Job No": "Modera",
"Region": "name@test.com",
"Panels?": "Yes",
"Roof GSF": "https://box.com/s/",
"Architect": "https://app.box.com/file/",
"Estimator": "2-1-21 Final Sale",
"Gross SQFT": "",
"Porch SQFT": "",
"Unit Count": "",
"Roof Incld?": "Full",
"GSF L1 Walls": "South East",
"GSF L2 Walls": "Type III – Apartments - 5 Levels",
"GSF L3 Walls": "Ellinwood & Machado",
"GSF L4 Walls": "Cooper Carry",
"GSF L5 Walls": 194,
"Labor Incld?": 282046,
"Walls Incld?": 11390,
"Floors Incld?": 0,
"GSF L1 Floors": 9,
"GSF L2 Floors": 9,
"GSF L3 Floors": 9,
"GSF L4 Floors": 9,
"GSF L5 Floors": 11,
"Multiple Rows": "",
"Take Off Type": 0,
"CS Sheet Excel": 45800,
"L1 BC Wall Hgt": 53146,
"L2 BC Wall Hgt": 57818,
"L3 BC Wall Hgt": 57805,
"L4 BC Wall Hgt": 56087,
"L5 BC Wall Hgt": 0,
"Structural Eng": 45293,
"Take Off Excel": 52821,
"Entry Completed": 57194,
"Hardware Incld?": 57189,
"Tall @ Parallel": 57231,
"Aggregate Project": true,
"Specific Bldg Type": true,
"Win & Doors Incld?": "",
"Building Wrap Incld?": true,
"Non-standard takeoff?": true,
"GSF Walls below Podium": true,
"CS Sheet Sub Sheet Name": true,
"Below Podium Wall Height": true
}
w-gitops
zzstoatzz
Madhukarsprabhakara
octavia-squidington-iii
LOG
``` 2021-11-18 15:16:45 INFO () WorkerRun(call):47 - Executing worker wrapper. Airbyte version: 0.30.20-alpha 2021-11-18 15:16:45 INFO () TemporalAttemptExecution(get):94 - Executing worker wrapper. Airbyte version: 0.30.20-alpha 2021-11-18 15:16:45 WARN () Databases(createPostgresDatabaseWithRetry):38 - Waiting for database to become available... 2021-11-18 15:16:45 INFO () JobsDatabaseInstance(lambda$static$2):25 - Testing if jobs database is ready... 2021-11-18 15:16:45 INFO () Databases(createPostgresDatabaseWithRetry):55 - Database available! 2021-11-18 15:16:45 INFO () DefaultReplicationWorker(run):82 - start sync worker. job id: 101 attempt id: 2 2021-11-18 15:16:45 INFO () DefaultReplicationWorker(run):91 - configured sync modes: {null.Triton 705 | 2021-W42 Cable Length Problems=full_refresh - overwrite} 2021-11-18 15:16:45 INFO () DefaultAirbyteDestination(start):58 - Running destination... 2021-11-18 15:16:45 INFO () LineGobbler(voidCall):65 - Checking if airbyte/destination-postgres:0.3.12 exists... 2021-11-18 15:16:45 INFO () LineGobbler(voidCall):65 - airbyte/destination-postgres:0.3.12 was found locally. 2021-11-18 15:16:45 INFO () DockerProcessFactory(create):127 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/101/2 --network host --log-driver none airbyte/destination-postgres:0.3.12 write --config destination_config.json --catalog destination_catalog.json 2021-11-18 15:16:45 INFO () LineGobbler(voidCall):65 - Checking if airbyte/source-smartsheets:0.1.6 exists... 2021-11-18 15:16:45 INFO () LineGobbler(voidCall):65 - airbyte/source-smartsheets:0.1.6 was found locally. 2021-11-18 15:16:45 INFO () DockerProcessFactory(create):127 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/101/2 --network host --log-driver none airbyte/source-smartsheets:0.1.6 read --config source_config.json --catalog source_catalog.json --state input_state.json 2021-11-18 15:16:45 INFO () DefaultReplicationWorker(run):119 - Waiting for source thread to join. 2021-11-18 15:16:45 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):190 - Replication thread started. 2021-11-18 15:16:45 INFO () DefaultReplicationWorker(lambda$getDestinationOutputRunnable$3):226 - Destination output thread started. 2021-11-18 15:16:46 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:46 [32mINFO[m i.a.i.d.p.PostgresDestination(main):69 - {} - starting destination: class io.airbyte.integrations.destination.postgres.PostgresDestination 2021-11-18 15:16:46 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:46 [32mINFO[m i.a.i.b.IntegrationRunner(run):76 - {} - Running integration: io.airbyte.integrations.base.ssh.SshWrappedDestination 2021-11-18 15:16:46 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:46 [32mINFO[m i.a.i.b.IntegrationCliParser(parseOptions):118 - {} - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json} 2021-11-18 15:16:46 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:46 [32mINFO[m i.a.i.b.IntegrationRunner(run):80 - {} - Command: WRITE 2021-11-18 15:16:46 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:46 [32mINFO[m i.a.i.b.IntegrationRunner(run):81 - {} - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'} 2021-11-18 15:16:46 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:46 [33mWARN[m c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2021-11-18 15:16:46 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:46 [33mWARN[m c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2021-11-18 15:16:46 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:46 [33mWARN[m c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2021-11-18 15:16:46 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:46 [33mWARN[m c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword multiline - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2021-11-18 15:16:46 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:46 [32mINFO[m i.a.i.b.s.SshTunnel(getInstance):170 - {} - Starting connection with method: NO_TUNNEL 2021-11-18 15:16:46 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:46 [32mINFO[m i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):96 - {} - Write config: WriteConfig{streamName=Triton 705 | 2021-W42 Cable Length Problems, namespace=ss_raw, outputSchemaName=ss_raw, tmpTableName=_airbyte_tmp_dgo_Triton_705___2021_W42_Cable_Length_Problems, outputTableName=_airbyte_raw_Triton_705___2021_W42_Cable_Length_Problems, syncMode=overwrite} 2021-11-18 15:16:46 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:46 [32mINFO[m i.a.i.d.b.BufferedStreamConsumer(startTracked):124 - {} - class io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer started. 2021-11-18 15:16:46 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:46 [32mINFO[m i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):121 - {} - Preparing tmp tables in destination started for 1 streams 2021-11-18 15:16:46 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:46 [32mINFO[m i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):125 - {} - Preparing tmp table in destination started for stream Triton 705 | 2021-W42 Cable Length Problems. schema: ss_raw, tmp table name: _airbyte_tmp_dgo_Triton_705___2021_W42_Cable_Length_Problems 2021-11-18 15:16:48 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:48 [32mINFO[m i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):131 - {} - Preparing tables in destination completed. 2021-11-18 15:16:54 INFO () DefaultReplicationWorker(run):121 - Source thread complete. 2021-11-18 15:16:54 INFO () DefaultReplicationWorker(run):122 - Waiting for destination thread to join. 2021-11-18 15:16:54 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:54 [32mINFO[m i.a.i.b.FailureTrackingAirbyteMessageConsumer(close):60 - {} - Airbyte message consumer: succeeded. 2021-11-18 15:16:54 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:54 [32mINFO[m i.a.i.d.b.BufferedStreamConsumer(close):199 - {} - executing on success close procedure. 2021-11-18 15:16:55 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:55 [32mINFO[m i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onCloseFunction$3):160 - {} - Finalizing tables in destination started for 1 streams 2021-11-18 15:16:55 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:55 [32mINFO[m i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onCloseFunction$3):165 - {} - Finalizing stream Triton 705 | 2021-W42 Cable Length Problems. schema ss_raw, tmp table _airbyte_tmp_dgo_Triton_705___2021_W42_Cable_Length_Problems, final table _airbyte_raw_Triton_705___2021_W42_Cable_Length_Problems 2021-11-18 15:16:56 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:56 [32mINFO[m i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onCloseFunction$3):178 - {} - Executing finalization of tables. 2021-11-18 15:16:56 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:56 [32mINFO[m i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onCloseFunction$3):180 - {} - Finalizing tables in destination completed. 2021-11-18 15:16:56 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:56 [32mINFO[m i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onCloseFunction$3):183 - {} - Cleaning tmp tables in destination started for 1 streams 2021-11-18 15:16:56 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:56 [32mINFO[m i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onCloseFunction$3):187 - {} - Cleaning tmp table in destination started for stream Triton 705 | 2021-W42 Cable Length Problems. schema ss_raw, tmp table name: _airbyte_tmp_dgo_Triton_705___2021_W42_Cable_Length_Problems 2021-11-18 15:16:56 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:56 [32mINFO[m i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onCloseFunction$3):192 - {} - Cleaning tmp tables in destination completed. 2021-11-18 15:16:56 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:56 [32mINFO[m i.a.i.b.IntegrationRunner(run):133 - {} - Completed integration: io.airbyte.integrations.base.ssh.SshWrappedDestination 2021-11-18 15:16:56 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-11-18 15:16:56 [32mINFO[m i.a.i.d.p.PostgresDestination(main):71 - {} - completed destination: class io.airbyte.integrations.destination.postgres.PostgresDestination 2021-11-18 15:16:56 INFO () DefaultReplicationWorker(run):124 - Destination thread complete. 2021-11-18 15:16:56 INFO () DefaultReplicationWorker(run):152 - sync summary: io.airbyte.config.ReplicationAttemptSummary@67852ab8[status=completed,recordsSynced=478,bytesSynced=1406131,startTime=1637248605255,endTime=1637248616466] 2021-11-18 15:16:56 INFO () DefaultReplicationWorker(run):161 - Source did not output any state messages 2021-11-18 15:16:56 WARN () DefaultReplicationWorker(run):169 - State capture: No new state, falling back on input state: io.airbyte.config.State@261bd7c7[state={}] 2021-11-18 15:16:56 INFO () TemporalAttemptExecution(get):115 - Stopping cancellation check scheduling... 2021-11-18 15:16:56 INFO () SyncWorkflow$ReplicationActivityImpl(replicate):178 - sync summary: io.airbyte.config.StandardSyncOutput@127c1349[standardSyncSummary=io.airbyte.config.StandardSyncSummary@12d7895e[status=completed,recordsSynced=478,bytesSynced=1406131,startTime=1637248605255,endTime=1637248616466],state=io.airbyte.config.State@261bd7c7[state={}],outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@cf7af28[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@64a65c1a[stream=io.airbyte.protocol.models.AirbyteStream@1632ed9c[name=Triton 705 | 2021-W42 Cable Length Problems,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"#":{"type":"string"},"FU":{"type":"string"},"BGR":{"type":"string"},"POS":{"type":"string"},"RPS":{"type":"string"},"SRTP":{"type":"string"},"EndEU":{"type":"string"},"Eq DK":{"type":"string"},"ItemID":{"type":"string"},"Status":{"type":"string"},"is_12G":{"type":"string"},"is_dmx":{"type":"string"},"End EEL":{"type":"string"},"EndDeck":{"type":"string"},"EndRoom":{"type":"string"},"StartEU":{"type":"string"},"Column58":{"type":"string"},"EndBlock":{"type":"string"},"Equip FZ":{"type":"string"},"Resolved":{"type":"string"},"AreaLabel":{"type":"string"},"CableType":{"type":"string"},"Highlight":{"type":"string"},"OldStatus":{"type":"string"},"Start EEL":{"type":"string"},"StartDeck":{"type":"string"},"StartRoom":{"type":"string"},"Lock Owner":{"type":"string"},"StartBlock":{"type":"string"},"CableLength":{"type":"string"},"Description":{"type":"string"},"End is Rack":{"type":"string"},"EndFirezone":{"type":"string"},"EvolutionID":{"type":"string"},"WTIRelevant":{"type":"string"},"end_is_rack":{"type":"string"},"venue_4code":{"type":"string"},"7500 End JLM":{"type":"string"},"8100 End JLM":{"type":"string"},"Display name":{"type":"string"},"EndEquipment":{"type":"string"},"EndFromBelow":{"type":"string"},"Reviewed HMC":{"type":"string"},"plant_number":{"type":"string"},"8100 Ticketed":{"type":"string"},"CableLengthWD":{"type":"string"},"Creation date":{"type":"string"},"DistanceCable":{"type":"string"},"EndEntryPoint":{"type":"string"},"Flag for 7500":{"type":"string"},"Flag for 8100":{"type":"string"},"Start is Rack":{"type":"string"},"StartFirezone":{"type":"string"},"start_is_rack":{"type":"string"},"7500 Start JLM":{"type":"string"},"8100 Start JLM":{"type":"string"},"BGRResponsible":{"type":"string"},"End JLM (Dash)":{"type":"string"},"EndCompartment":{"type":"string"},"EndExtraLength":{"type":"string"},"Instance Title":{"type":"string"},"Mailed to Yard":{"type":"string"},"Probably Fixed":{"type":"string"},"StartEquipment":{"type":"string"},"StartFromBelow":{"type":"string"},"7500 Cable Type":{"type":"string"},"Called Out 0100":{"type":"string"},"MarkPointLength":{"type":"string"},"ProductionStage":{"type":"string"},"SRTPDescription":{"type":"string"},"StartEntryPoint":{"type":"string"},"DrawingRelevance":{"type":"string"},"Not Easily Fixed":{"type":"string"},"Override End TLA":{"type":"string"},"Rack to Rack Tie":{"type":"string"},"Start JLM (Dash)":{"type":"string"},"StartCompartment":{"type":"string"},"StartExtraLength":{"type":"string"},"CableDescription1":{"type":"string"},"CableDescription2":{"type":"string"},"CableDescription3":{"type":"string"},"Description Error":{"type":"string"},"EndExtraLengthEQT":{"type":"string"},"Last modification":{"type":"string"},"MarkPointCableway":{"type":"string"},"Not a 7500 Change":{"type":"string"},"RPSRedundantCable":{"type":"string"},"Rev Report | Date":{"type":"string","format":"date"},"7500 Description 1":{"type":"string"},"7500 Description 2":{"type":"string"},"8100 Description 1":{"type":"string"},"8100 Description 2":{"type":"string"},"8100 Description 3":{"type":"string"},"Calculated End TLA":{"type":"string"},"Override Start TLA":{"type":"string"},"Revised Cable Type":{"type":"string"},"SRTPRedundantCable":{"type":"string"},"Has Fiber Available":{"type":"string"},"StartExtraLengthEQT":{"type":"string"},"Yard Problem Status":{"type":"string"},"Calculated Start TLA":{"type":"string"},"EndEquipmentPosition":{"type":"string"},"MarkPointDescription":{"type":"string"},"HMC Internal Comments":{"type":"string"},"IgnoreStatusTransition":{"type":"string"},"Revised End JLM (Dash)":{"type":"string"},"StartEquipmentPosition":{"type":"string"},"Yard Flipped Start-End":{"type":"string"},"Mail Resolution to Yard":{"type":"string"},"8100 Adjustment Complete":{"type":"string"},"HMC Suggested Resolution":{"type":"string"},"Revised Start JLM (Dash)":{"type":"string"},"Revised HMC End Equipment":{"type":"string"},"Rev Stats | Type of Change":{"type":"string"},"Revised HMC StartEquipment":{"type":"string"}}},supportedSyncModes=[full_refresh],sourceDefinedCursor=