airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.13k stars 4.12k forks source link

Source netsuite - Current netsuite datetime format does not supported by netsuite Restlet APIs, New format: dd.mm.yy #21871

Open zilehuda opened 1 year ago

zilehuda commented 1 year ago
## Environment - **Airbyte version**: v0.40.28 - **OS Version / Instance**: macOS - **Source Connector and version**: netsuite - **Step where error happened**: When you run any stream it will get failed. I have tested it on the `customer` and `employee` stream ## Current Behavior Streams are getting failed. ## Expected Behavior Stream should import the data without any error. ## Logs

{"type": "LOG", "log": {"level": "INFO", "message": "Syncing stream: customer "}} {"type": "LOG", "log": {"level": "WARN", "message": "Stream customer: cannot read using date format %m/%d/%Y"}} {"type": "LOG", "log": {"level": "WARN", "message": "Streamcustomer: retry using next date format%Y-%m-%d"}} {"type": "STATE", "state": {"type": "STREAM", "stream": {"stream_descriptor": {"name": "customer"}, "stream_state": {}}, "data": {"customer": {}}}} {"type": "LOG", "log": {"level": "WARN", "message": "Stream customer: cannot read using date format %Y-%m-%d"}} {"type": "LOG", "log": {"level": "ERROR", "message": "DATE FORMAT exception. Cannot read using known formats ['%m/%d/%Y', '%Y-%m-%d']"}} {"type": "LOG", "log": {"level": "ERROR", "message": "Streamcustomer`: INVALID_PARAMETER error occured, full error message: Invalid search query. Detailed unprocessed description follows. Search error occurred: Parse of date/time \"2022-07-01\" failed with date format \"dd.MM.yy\" in time zone America/Los_Angeles\nCaused by:\n\tjava.text.ParseException: Unparseable date: \"2022-07-01\"."}}

Are you willing to submit a PR?

Yes, I have resolved it locally and opened the PR.

zilehuda commented 1 year ago

I have fixed this bug and I will open the PR.

zilehuda commented 1 year ago

Hi @natalyjazzviolin, please review the PR

bazarnov commented 1 year ago

@zilehuda Hello, could you please specify the date format for the customer object you are trying to fetch, the lastModifiedDate or lastModified fields to check, in particular? Is it really "dd.MM.yy", like 11.02.2023?

jsvgjsvg commented 1 year ago

Hello!

The dateformat in our setup is DD.MM.YYYY. Having the same issues with the Netsuite sync - can't sync customer object.