search

Daimler666 / Data-Lake-Ticket-Platform

1 stars 0 forks source link

SFDC data quality issue again #7

Closed Daimler666 closed 7 years ago

Daimler666 commented 7 years ago

/Raised by Felix Liu/

Status:

  1. The wrong new line characters from comments column appears again: datalake_salesforce. dw_cust_sfdc_case;
  2. The data is on using, need be revised ASAP.

Suggestion:

  1. Since someone mentioned that the raw tables have been changed to ORC, please check is any issues in that.
Daimler666 commented 7 years ago

/Replied by Felix Liu/

Issues Summary: The following three tables has the same problem and need someone to check it ASAP. -datalake_salesforce. dw_cust_sfdc_case -datalake_salesforce. dw_cust_sfdc_complaint

Daimler666 commented 7 years ago

/Replied by Madhu/

Status:

  1. Have checked through PROD server, no any issue appeared.

Solutions:

  1. We will truncated the existing tables and re-sqoop the full dump data to avoid such issues.
  2. Currently, the PROD server can be used with the right data and the INT server is ongoing checking.
Daimler666 commented 7 years ago

/Replied by Debdatta/

Status:

  1. After checking, Issue does not exit in Prod env (details can be seen from Madhu).

Suggestion:

  1. PROD env be refferred for all sources migrated.
Daimler666 commented 7 years ago

/Replied by Madhu/

Status:

  1. The issue is resolved now and please validate.
Daimler666 commented 7 years ago

/Closed by Derek/

Status:

After checking, the status of table looks fine now.