openedx-unsupported / edx-analytics-pipeline

GNU Affero General Public License v3.0
91 stars 116 forks source link

fix: map mysql datatype to snowflake datatype. #883

Closed Hamza442 closed 1 year ago

Hamza442 commented 1 year ago

Job snowflake-lms-read-replica-import-from-s3 is failing due to following error: snowflake-lms-read-replica-import-from-s3: Unsupported data type 'JSON'. Upon investigation found that discussions_discussiontopiclink has a column context with datatype json, this data type is not supported in snowflake. The data type for json in snowflake is object, snowflake documentation : https://docs.snowflake.com/en/sql-reference/data-types-semistructured.html#object

Original Error: https://jenkins.analytics.edx.org/job/snowflake-lms-read-replica-import-from-s3/802/consoleFull Make sure that the following steps are done before merging:

codecov[bot] commented 1 year ago

Codecov Report

Base: 76.30% // Head: 76.29% // Decreases project coverage by -0.00% :warning:

Coverage data is based on head (05baf01) compared to base (35123ac). Patch coverage: 0.00% of modified lines in pull request are covered.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #883 +/- ## ========================================== - Coverage 76.30% 76.29% -0.01% ========================================== Files 150 150 Lines 17115 17117 +2 ========================================== Hits 13059 13059 - Misses 4056 4058 +2 ``` | [Impacted Files](https://codecov.io/gh/openedx/edx-analytics-pipeline/pull/883?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openedx) | Coverage Δ | | |---|---|---| | [...asks/warehouse/load\_internal\_reporting\_database.py](https://codecov.io/gh/openedx/edx-analytics-pipeline/pull/883/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openedx#diff-ZWR4L2FuYWx5dGljcy90YXNrcy93YXJlaG91c2UvbG9hZF9pbnRlcm5hbF9yZXBvcnRpbmdfZGF0YWJhc2UucHk=) | `37.29% <0.00%> (-0.41%)` | :arrow_down: | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openedx). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openedx)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.