Transform release_date attributes in netflix and imdb to year

Humorloos / IE683

0 stars 0 forks source link

Transform release_date attributes in netflix and imdb to year #50

Closed Humorloos closed 3 years ago

Humorloos commented 3 years ago

in streaming dataset there is only year, but this is basically the same and release dates are matched by year anyways so this would make sense

rename year to releaseYear

ashishrana160796 commented 3 years ago

So, as development work: do you want me to transform the schema mapping script & create new xml files? I can do that before EOD or may be even early. Thanks, for pointing out this one!

ashishrana160796 commented 3 years ago

I am also thinking that this problem can be resolved at data fusion stage instead of schema mapping one. Reason being release_date should description wise mean a complete date not specific year of release even if functionally the later is more acceptable in this use-case.

What say, do you guys want to transform it beforehand or later?

ashishrana160796 commented 3 years ago

Transforming the release date field to release year, plus have removed year field from the integrated schema in itself.
Will be mapping the fields accordingly in the datasets.
I am currently also not seeing title field being mapped to anything in IMDb dataset. So, yea!

ashishrana160796 commented 3 years ago

PR #53 & #54, completely removes this issue.