Fix ecds extract - Githubissues

The-Strategy-Unit / nhp_data

Databricks scripts for processing data required for the New Hospitals Programe model

https://connect.strategyunitwm.nhs.uk/nhp/project_information/

MIT License

0 stars 0 forks source link

Fix ecds extract #5

Closed tomjemmett closed 1 month ago

tomjemmett commented 1 month ago

filter out rows that are not Male/Female
make sure null values don't appear in the is_ columns (default to False)
remove rows that are marked as deleted

StatsRhian commented 1 month ago

filter out rows that are not Male/Female

I'm assuming we've had a discussion at some point about this?

tomjemmett commented 1 month ago

filter out rows that are not Male/Female

I'm assuming we've had a discussion at some point about this?

yes, it's specifically to do with the demographic modelling. That needs to have "valid" values for M/F, and age. so we have to exclude values that can't be joined to the demographic part of the model