Big-Life-Lab / PHES-ODM

The Public Health Environmental Surveillance Open Data Model (PHES-ODM, or ODM). A data model, dictionary and support tools for environmental surveillance.
Creative Commons Attribution Share Alike 4.0 International
54 stars 18 forks source link

Sharing implementation of function in code for filtering rows #191

Closed GauriSaran closed 2 years ago

GauriSaran commented 3 years ago

the code consists of following parts:

  1. check the rules that apply to the organization user requested
  2. Iterate through each organization rule (create_dataset.py file has main function for that)
  3. Iterate through each table in rule to check (loop_through_tables.py)
  4. create list of variables of different datatypes in current rule (create_list_of_datatypes_variables.py)
  5. Iterate through each ruleValue present (checks if it is of range type or single value) (loop_through_rules.py)
  6. Checks the datatype of ruleValue as well to check if it is of datetime or not using regular expression. (filter_by_range_datetime_vars.py and filter_by_range_numeric_vars.py)
  7. Iterate through each variable in the list of specific datatype found to check if the ruleValue exists in each of those variables. (filter_by_range_datetime_vars.py and filter_by_range_numeric_vars.py and filter_single_value.py)
  8. Filters the rows.
  9. Returns the summary of the current data with rows removed. (in the final create_dataset.py file)
GauriSaran commented 3 years ago

Modified code files to format. They are now shared on this branch.