HDRUK / CaRROT-CDM

MIT License
6 stars 1 forks source link

Log unique people #141

Open calmacx opened 2 years ago

calmacx commented 2 years ago

Request from @erummas to add a useful feature to print unique numbers.

For each concept, e.g.:

- CommonDataModel - INFO - finished Hypertensive disorder 3050 (0x122ea8280) ... 4/4 completed, 12336 rows

-->

- CommonDataModel - INFO - finished Hypertensive disorder 3050 (0x122ea8280) ... 4/4 completed, 12336 rows
- CommonDataModel - INFO - finished Hypertensive disorder 3050 (0x122ea8280) ...  12336 rows, from XXX unique people

Alternatively... in logging we have something like..

- CommonDataModel - INFO - {
      "version": "0.0.0",
      "created_by": "calummacdonald",
      "created_at": "2022-05-18T091400",
      "dataset": "CommonDataModel::FAILED: ExampleV4",
      "total_data_processed": {
            "person": 99471,
            "observation": 97817,
            "condition_occurrence": 101063,
            "measurement": 86795
      }
}

This could say:

 "total_data_processed": {
            "person": 99471,
            "observation": 97817,
            "condition_occurrence": 101063,
            "measurement": 86795
      },
"unique": {
            "observation":X,
            "condition_occurrence": Y,
            "measurement":Z
      }