HelenCEBM
commented
2 years ago There is just one notebook for this particular task but it was developed in the server. The latest version is here
- [ ] I need to release the code as a txt file in order to recreate it outside of the server.
The notebook is run in line with these instructions once this PR is merged. Briefly:
- Clone repo into
e:/<yourname> - Open git Bash and
cdto the repo directory - Run
/e/bin/database-notebook.sh - A local path for running Jupyterlabs should be created - paste into Chrome
- Open the notebook and run/edit as appropriate
- Export as html and copy to workspace folder in
e:/FILESFORL4/workspaces/raw-data-plausibility-checkingfor syncing to L4. If the sync to L4 doesn't happen, check with Simon. On L4, releasing from this folder isn't working but it works fromworkspaces/raw-data-plausibility-checks-devso it seems it's currently necessary to copy files to there. - A diffable python version of the notebook should also be created in the subfolder - if you have made changes to the code, save this as txt for release as well.
- Shut down Jupyterlabs when finished to close the container (check via
docker ps)
However, it currently still requires someone to discover that the name has changed before we run it. There are several possible solutions e.g.:
- make this notebook automatable / a reusable action etc for easier running
- ideally make a simple csv output of vaccine names from this notebook that can be automatically released when such a process is in place
- create a script that runs routinely to detect any changes in vaccine names and alert us before it breaks someone's study
- make the whole vaccine reference table available in L4 so users can more easily self-serve vaccine names for any target disease (not just COVID vaccines)
- Use the four-digit TPP vaccine codes instead of names (but we would need to make the ref table easily accessible to make these meaningful)
- Use SNOMED codes
milanwiedemann
iaindillingham
Vaccine names change. However, researchers only discover this when their studies return unexpected results (see this Slack thread, for example). @HelenCEBM has several notebooks that report the most recent vaccine names. We should learn how to run the notebooks, and how to report the most recent vaccine names.
The notebooks are within
/e/helen/raw-data-plausibility-checks(L3), which is a clone of opensafely/raw-data-plausibility-checks. The HTML versions of the notebooks are within:/srv/medium_privacy/workspaces/raw-data-plausibility-checking(L3 VM)/d/Level4Files/workspaces/raw-data-plausibility-checking(L4)There is a separate issue for replacing vaccine names with snomed codes.