Establish a clear sign off procedure for co-variates

CarolineMorton commented 4 years ago

We need to establish a way of signing off as finished covariates. The covariates are currently within git issues (and associated google doc) with code lists attached.

My ideas would be: LABEL: signed off for covariates which have been through sign off Clinical Sign off: LSHTM usually have at least one clinician sign off code lists in my experience although we can check that with them. That would be Amir, Brian, Henry and me probably. Epidemiology Sign off: Another person to sign off on the code list Final sign off should include a comment on:

Definition
potential biases
Who has signed it off

CarolineMorton commented 4 years ago

We could structure sign off template:

DEFINITION:

POTENTIAL BIASES

CLINICAL SIGN OFF & DATE:

EPIDEMIOLOGY SIGN OFF & DATE

FINAL SIGN OFF DATE (and apply label)

alexwalkerepi commented 4 years ago

I agree that at least one clinician and epidemiology person to sign off. Regarding the definition, I think we should have a firm version of it ready before sign off starts, perhaps including:

Any code lists
A description of the logic to be used to define them (e.g. date ranges)
Any possible flexibility that might be needed between different studies (e.g. for study 1 we want to look at codes in the whole record, study 2, just the last 3 months
Any repercussions that the co-variate definition has on cohort selection (e.g. patients must have registered for the whole 3 months where we're looking for codes).

For final sign off, do we need the exact code to be used, or just a narrative description still?

CarolineMorton commented 4 years ago

Agreed. Let's ask @evansd or @sebbacon re the code or narrative description. I think it will probably be the narrative description but good to check!

sebbacon commented 4 years ago

This sounds great. I like the template idea. There's even some way of defining issue templates in github (though now we've already made a load of issues I guess that's moot - unless we want to create a new issue referencing other discussions in @evansd 's new repo?)

The kind of detail Alex is talking about is going to be hard to capture in every case and will require some hard thinking for everyone. For example Dave and I were just discussing the proposed "a quantity of at least one in three consecutive quarters" measurement, which begs the question: what do we mean by "quarter"? Is it Jan-Mar, April-Jun etc, or is it any consecutive 3 month period?!

We don't need code, just a description. The signed off version can the become the documentation for the actual code and could be referenced in the tests. We will need a code review step for the actual code which will serve as the "sign off" of that bit - might be worth not only an engineer but also a clinical/coder person doing that code review.

CarolineMorton commented 4 years ago

Sounds like we are all on the same page then. I can make a label for "definition sign off" and we could have a "code sign off" at some point. I am very happy to be part of the code review process, and @alexwalkercebm would be good too, as we will need SQL people. @HelenCEBM also?

alexwalkerepi commented 4 years ago

As well as a formal sign off, is it worth also posting in Slack and waiting an hour or two, so that a wider group of people can take a look if interested?

CarolineMorton commented 4 years ago

Yes that sounds like a great idea.

Are we still happy with this being the template?

DEFINITION:

POTENTIAL BIASES

CLINICAL SIGN OFF & DATE:

EPIDEMIOLOGY SIGN OFF & DATE

HAVE YOU POSTED TO WIDER TEAM FOR COMMENTS? (Yes/No)

FINAL SIGN OFF DATE (and apply label)

sebbacon commented 4 years ago

Yes, perhaps with Alex's suggestions for detailed info as subpoints under DEFINITION?

CarolineMorton commented 4 years ago

brill - let's do that.

alexwalkerepi commented 4 years ago

See here for first example - signing off age https://github.com/ebmdatalab/tpp-sql-notebook/issues/33#issuecomment-607163668

CarolineMorton commented 4 years ago

TEMPLATE

DEFINITION:

CODE LISTS: None
FLEXIBILITY NEEDED BETWEEN STUDIES:
EFFECTS ON COHORT SELECTION: POTENTIAL BIASES:

CLINICAL SIGN OFF & DATE:

EPIDEMIOLOGY SIGN OFF & DATE:

SHARED WITH WIDER TEAM: Yes/No

FINAL SIGN OFF DATE (and apply label)

CarolineMorton commented 4 years ago

@alexwalkercebm To incorporate what we discussed today, for sign off of code list generated from Read v2, we need to ensure the definition includes, points mentioned here https://github.com/ebmdatalab/tpp-sql-notebook/issues/59#issuecomment-608384996

1) Read code list 2) QOF cluster codes included 3) Snowmed terms, and key words

I will try this for #7 now

opensafely / tpp-sql-notebook

Establish a clear sign off procedure for co-variates #45