To run locally use: _python datatransfer/main.py TFA Newcastle. It's actually site agnostic so you only need to run it once and all sites are covered. The site input param needs to be valid but it doesn't matter if it's Newcastle or one of the other sites.
Note: when testing I normally comment out line 40 on tfa.py in dags to avoid files being uploaded to DMP.
I still have to implement an effective whitelist as around 14 files contain ThinkFAST data but no CANTAB data. I suspect a labelling issue is the cause which we can resolve.
To run locally use: _python datatransfer/main.py TFA Newcastle. It's actually site agnostic so you only need to run it once and all sites are covered. The site input param needs to be valid but it doesn't matter if it's Newcastle or one of the other sites.
Note: when testing I normally comment out line 40 on tfa.py in dags to avoid files being uploaded to DMP.
I still have to implement an effective whitelist as around 14 files contain ThinkFAST data but no CANTAB data. I suspect a labelling issue is the cause which we can resolve.