Pooling service to support multiple cohorts

[nickdotreid]

Our goal is to support the pooling service being able to create hyper-parameters for two different cohorts. To do this, the pooling service will need knowledge of which cohort a participant belongs to. The pooling service will use this information to sort and process participants internally.

• [ ] HeartSteps server to send list of participants in pooling condition nightly. The HeartSteps Server will send a list of objects formatted in JSON, with the following keys: 'participant-id', 'cohort name', 'study name'.
• [ ] Pooling service will filter participants into two lists, one for Aim 2 participants, and a second for Aim 3 participants. It will be easiest to hard code these values.
• [ ] Pooling service will calculate hyper parameters for Aim 2 and Aim 3 participants.

[nickdotreid]

@Sabina321 Tell me how this outline of work sounds to you.

[Sabina321]

Hi Nick,

I will update this later today.

The pooling service shouldn't need to know what aim a participant is in. I think it will be much simpler.

The walking-suggestion service may need to know, but the pooling service I think will not.

On Mon, Dec 30, 2019 at 9:19 AM Nick Reid notifications@github.com wrote:

@Sabina321 https://github.com/Sabina321 Tell me how this outline of work sounds to you.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kpwhri/heartsteps/issues/119?email_source=notifications&email_token=AATDEQU7UP2ZLDJX42K4YKLQ3IUTJA5CNFSM4KBOQIO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH2YOKA#issuecomment-569739048, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATDEQT7ALGKQ45LGXROHU3Q3IUTJANCNFSM4KBOQIOQ .

[samurphy11]

Hi All, Sabina, we don't want to pool data across users in aim 2 and aim 3. I would have thought that you have to keep these users separate to avoid sharing data across them... Best, susan

On Mon, Dec 30, 2019 at 1:18 PM Sabina321 notifications@github.com wrote:

Hi Nick,

I will update this later today.

The pooling service shouldn't need to know what aim a participant is in. I think it will be much simpler.

The walking-suggestion service may need to know, but the pooling service I think will not.

On Mon, Dec 30, 2019 at 9:19 AM Nick Reid notifications@github.com wrote:

@Sabina321 https://github.com/Sabina321 Tell me how this outline of work sounds to you.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/kpwhri/heartsteps/issues/119?email_source=notifications&email_token=AATDEQU7UP2ZLDJX42K4YKLQ3IUTJA5CNFSM4KBOQIO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH2YOKA#issuecomment-569739048 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AATDEQT7ALGKQ45LGXROHU3Q3IUTJANCNFSM4KBOQIOQ

.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kpwhri/heartsteps/issues/119?email_source=notifications&email_token=ADTRRQLDY3NP3NWIVECZH7LQ3I3OZA5CNFSM4KBOQIO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH24E4Q#issuecomment-569754226, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADTRRQNU4LX22KWSYR6IGX3Q3I3OZANCNFSM4KBOQIOQ .

--

---

Susan A. Murphy Professor of Statistics, Radcliffe Alumnae Professor at the Radcliffe Institute and Professor of Computer Science at the Harvard John A. Paulson School of Engineering and Applied Sciences Harvard University website: http://people.seas.harvard.edu/~samurphy/ http://people.seas.harvard.edu/~samurphy/

---

[Sabina321]

Hi Susan,

Yes that is correct. But the way this is handled does not require arguments being passed in on my side. As far as I can tell this only creates work for Nick and serves no purpose for me.

I think Peng needs to know which aim people are in for his analysis, but the pooling service can be set up without requiring arguments.

Nick and I will discuss the least labor-intensive way to do this today.

On Mon, Dec 30, 2019 at 11:29 AM Susan A Murphy notifications@github.com wrote:

Hi All, Sabina, we don't want to pool data across users in aim 2 and aim 3. I would have thought that you have to keep these users separate to avoid sharing data across them... Best, susan

On Mon, Dec 30, 2019 at 1:18 PM Sabina321 notifications@github.com wrote:

Hi Nick,

I will update this later today.

The pooling service shouldn't need to know what aim a participant is in. I think it will be much simpler.

The walking-suggestion service may need to know, but the pooling service I think will not.

On Mon, Dec 30, 2019 at 9:19 AM Nick Reid notifications@github.com wrote:

@Sabina321 https://github.com/Sabina321 Tell me how this outline of work sounds to you.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <

https://github.com/kpwhri/heartsteps/issues/119?email_source=notifications&email_token=AATDEQU7UP2ZLDJX42K4YKLQ3IUTJA5CNFSM4KBOQIO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH2YOKA#issuecomment-569739048

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AATDEQT7ALGKQ45LGXROHU3Q3IUTJANCNFSM4KBOQIOQ

.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub < https://github.com/kpwhri/heartsteps/issues/119?email_source=notifications&email_token=ADTRRQLDY3NP3NWIVECZH7LQ3I3OZA5CNFSM4KBOQIO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH24E4Q#issuecomment-569754226 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ADTRRQNU4LX22KWSYR6IGX3Q3I3OZANCNFSM4KBOQIOQ

.

--

---

Susan A. Murphy Professor of Statistics, Radcliffe Alumnae Professor at the Radcliffe Institute and Professor of Computer Science at the Harvard John A. Paulson School of Engineering and Applied Sciences Harvard University website: http://people.seas.harvard.edu/~samurphy/ http://people.seas.harvard.edu/~samurphy/

---

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kpwhri/heartsteps/issues/119?email_source=notifications&email_token=AATDEQTMWYHA35O3HVD7BZ3Q3JD2JA5CNFSM4KBOQIO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH3AT5I#issuecomment-569772533, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATDEQRXLQ6KCFJG25ZRLS3Q3JD2JANCNFSM4KBOQIOQ .

[nickdotreid]

@Sabina321 I've completed implementing the changes we discussed -- so we should be ready for testing.

I have changed:

• Pooling service to run update.sh with only Aim 2 participant usernames that are sent via post request
• Pooling service to write join_dates.csv before update.sh, with Aim 3 usernames and walking suggestion service initialized dates (formatted month/day/year)
• I removed errorfile_pooled.txt and join_dates.csv from github, and set a .gitignore, because these are generated files and should not be saved. If you want to save these files, write them to the data directory.

The accepted POST request for the pooling service looks like this

{"participants": 
    [
        {
            "username": "test-nickreid",
            "study": "kpwhri",
            "cohort": "Aim 3",
            "start": "2012-12-12"
        },
        {
            "username": "test-test",
            "study": "kpwhri",
            "cohort": "Aim 3",
            "start": "2019-7-2"
        },
        {
            "username": "test-1234",
            "study": "kpwhri",
            "cohort": "Aim 3",
            "start": "2020-1-20"
        }
    ]
}