Legacy MM: Audit views of erroneous program letters

mitodl / micromasters

Portal for learners and course teams to access MITx Micromasters® programs

https://mm.mit.edu

BSD 3-Clause "New" or "Revised" License

28 stars 16 forks source link

Legacy MM: Audit views of erroneous program letters #5251

Closed pdpinch closed 1 year ago

pdpinch commented 1 year ago

The course team would like to know a few things about the erroneous program letters:

When were they available for viewing? It seems like the records were created well before the letter was added to the CMS.
Were they viewed? We capture pages views in Google Analytics

part of #5248

Acceptance Criteria:

[x] Confirm that letters cannot be viewed when the letter template does not exist in the CMS
[x] Identify when the letter template for program id=5 was created.
[ ] Compare GA page view data to the list of erroneous program letters

Relates issues

RCA erroneous program letters created on 12/19/2019
RCA erroneous program letters created after 12/19/2019

pdpinch commented 1 year ago

Looking at the wagtail history UI, I believe the program letter template for program 5 was created sometime in February 2022. Can we confirm that by looking through the wagtail version history?

pdpinch commented 1 year ago

You can contact me for a dump of the GA page view data. @arslanashraf7 and I have a list of the erroneous program letters.

arslanashraf7 commented 1 year ago

Updates:

Can we confirm that by looking through the wagtail version history?

Regarding this, It looks like the letter template was added in https://micromasters.mit.edu/cms/pages/55/revisions/compare/3966...3971/ Which was drafted on 14/02/2022 and was published around 16/02/2022.

I was also tracking things back to when more than 1000 letters were created and I saw this release but nothing in that release looks like a culprit. Maybe we ran the Final grades creation command that could have created those letters? But I will need to dig some more to be sure.

Expect more updates on this.

arslanashraf7 commented 1 year ago

When were they available for viewing? It seems like the records were created well before the letter was added to the CMS.

They should have been available for viewing when the template was added for the program which is Feb 16, 2022.

Were they viewed? We capture pages views in Google Analytics

Can I get the Analytics data to check how many were viewed?

pdpinch commented 1 year ago

Can I get the Analytics data to check how many were viewed?

I dumped this from Google Analytics. I can give you access to the analytics account if you think that would be more helpful.