Add Archive status for experiments

[SritterCL]

We should add an "archived" status for experiments. Archived status can be reached from any status except enrolling (if an experiment is enrolling and needs to be stopped, it should go to cancelled status).

The semantics of archived status is described in the spec: https://docs.google.com/document/d/1X-DBcyz7nTwk_MLg8kYfWg1Uspb9bx0y9OEgi_T0tLI/edit#heading=h.1z8xhfd43vmo

Basically, "archived" status exists just as a historical record that an experiment existed, but nothing at runtime can reference the experiment.

[amurphy-cl]

@SritterCL @nirmalpatel this feature will become more valuable as we continue to create experiments and want to refer back without deleting them. Pinging to think about prioritizing this feature - marking it low for now, but will be helpful to have moving forward.

[amurphy-cl]

Adding some additional details here.

As a user of UpGrade, I want to be able to be able to view a history of experiments that have been run but are now completed or otherwise no longer supported by UpGrade. "No longer supported" means that students are no longer being assigned conditions (either through enrollment or a post-experiment rule), and any client application code that delivers conditions no longer needs to be present (i.e., we will not check for validity of conditions, decision points, or other parameters in the context metadata). Archived status is permanent; once archived, an experiment can't transition to another status.

We expect the list of Archived experiments to keep growing and (eventually) be substantial, so leaving them in UI of the main list of experiments is not a good long-term option. One possibility would be to have the Archived list show up on the left-hand menu, or access Archived experiments another way such as hiding them in the list of experiments unless a box is checked, or other option. @zackcl will work on some mockups for the UI.

Archiving an experiment maintains the parameter settings and the "last known state" of enrollment and metric counts. Data should be downloadable from an Archived experiment (edited). Archiving should also be coordinated with any "un-staging" of client app code that previously supported the experiment, but we leave the specifics of that process to individual client app developers.

[SritterCL]

I like the idea of hiding archived experiments. We might also want to be able to permanently delete them, just to clean things up,

I would allow data to be downloaded from archived experiments because if you archive an experiment before downloading data, you'd have no way to get the data.

[amurphy-cl]

Yeah, I don't see why the delete option that we have for all experiments should be disabled for archived ones. Somewhat related, the ability to batch export experiment data might be handy for when we do clean-up.

Good point about the status change and data access. Edited the above to reflect this.

[amurphy-cl]

Bumping this up to revisit -- this feature would be handy to have with the numerous xprize experiments after they're done running.

[nirmalpatel]

Here's a big question about the archived experiments: can they be unarchived? I think it will be easier to build this feature if experiments can't be unarchived. What would happen to the post-experiment rule once an experiment is archived?

On Wed, Jan 18, 2023 at 12:20 AM amurphy-cl @.***> wrote:

Bumping this up to revisit -- this feature would be handy to have with the numerous xprize experiments after they're done running.

— Reply to this email directly, view it on GitHub https://github.com/CarnegieLearningWeb/UpGrade/issues/30#issuecomment-1385886404, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGKXQH2FG4CLIEJBTBR3CTWS3SXVANCNFSM432CC3MA . You are receiving this because you were mentioned.Message ID: @.***>

-- Nirmal Patel nirmalpatel.net

[amurphy-cl]

No, I don't think they should be allowed to be unarchived, but we should be able to say, export the experiment design json. IIRC when we previously discussed how the post rule would operate with respect to archiving, we said that it would not function anymore -- that Archived experiments would be effectively cancelled (insofar as they wouldn't deliver conditions live or in post), as well as possibly be hidden from the UI.

[danoswaltCL]

once an experiment goes from complete to archived, it seems like the researcher must be committing to transitioning the experiment permanently into read-only, historical status. At that point, it seems like it should no longer be upgrade's job to keep sending a post-rule condition, someone should have decided whether or not to change the implementation at the experiment site in the platform.

do we want to allow for a separate read-only database for archived experiments that we can point to? To separate out the work-a-day experiments from the retired ones that are just there to be analyzed and queried?

[zackcl]

Like April said, the archived experiments can't be unarchived or changed to a different status.

[zackcl]

I need to QA this.

[zackcl]

If I try to change the experiment status to "Archived", it doesn't work and I get the following error on the backend console:

TypeError: Cannot read properties of undefined (reading 'experiment')
    at QueryService.<anonymous> (/Users/zlee/Desktop/code/UpGrade/backend/packages/Upgrade/src/api/services/QueryService.ts:71:26)
    at Generator.next (<anonymous>)
    at fulfilled (/Users/zlee/Desktop/code/UpGrade/backend/packages/Upgrade/node_modules/tslib/tslib.js:114:62)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

Here's the screen recording showing the issue:

https://github.com/CarnegieLearningWeb/UpGrade/assets/90279765/f78ac115-7aea-48b2-ac54-e614dbd8ce9a

[zackcl]

Another (minor) issue with the UI is that when the "Archived" status is selected from the dropdown, a checkbox appears which should enable and disable the 'SAVE' button. However, once the 'SAVE' button is enabled, disabling the checkbox does not affect the button. This should be fixed.

https://github.com/CarnegieLearningWeb/UpGrade/assets/90279765/f9df7b66-214b-4983-a706-2b63433ef888

[ppratikcr7]

@zackcl Can you please review the fix in this PR.

[zackcl]

QA: Passed