Removal of inactive projects

[bogdanromanx]

For deployments that are meant to showcase the Nexus features, like for example one that supports an EDX course, a lot of projects are created for test purposes. Having the ability to archive and remove inactive projects will free resources without the need for human intervention.

The system should allow the control of an administrator to delete existing projects through:

• an API call to Delta
• a simple to use control interface in Fusion

Expected behavior:

• the function needs to be guarded by a custom permission projects/delete
• the Fusion control needs to be made available only to users that have the required permission (mock-ups to be defined and agreed with @adulbrich)
• the Fusion control needs to ask for user confirmation through typing, similar to the Github repository deletion, such that projects are not deleted by mistake
• after the deletion function is invoked, no further changes can be made to the project and Fusion will display a 'Deleting' status for the project
• upon completion of the delete function, the project reference is completely removed from the system:
  • all Delta managed resources (views, resolvers, storages, etc.) will be removed from caches
  • the project will no longer appear in listing operations
  • a new project with the same name can be created as if the project never existed
  • Fusion no longer displays the project in 'Deleting' status, the project is completely removed from listing operations

Other mentions:

• it is expected that the delete function is asynchronous; the API call would return a 202 Accepted status code upon successful execution
• keep track of project deletions in the logs (projects to be deleted will persist in the progress table)

Not in scope:

• Aggregated views
• CrossProjectResolvers
• projects configured with a RemoteDiskStorage or S3Storage cannot be archived unless the users explicitly consent that the files stored will be not included in the archive:
  • Fusion will inform the user that the project contains files on a RemoteDiskStorage or S3Storage that will not be included in the archive
  • the user will have to agree to execute the function even though files managed on these storage types will not be "handled" (will not be included in the archive and will not be removed from the respective storages)
  • Delta will require a query parameter on the API call ignoreRemoteFiles=true as means of "expressing consent"
  • attempting to archive a project without "expressing consent" will result in a 400 BadRequest with a detail as to why the action cannot be performed
  • deprecated projects can be archived
• the archive would be stored on disk on the Delta instance
• the archive needs to contain both resources and files

[imsdu]

• In the first paragraph, it is written that it will not need human intervention but it appears that it does as it needs actions in fusion
• The archives will be of some use at some point ? If they are "test projects" done though the EDX course, they don't have much value no ?
• Is it worth adding a new status to projects and delta/fusion for this ? Because we won't have other uses than for the course no ?

if the main idea is to eliminate free up resources for the env, could we imagine having disposable instances ? We could have N instances of Delta that would point to n namespaces (so same instance of Cassandra/ES/BG but different keyspaces/prefixes)

For example: A user who starts their course on a given month will land a given instance for the duration of its course A user who starts the course on a different month will land on a different instance When a given instance hits its time-to-live, we just back it up / trash it / wipe it / recycle it. We would need some kind of table that would link for example the hash of the username to an instance. Entries in this table would have the same time to live so when a user is not in the table, he will hit the newest instance (so he can start again).

As we have nginx, we could use a lua script to do this ? The lua module in nginx can be used for this type of stuff: https://github.com/openresty/lua-nginx-module#typical-uses lua is not difficult and what we want to do here would be just extract the username from the jwt and handling a table

I hope I am clear and it does not sound so crazy :)

[imsdu]

If we go the archiving way, we must not forget to kill persistence actors related to the project

[bogdanromanx]

In the first paragraph, it is written that it will not need human intervention but it appears that it does as it needs actions in fusion

human intervention == manually removing the data from the database + indices

The archives will be of some use at some point ? If they are "test projects" done though the EDX course, they don't have much value no ?

Discussion lead to the proposal to Delete Projects completely instead of archiving, because it's simpler to implement (archiving = deletion + saving data into an archive); additionally we can rely on backups if we'd like to perform a restoration. @samuel-kerrien, would you be comfortable with this?

[samuel-kerrien]

Yep comfortable with this. Seems to me like the archiving is not strictly necessary for the MOOC so I would support doing first deletion and schedule the archiving at a later time when it is really needed.

[umbreak]

The following are opened questions regarding removal of projects:

• What happens the project is used as a CrossProjectResolver?
• What happens if the project is used on an Aggregated[ElasticSearch|Blazegraph]View?
• Is this feature expected to work without any downtime?

[umbreak]

1. Create a project event DeleteProject. Any subsequent writes on resources will be blocked.

2. Deprecate all the views (this will delete its indices/namespaces).

3. Delete all files

   • Delete the DiskStorage project folder. Postpone the implementation for RemoteDiskStorage and S3Storage.
   • Go through each file (eventsByTag) and delete it.

4. Delete the caches for views, storages, schemas, resolvers, projects.

5. Delete the project entry from the ProjectsCounts cache

6. currentEventsByTag(project) and for each persistenceId (this can be done in batch and in parallel):

   1. Stop the persistence actor.
   2. Call cleanup (delete the events from the primary store).