ome / prod-playbooks

Playbooks used by the OME team for deploying production services including OMERO
https://www.openmicroscopy.org/omero
BSD 2-Clause "Simplified" License
4 stars 19 forks source link

Learning maintenance: evaluate script moving users to a disabled group #363

Open sbesson opened 2 years ago

sbesson commented 2 years ago

The UoD SLS learning system is getting heavily used during the academic term with new students logging into the resource using their University credentials and being mapped into the default group containing the course supporting data to allow access - see https://github.com/ome/prod-playbooks/blob/b2e01370ca4d47b95176302cd414fa279b10a5a3/omero/learning.yml#L57-L61.

Eventually, many teaching semesters lead to the creation of several 100s of users in the single group. This causes performance degradation on the OMERO.web deployment as several of the queries start to run more slowly.

At the moment, the OME team runs periodically ad-hoc maintenance scripts on the system to reduce the number of users in the group. On the image.sc forum, a similar problem was discussed with a script being shared which allows to move stale users into a "graveyard" group - see https://forum.image.sc/t/omero-user-scripted-inactivation/70767/3.

For the next maintenance, we might want to evaluate whether this script could be adapted to the requirements of the SLS learning deployment and used to move students from the former year to another group to restore performance.

pwalczysko commented 2 years ago

Tested on a local omero server with 61 users in 79 groups.

The script is working without problems, the situation before

``` omero user list Using session for root@localhost:4064. Idle timeout: 10 min. Current group: system id | login | first name | last name | email | active | ldap | admin | member of | owner of ----+--------------------------------------+-------------+------------+-------+--------+-------+-------+-----------+---------- 0 | root | root | root | | Yes | False | Yes | 3 | 1 | guest | Guest | Account | | | False | | 2 | 2 | user-1 | user-1 | user-1 | | Yes | False | | | 3 3 | user-2 | user-2 | user-2 | | Yes | False | | 3,4 | 4 | user-3 | user-3 | user-3 | | Yes | False | | 5 | 4 5 | user-4 | user-4 | user-4 | | Yes | False | | 4 | 6 6 | user-5 | user-5 | user-5 | | Yes | False | | 6 | 5 7 | user-6 | user-6 | user-6 | | Yes | False | Yes | 3,4,5,6 | 8 | user-7 | user-7 | user-7 | | Yes | False | | | 3 9 | user-8 | user-8 | user-8 | | Yes | False | | 3,4 | 10 | user-9 | user-9 | user-9 | | Yes | False | | 5 | 4 11 | user-10 | user-10 | user-10 | | Yes | False | | 4 | 6 12 | user-11 | user-11 | user-11 | | Yes | False | | 6 | 5 13 | user-12 | user-12 | user-12 | | Yes | False | Yes | 3,4,5,6 | 14 | adm-user-1 | adm-user-1 | adm-user-1 | | Yes | False | | | 7 15 | adm-user-2 | adm-user-2 | adm-user-2 | | Yes | False | | 7,8 | 16 | adm-user-3 | adm-user-3 | adm-user-3 | | Yes | False | | 9 | 8 17 | adm-user-4 | adm-user-4 | adm-user-4 | | Yes | False | | 8 | 10 18 | adm-user-5 | adm-user-5 | adm-user-5 | | Yes | False | | 10 | 9 19 | adm-user-6 | adm-user-6 | adm-user-6 | | Yes | False | Yes | 7,8,9,10 | 52 | 364e383a-6dc4-456f-a714-a0e7a9a4fc28 | integration | tester | | Yes | False | | 53 | 53 | 93c4347c-67b3-487d-87be-dee90c9d9053 | integration | tester | | Yes | False | | | 54 54 | 0884834e-5c96-486f-b7b6-3896da90f342 | integration | tester | | Yes | False | | 55,54 | 55 | cf599c11-b462-4a03-b23d-d07bd14636f9 | integration | tester | | Yes | False | Yes | | 56 56 | 7a54f780-0c29-4098-850e-87c6444705a4 | integration | tester | | Yes | False | | 57,56 | 57 | 8830379a-9e0e-4abc-be81-7de098121c1e | integration | tester | | Yes | False | Yes | | 58 58 | 39f8cc5e-5f08-4090-8829-ccec34f1185f | integration | tester | | Yes | False | | 59,58 | 59 | c826120c-9d35-46ec-8365-0606c6c6f2f1 | integration | tester | | Yes | False | | 60 | 60 | 6037f119-c9c7-437a-b5fe-6284485aca8b | integration | tester | | Yes | False | | | 61 61 | f6df8848-faee-43e9-a2b4-aba4e80e6500 | a | user | | Yes | False | | 61 | 62 | e8f43db7-775c-4bd7-99c9-b893ad20ac2f | integration | tester | | Yes | False | | | 62 63 | 0ea1b3b8-2f3f-4c18-9064-cff1b167659c | a | user | | Yes | False | | 62 | 64 | 5a2caab3-f285-47b2-98a2-6d19548c2d66 | integration | tester | | Yes | False | | | 63 65 | e13978ce-da76-49cc-bb88-b306283a5d13 | a | user | | Yes | False | | 63 | 66 | 37dd3bb3-db17-48f3-b80c-932ce2eb7d01 | integration | tester | | Yes | False | | | 64 67 | e85840ea-7e0c-4c14-9441-11b252330962 | a | user | | Yes | False | | 64 | 68 | 938e3121-574e-499b-b088-1b8e65936f7f | integration | tester | | Yes | False | | | 65 69 | 3135fc9c-824e-4191-983e-dc93a71c8e2e | integration | tester | | Yes | False | | 65 | 70 | 73e745d4-015a-49fb-8fe5-f80a08304f4d | integration | tester | | Yes | False | | | 66 71 | 16ee4b03-b1a0-453d-a6db-2a141054185e | a | user | | Yes | False | | 66 | 72 | b9eb9948-21ce-4e0a-baf9-b050891bb084 | integration | tester | | Yes | False | | | 67 73 | 5fc7c388-1ba4-45b3-a3a4-e10a1c72c06c | a | user | | Yes | False | | 67 | 74 | 2e3d764c-d680-47c8-b596-10896185758e | integration | tester | | Yes | False | | 68 | 75 | 0e28f224-5928-4a02-b046-92ecc927fb16 | integration | tester | | Yes | False | | 69 | 76 | 322fd2d4-3c16-40cd-a052-97135d7c6a12 | integration | tester | | Yes | False | | 70 | 77 | 1e8fa56c-062d-4d86-9b9e-d2790367123a | integration | tester | | Yes | False | | | 71 78 | 3048698d-5655-4193-bed7-9ce00d67eb2b | a | user | | Yes | False | | 71 | 79 | 2022d769-14a3-49f0-b618-afdc407ce81c | integration | tester | | Yes | False | | | 72 80 | 9a3f51d6-8f22-4ddb-8bd6-63b52fc78043 | a | user | | Yes | False | | 72 | 81 | 1976e5bc-d1c1-49ad-a389-cc4db421cdf9 | integration | tester | | Yes | False | | | 73 82 | ac3a7053-0f80-4f7e-acb9-3c0287bee36c | a | user | | Yes | False | | 73 | 83 | 1999865b-e21f-4f0f-9d99-0ad52d1a466c | integration | tester | | Yes | False | | | 74 84 | 23fa9d7e-7228-4de3-889e-0cecebb18979 | a | user | | Yes | False | | 74 | 85 | 9adb5f03-cc08-4bed-9b23-46e3d7d242ec | integration | tester | | Yes | False | | | 75 86 | bbb177c5-22f8-42ab-be69-979077f1cfba | integration | tester | | Yes | False | | 75 | 87 | eafc9ef7-9aa9-4b6c-a2d6-a334bf5d75ee | integration | tester | | Yes | False | | | 76 88 | 83e7a307-713c-4dcf-957d-3bd7189d9b79 | a | user | | Yes | False | | 76 | 89 | a3f9c91e-58ab-4394-b480-7e37ae4d9df2 | integration | tester | | Yes | False | | | 77 90 | 93d63cc8-365e-47f4-92c8-750ba9a8d589 | a | user | | Yes | False | | 77 | 91 | 52350d61-d76a-4a27-9b4e-0c43eee8c8d9 | integration | tester | | Yes | False | | 78 | 92 | 3c05ca72-323b-48a2-8fd3-a8789ee566da | integration | tester | | Yes | False | | 79 | omero group list Using session for root@localhost:4064. Idle timeout: 10 min. Current group: system id | name | perms | ldap | # of owners | # of members ----+--------------------------------------+--------+-------+-------------+-------------- 0 | system | rw---- | False | 1 | 5 1 | user | rwr-r- | False | 0 | 60 2 | guest | rw---- | False | 0 | 1 3 | private-2 | rw---- | False | 2 | 5 4 | read-only-1 | rwr--- | False | 2 | 6 5 | read-annotate-1 | rwra-- | False | 2 | 4 6 | read-write-1 | rwrw-- | False | 2 | 4 7 | adm-private-2 | rw---- | False | 1 | 2 8 | adm-read-only-1 | rwr--- | False | 1 | 3 9 | adm-read-annotate-1 | rwra-- | False | 1 | 2 10 | adm-read-write-1 | rwrw-- | False | 1 | 2 53 | aac57b72-ba1d-4900-869a-57aec5db668b | rw---- | False | 0 | 1 54 | 0913db98-9096-4cd2-b528-50662bdf426d | rwr--- | False | 1 | 1 55 | 8d123466-c1b5-4429-9135-db16da7e746a | rwr--- | False | 0 | 1 56 | 31f8a6fa-0e41-4447-8f75-ee668cc9b472 | rwr--- | False | 1 | 1 57 | 5a2a53d5-2624-402f-8485-cc9938453f51 | rwr--- | False | 0 | 1 58 | 85e54459-ddad-41d4-bb61-836f80e6e2e9 | rwr--- | False | 1 | 1 59 | 92fd7d0b-3698-496f-9412-1d32b211b054 | rwr--- | False | 0 | 1 60 | acc4112d-c7d6-4122-9db0-da7922c8d723 | rw---- | False | 0 | 1 61 | c1ab635d-f238-4cb7-9ec8-98d3509999f7 | rwr--- | False | 1 | 1 62 | 7ec9e07d-20c8-4aa7-9c5c-bc5beddef9a6 | rwr--- | False | 1 | 1 63 | c2bcfcbf-e4b6-4725-ac85-af9dadfa81cf | rwr--- | False | 1 | 1 64 | 94b0c3cc-7c8c-4184-858c-673ed306b5e2 | rwr--- | False | 1 | 1 65 | 45dbefda-d63f-4930-bcf1-b35e255824ad | rwr--- | False | 1 | 1 66 | c6e5ddcf-97d8-4405-be83-e89ae302a424 | rwr--- | False | 1 | 1 67 | 58dd561c-8419-4e0f-a7c0-75b220901426 | rwr--- | False | 1 | 1 68 | 9d63e4e4-2aba-472c-a636-bcb275d18fb9 | rw---- | False | 0 | 1 69 | 7d2a95c2-a929-41ec-9844-456cb0d32583 | rwr--- | False | 0 | 1 70 | 1977e9f9-6b62-403e-ba1a-c2030c216114 | rw---- | False | 0 | 1 71 | c408630e-6301-436c-90ca-7b7c166dbb24 | rwr--- | False | 1 | 1 72 | 8c3b27d5-7e0b-463e-b865-dad34970ed61 | rwr--- | False | 1 | 1 73 | 6c5c72fc-9ad6-4552-b51a-35a2b56d2bb6 | rwr--- | False | 1 | 1 74 | 5596e598-33f2-48fc-b017-f48f9f9e789b | rwr--- | False | 1 | 1 75 | ee4223ed-3a4f-4818-ad67-47514427b756 | rwr--- | False | 1 | 1 76 | 2d2b858b-a25d-4d6a-8c39-3312e79fa79a | rwr--- | False | 1 | 1 77 | 4c612944-3f1f-4a84-a963-5f6c61a53d67 | rwr--- | False | 1 | 1 78 | c65b6089-9072-443a-8f64-0248f5c8c4f2 | rw---- | False | 0 | 1 79 | 9fab258b-3bad-4a78-b3b6-b7f9bee6552e | rwr--- | False | 0 | 1 (38 rows) ```

The group with ID 79 was used as graveyard group in the test. The setting of necessary inactive days was 300 (this is a not-often used server and DB). This singled out user-3 (very plausible, the most used user) as active . The script also omitted root, Guest and the logged-in user (user-6). See the output of the script below

``` python inactivate_users.py Enter username:user-6 Password: Enter host IP:localhost Enter the group ID for the inactive users:79 Enter the minimum amount of days a user must have been inactive:300 Ignoring "user-6" (#7) who is logged in. Ignoring "user-3" (#4) who logged in recently. found these 57 users {2: 'user-1', 3: 'user-2', 5: 'user-4', 6: 'user-5', 8: 'user-7', 9: 'user-8', 10: 'user-9', 11: 'user-10', 12: 'user-11', 13: 'user-12', 14: 'adm-user-1', 15: 'adm-user-2', 16: 'adm-user-3', 17: 'adm-user-4', 18: 'adm-user-5', 19: 'adm-user-6', 52: '364e383a-6dc4-456f-a714-a0e7a9a4fc28', 53: '93c4347c-67b3-487d-87be-dee90c9d9053', 54: '0884834e-5c96-486f-b7b6-3896da90f342', 56: '7a54f780-0c29-4098-850e-87c6444705a4', 55: 'cf599c11-b462-4a03-b23d-d07bd14636f9', 67: 'e85840ea-7e0c-4c14-9441-11b252330962', 58: '39f8cc5e-5f08-4090-8829-ccec34f1185f', 57: '8830379a-9e0e-4abc-be81-7de098121c1e', 59: 'c826120c-9d35-46ec-8365-0606c6c6f2f1', 60: '6037f119-c9c7-437a-b5fe-6284485aca8b', 61: 'f6df8848-faee-43e9-a2b4-aba4e80e6500', 62: 'e8f43db7-775c-4bd7-99c9-b893ad20ac2f', 68: '938e3121-574e-499b-b088-1b8e65936f7f', 63: '0ea1b3b8-2f3f-4c18-9064-cff1b167659c', 64: '5a2caab3-f285-47b2-98a2-6d19548c2d66', 65: 'e13978ce-da76-49cc-bb88-b306283a5d13', 66: '37dd3bb3-db17-48f3-b80c-932ce2eb7d01', 69: '3135fc9c-824e-4191-983e-dc93a71c8e2e', 70: '73e745d4-015a-49fb-8fe5-f80a08304f4d', 71: '16ee4b03-b1a0-453d-a6db-2a141054185e', 72: 'b9eb9948-21ce-4e0a-baf9-b050891bb084', 73: '5fc7c388-1ba4-45b3-a3a4-e10a1c72c06c', 74: '2e3d764c-d680-47c8-b596-10896185758e', 75: '0e28f224-5928-4a02-b046-92ecc927fb16', 76: '322fd2d4-3c16-40cd-a052-97135d7c6a12', 77: '1e8fa56c-062d-4d86-9b9e-d2790367123a', 78: '3048698d-5655-4193-bed7-9ce00d67eb2b', 79: '2022d769-14a3-49f0-b618-afdc407ce81c', 80: '9a3f51d6-8f22-4ddb-8bd6-63b52fc78043', 81: '1976e5bc-d1c1-49ad-a389-cc4db421cdf9', 82: 'ac3a7053-0f80-4f7e-acb9-3c0287bee36c', 83: '1999865b-e21f-4f0f-9d99-0ad52d1a466c', 84: '23fa9d7e-7228-4de3-889e-0cecebb18979', 85: '9adb5f03-cc08-4bed-9b23-46e3d7d242ec', 86: 'bbb177c5-22f8-42ab-be69-979077f1cfba', 87: 'eafc9ef7-9aa9-4b6c-a2d6-a334bf5d75ee', 88: '83e7a307-713c-4dcf-957d-3bd7189d9b79', 89: 'a3f9c91e-58ab-4394-b480-7e37ae4d9df2', 90: '93d63cc8-365e-47f4-92c8-750ba9a8d589', 91: '52350d61-d76a-4a27-9b4e-0c43eee8c8d9', 92: '3c05ca72-323b-48a2-8fd3-a8789ee566da'} ______________________________________________ ______________________________________________ user user-1 (2) removed from 2 groups [{1: 'user'}, {3: 'private-2'}] user user-2 (3) removed from 3 groups [{1: 'user'}, {3: 'private-2'}, {4: 'read-only-1'}] user user-4 (5) removed from 3 groups [{1: 'user'}, {4: 'read-only-1'}, {6: 'read-write-1'}] user user-5 (6) removed from 3 groups [{1: 'user'}, {5: 'read-annotate-1'}, {6: 'read-write-1'}] user user-7 (8) removed from 2 groups [{1: 'user'}, {3: 'private-2'}] user user-8 (9) removed from 3 groups [{1: 'user'}, {3: 'private-2'}, {4: 'read-only-1'}] user user-9 (10) removed from 3 groups [{1: 'user'}, {4: 'read-only-1'}, {5: 'read-annotate-1'}] user user-10 (11) removed from 3 groups [{1: 'user'}, {4: 'read-only-1'}, {6: 'read-write-1'}] user user-11 (12) removed from 3 groups [{1: 'user'}, {5: 'read-annotate-1'}, {6: 'read-write-1'}] user user-12 (13) removed from 6 groups [{0: 'system'}, {1: 'user'}, {3: 'private-2'}, {4: 'read-only-1'}, {5: 'read-annotate-1'}, {6: 'read-write-1'}] user adm-user-1 (14) removed from 2 groups [{1: 'user'}, {7: 'adm-private-2'}] user adm-user-2 (15) removed from 3 groups [{1: 'user'}, {7: 'adm-private-2'}, {8: 'adm-read-only-1'}] user adm-user-3 (16) removed from 3 groups [{1: 'user'}, {8: 'adm-read-only-1'}, {9: 'adm-read-annotate-1'}] user adm-user-4 (17) removed from 3 groups [{1: 'user'}, {8: 'adm-read-only-1'}, {10: 'adm-read-write-1'}] user adm-user-5 (18) removed from 3 groups [{1: 'user'}, {9: 'adm-read-annotate-1'}, {10: 'adm-read-write-1'}] user adm-user-6 (19) removed from 6 groups [{0: 'system'}, {1: 'user'}, {7: 'adm-private-2'}, {8: 'adm-read-only-1'}, {9: 'adm-read-annotate-1'}, {10: 'adm-read-write-1'}] user 364e383a-6dc4-456f-a714-a0e7a9a4fc28 (52) removed from 2 groups [{1: 'user'}, {53: 'aac57b72-ba1d-4900-869a-57aec5db668b'}] user 93c4347c-67b3-487d-87be-dee90c9d9053 (53) removed from 2 groups [{1: 'user'}, {54: '0913db98-9096-4cd2-b528-50662bdf426d'}] user 0884834e-5c96-486f-b7b6-3896da90f342 (54) removed from 3 groups [{1: 'user'}, {54: '0913db98-9096-4cd2-b528-50662bdf426d'}, {55: '8d123466-c1b5-4429-9135-db16da7e746a'}] user 7a54f780-0c29-4098-850e-87c6444705a4 (56) removed from 3 groups [{1: 'user'}, {56: '31f8a6fa-0e41-4447-8f75-ee668cc9b472'}, {57: '5a2a53d5-2624-402f-8485-cc9938453f51'}] user cf599c11-b462-4a03-b23d-d07bd14636f9 (55) removed from 3 groups [{0: 'system'}, {1: 'user'}, {56: '31f8a6fa-0e41-4447-8f75-ee668cc9b472'}] user e85840ea-7e0c-4c14-9441-11b252330962 (67) removed from 2 groups [{1: 'user'}, {64: '94b0c3cc-7c8c-4184-858c-673ed306b5e2'}] user 39f8cc5e-5f08-4090-8829-ccec34f1185f (58) removed from 3 groups [{1: 'user'}, {58: '85e54459-ddad-41d4-bb61-836f80e6e2e9'}, {59: '92fd7d0b-3698-496f-9412-1d32b211b054'}] user 8830379a-9e0e-4abc-be81-7de098121c1e (57) removed from 3 groups [{0: 'system'}, {1: 'user'}, {58: '85e54459-ddad-41d4-bb61-836f80e6e2e9'}] user c826120c-9d35-46ec-8365-0606c6c6f2f1 (59) removed from 2 groups [{1: 'user'}, {60: 'acc4112d-c7d6-4122-9db0-da7922c8d723'}] user 6037f119-c9c7-437a-b5fe-6284485aca8b (60) removed from 2 groups [{1: 'user'}, {61: 'c1ab635d-f238-4cb7-9ec8-98d3509999f7'}] user f6df8848-faee-43e9-a2b4-aba4e80e6500 (61) removed from 2 groups [{1: 'user'}, {61: 'c1ab635d-f238-4cb7-9ec8-98d3509999f7'}] user e8f43db7-775c-4bd7-99c9-b893ad20ac2f (62) removed from 2 groups [{1: 'user'}, {62: '7ec9e07d-20c8-4aa7-9c5c-bc5beddef9a6'}] user 938e3121-574e-499b-b088-1b8e65936f7f (68) removed from 2 groups [{1: 'user'}, {65: '45dbefda-d63f-4930-bcf1-b35e255824ad'}] user 0ea1b3b8-2f3f-4c18-9064-cff1b167659c (63) removed from 2 groups [{1: 'user'}, {62: '7ec9e07d-20c8-4aa7-9c5c-bc5beddef9a6'}] user 5a2caab3-f285-47b2-98a2-6d19548c2d66 (64) removed from 2 groups [{1: 'user'}, {63: 'c2bcfcbf-e4b6-4725-ac85-af9dadfa81cf'}] user e13978ce-da76-49cc-bb88-b306283a5d13 (65) removed from 2 groups [{1: 'user'}, {63: 'c2bcfcbf-e4b6-4725-ac85-af9dadfa81cf'}] user 37dd3bb3-db17-48f3-b80c-932ce2eb7d01 (66) removed from 2 groups [{1: 'user'}, {64: '94b0c3cc-7c8c-4184-858c-673ed306b5e2'}] user 3135fc9c-824e-4191-983e-dc93a71c8e2e (69) removed from 2 groups [{1: 'user'}, {65: '45dbefda-d63f-4930-bcf1-b35e255824ad'}] user 73e745d4-015a-49fb-8fe5-f80a08304f4d (70) removed from 2 groups [{1: 'user'}, {66: 'c6e5ddcf-97d8-4405-be83-e89ae302a424'}] user 16ee4b03-b1a0-453d-a6db-2a141054185e (71) removed from 2 groups [{1: 'user'}, {66: 'c6e5ddcf-97d8-4405-be83-e89ae302a424'}] user b9eb9948-21ce-4e0a-baf9-b050891bb084 (72) removed from 2 groups [{1: 'user'}, {67: '58dd561c-8419-4e0f-a7c0-75b220901426'}] user 5fc7c388-1ba4-45b3-a3a4-e10a1c72c06c (73) removed from 2 groups [{1: 'user'}, {67: '58dd561c-8419-4e0f-a7c0-75b220901426'}] user 2e3d764c-d680-47c8-b596-10896185758e (74) removed from 2 groups [{1: 'user'}, {68: '9d63e4e4-2aba-472c-a636-bcb275d18fb9'}] user 0e28f224-5928-4a02-b046-92ecc927fb16 (75) removed from 2 groups [{1: 'user'}, {69: '7d2a95c2-a929-41ec-9844-456cb0d32583'}] user 322fd2d4-3c16-40cd-a052-97135d7c6a12 (76) removed from 2 groups [{1: 'user'}, {70: '1977e9f9-6b62-403e-ba1a-c2030c216114'}] user 1e8fa56c-062d-4d86-9b9e-d2790367123a (77) removed from 2 groups [{1: 'user'}, {71: 'c408630e-6301-436c-90ca-7b7c166dbb24'}] user 3048698d-5655-4193-bed7-9ce00d67eb2b (78) removed from 2 groups [{1: 'user'}, {71: 'c408630e-6301-436c-90ca-7b7c166dbb24'}] user 2022d769-14a3-49f0-b618-afdc407ce81c (79) removed from 2 groups [{1: 'user'}, {72: '8c3b27d5-7e0b-463e-b865-dad34970ed61'}] user 9a3f51d6-8f22-4ddb-8bd6-63b52fc78043 (80) removed from 2 groups [{1: 'user'}, {72: '8c3b27d5-7e0b-463e-b865-dad34970ed61'}] user 1976e5bc-d1c1-49ad-a389-cc4db421cdf9 (81) removed from 2 groups [{1: 'user'}, {73: '6c5c72fc-9ad6-4552-b51a-35a2b56d2bb6'}] user ac3a7053-0f80-4f7e-acb9-3c0287bee36c (82) removed from 2 groups [{1: 'user'}, {73: '6c5c72fc-9ad6-4552-b51a-35a2b56d2bb6'}] user 1999865b-e21f-4f0f-9d99-0ad52d1a466c (83) removed from 2 groups [{1: 'user'}, {74: '5596e598-33f2-48fc-b017-f48f9f9e789b'}] user 23fa9d7e-7228-4de3-889e-0cecebb18979 (84) removed from 2 groups [{1: 'user'}, {74: '5596e598-33f2-48fc-b017-f48f9f9e789b'}] user 9adb5f03-cc08-4bed-9b23-46e3d7d242ec (85) removed from 2 groups [{1: 'user'}, {75: 'ee4223ed-3a4f-4818-ad67-47514427b756'}] user bbb177c5-22f8-42ab-be69-979077f1cfba (86) removed from 2 groups [{1: 'user'}, {75: 'ee4223ed-3a4f-4818-ad67-47514427b756'}] user eafc9ef7-9aa9-4b6c-a2d6-a334bf5d75ee (87) removed from 2 groups [{1: 'user'}, {76: '2d2b858b-a25d-4d6a-8c39-3312e79fa79a'}] user 83e7a307-713c-4dcf-957d-3bd7189d9b79 (88) removed from 2 groups [{1: 'user'}, {76: '2d2b858b-a25d-4d6a-8c39-3312e79fa79a'}] user a3f9c91e-58ab-4394-b480-7e37ae4d9df2 (89) removed from 2 groups [{1: 'user'}, {77: '4c612944-3f1f-4a84-a963-5f6c61a53d67'}] user 93d63cc8-365e-47f4-92c8-750ba9a8d589 (90) removed from 2 groups [{1: 'user'}, {77: '4c612944-3f1f-4a84-a963-5f6c61a53d67'}] user 52350d61-d76a-4a27-9b4e-0c43eee8c8d9 (91) removed from 2 groups [{1: 'user'}, {78: 'c65b6089-9072-443a-8f64-0248f5c8c4f2'}] user 3c05ca72-323b-48a2-8fd3-a8789ee566da (92) removed from 1 groups [{1: 'user'}] ################# #######DONE###### ################# ```

After the run of the script, the output of group list and user list was

``` omero group list Using session for root@localhost:4064. Idle timeout: 10 min. Current group: system id | name | perms | ldap | # of owners | # of members ----+--------------------------------------+--------+-------+-------------+-------------- 0 | system | rw---- | False | 1 | 1 1 | user | rwr-r- | False | 0 | 3 2 | guest | rw---- | False | 0 | 1 3 | private-2 | rw---- | False | 0 | 2 4 | read-only-1 | rwr--- | False | 1 | 1 5 | read-annotate-1 | rwra-- | False | 0 | 2 6 | read-write-1 | rwrw-- | False | 0 | 1 7 | adm-private-2 | rw---- | False | 0 | 0 8 | adm-read-only-1 | rwr--- | False | 0 | 0 9 | adm-read-annotate-1 | rwra-- | False | 0 | 0 10 | adm-read-write-1 | rwrw-- | False | 0 | 0 53 | aac57b72-ba1d-4900-869a-57aec5db668b | rw---- | False | 0 | 0 54 | 0913db98-9096-4cd2-b528-50662bdf426d | rwr--- | False | 0 | 0 55 | 8d123466-c1b5-4429-9135-db16da7e746a | rwr--- | False | 0 | 0 56 | 31f8a6fa-0e41-4447-8f75-ee668cc9b472 | rwr--- | False | 0 | 0 57 | 5a2a53d5-2624-402f-8485-cc9938453f51 | rwr--- | False | 0 | 0 58 | 85e54459-ddad-41d4-bb61-836f80e6e2e9 | rwr--- | False | 0 | 0 59 | 92fd7d0b-3698-496f-9412-1d32b211b054 | rwr--- | False | 0 | 0 60 | acc4112d-c7d6-4122-9db0-da7922c8d723 | rw---- | False | 0 | 0 61 | c1ab635d-f238-4cb7-9ec8-98d3509999f7 | rwr--- | False | 0 | 0 62 | 7ec9e07d-20c8-4aa7-9c5c-bc5beddef9a6 | rwr--- | False | 0 | 0 63 | c2bcfcbf-e4b6-4725-ac85-af9dadfa81cf | rwr--- | False | 0 | 0 64 | 94b0c3cc-7c8c-4184-858c-673ed306b5e2 | rwr--- | False | 0 | 0 65 | 45dbefda-d63f-4930-bcf1-b35e255824ad | rwr--- | False | 0 | 0 66 | c6e5ddcf-97d8-4405-be83-e89ae302a424 | rwr--- | False | 0 | 0 67 | 58dd561c-8419-4e0f-a7c0-75b220901426 | rwr--- | False | 0 | 0 68 | 9d63e4e4-2aba-472c-a636-bcb275d18fb9 | rw---- | False | 0 | 0 69 | 7d2a95c2-a929-41ec-9844-456cb0d32583 | rwr--- | False | 0 | 0 70 | 1977e9f9-6b62-403e-ba1a-c2030c216114 | rw---- | False | 0 | 0 71 | c408630e-6301-436c-90ca-7b7c166dbb24 | rwr--- | False | 0 | 0 72 | 8c3b27d5-7e0b-463e-b865-dad34970ed61 | rwr--- | False | 0 | 0 73 | 6c5c72fc-9ad6-4552-b51a-35a2b56d2bb6 | rwr--- | False | 0 | 0 74 | 5596e598-33f2-48fc-b017-f48f9f9e789b | rwr--- | False | 0 | 0 75 | ee4223ed-3a4f-4818-ad67-47514427b756 | rwr--- | False | 0 | 0 76 | 2d2b858b-a25d-4d6a-8c39-3312e79fa79a | rwr--- | False | 0 | 0 77 | 4c612944-3f1f-4a84-a963-5f6c61a53d67 | rwr--- | False | 0 | 0 78 | c65b6089-9072-443a-8f64-0248f5c8c4f2 | rw---- | False | 0 | 0 79 | 9fab258b-3bad-4a78-b3b6-b7f9bee6552e | rwr--- | False | 0 | 57 (38 rows) (cli) pwalczysko@ls31618~/inactivate-users$ omero user list Using session for root@localhost:4064. Idle timeout: 10 min. Current group: system id | login | first name | last name | email | active | ldap | admin | member of | owner of ----+--------------------------------------+-------------+------------+-------+--------+-------+-------+-----------+---------- 0 | root | root | root | | Yes | False | Yes | 3 | 1 | guest | Guest | Account | | | False | | 2 | 2 | user-1 | user-1 | user-1 | | | False | | 79 | 3 | user-2 | user-2 | user-2 | | | False | | 79 | 4 | user-3 | user-3 | user-3 | | Yes | False | | 5 | 4 5 | user-4 | user-4 | user-4 | | | False | | 79 | 6 | user-5 | user-5 | user-5 | | | False | | 79 | 7 | user-6 | user-6 | user-6 | | Yes | False | Yes | 3,4,5,6 | 8 | user-7 | user-7 | user-7 | | | False | | 79 | 9 | user-8 | user-8 | user-8 | | | False | | 79 | 10 | user-9 | user-9 | user-9 | | | False | | 79 | 11 | user-10 | user-10 | user-10 | | | False | | 79 | 12 | user-11 | user-11 | user-11 | | | False | | 79 | 13 | user-12 | user-12 | user-12 | | | False | | 79 | 14 | adm-user-1 | adm-user-1 | adm-user-1 | | | False | | 79 | 15 | adm-user-2 | adm-user-2 | adm-user-2 | | | False | | 79 | 16 | adm-user-3 | adm-user-3 | adm-user-3 | | | False | | 79 | 17 | adm-user-4 | adm-user-4 | adm-user-4 | | | False | | 79 | 18 | adm-user-5 | adm-user-5 | adm-user-5 | | | False | | 79 | 19 | adm-user-6 | adm-user-6 | adm-user-6 | | | False | | 79 | 52 | 364e383a-6dc4-456f-a714-a0e7a9a4fc28 | integration | tester | | | False | | 79 | 53 | 93c4347c-67b3-487d-87be-dee90c9d9053 | integration | tester | | | False | | 79 | 54 | 0884834e-5c96-486f-b7b6-3896da90f342 | integration | tester | | | False | | 79 | 55 | cf599c11-b462-4a03-b23d-d07bd14636f9 | integration | tester | | | False | | 79 | 56 | 7a54f780-0c29-4098-850e-87c6444705a4 | integration | tester | | | False | | 79 | 57 | 8830379a-9e0e-4abc-be81-7de098121c1e | integration | tester | | | False | | 79 | 58 | 39f8cc5e-5f08-4090-8829-ccec34f1185f | integration | tester | | | False | | 79 | 59 | c826120c-9d35-46ec-8365-0606c6c6f2f1 | integration | tester | | | False | | 79 | 60 | 6037f119-c9c7-437a-b5fe-6284485aca8b | integration | tester | | | False | | 79 | 61 | f6df8848-faee-43e9-a2b4-aba4e80e6500 | a | user | | | False | | 79 | 62 | e8f43db7-775c-4bd7-99c9-b893ad20ac2f | integration | tester | | | False | | 79 | 63 | 0ea1b3b8-2f3f-4c18-9064-cff1b167659c | a | user | | | False | | 79 | 64 | 5a2caab3-f285-47b2-98a2-6d19548c2d66 | integration | tester | | | False | | 79 | 65 | e13978ce-da76-49cc-bb88-b306283a5d13 | a | user | | | False | | 79 | 66 | 37dd3bb3-db17-48f3-b80c-932ce2eb7d01 | integration | tester | | | False | | 79 | 67 | e85840ea-7e0c-4c14-9441-11b252330962 | a | user | | | False | | 79 | 68 | 938e3121-574e-499b-b088-1b8e65936f7f | integration | tester | | | False | | 79 | 69 | 3135fc9c-824e-4191-983e-dc93a71c8e2e | integration | tester | | | False | | 79 | 70 | 73e745d4-015a-49fb-8fe5-f80a08304f4d | integration | tester | | | False | | 79 | 71 | 16ee4b03-b1a0-453d-a6db-2a141054185e | a | user | | | False | | 79 | 72 | b9eb9948-21ce-4e0a-baf9-b050891bb084 | integration | tester | | | False | | 79 | 73 | 5fc7c388-1ba4-45b3-a3a4-e10a1c72c06c | a | user | | | False | | 79 | 74 | 2e3d764c-d680-47c8-b596-10896185758e | integration | tester | | | False | | 79 | 75 | 0e28f224-5928-4a02-b046-92ecc927fb16 | integration | tester | | | False | | 79 | 76 | 322fd2d4-3c16-40cd-a052-97135d7c6a12 | integration | tester | | | False | | 79 | 77 | 1e8fa56c-062d-4d86-9b9e-d2790367123a | integration | tester | | | False | | 79 | 78 | 3048698d-5655-4193-bed7-9ce00d67eb2b | a | user | | | False | | 79 | 79 | 2022d769-14a3-49f0-b618-afdc407ce81c | integration | tester | | | False | | 79 | 80 | 9a3f51d6-8f22-4ddb-8bd6-63b52fc78043 | a | user | | | False | | 79 | 81 | 1976e5bc-d1c1-49ad-a389-cc4db421cdf9 | integration | tester | | | False | | 79 | 82 | ac3a7053-0f80-4f7e-acb9-3c0287bee36c | a | user | | | False | | 79 | 83 | 1999865b-e21f-4f0f-9d99-0ad52d1a466c | integration | tester | | | False | | 79 | 84 | 23fa9d7e-7228-4de3-889e-0cecebb18979 | a | user | | | False | | 79 | 85 | 9adb5f03-cc08-4bed-9b23-46e3d7d242ec | integration | tester | | | False | | 79 | 86 | bbb177c5-22f8-42ab-be69-979077f1cfba | integration | tester | | | False | | 79 | 87 | eafc9ef7-9aa9-4b6c-a2d6-a334bf5d75ee | integration | tester | | | False | | 79 | 88 | 83e7a307-713c-4dcf-957d-3bd7189d9b79 | a | user | | | False | | 79 | 89 | a3f9c91e-58ab-4394-b480-7e37ae4d9df2 | integration | tester | | | False | | 79 | 90 | 93d63cc8-365e-47f4-92c8-750ba9a8d589 | a | user | | | False | | 79 | 91 | 52350d61-d76a-4a27-9b4e-0c43eee8c8d9 | integration | tester | | | False | | 79 | 92 | 3c05ca72-323b-48a2-8fd3-a8789ee566da | integration | tester | | | False | | 79 | (61 rows) ```

which is plausible imho. The login of user-3 and user-6 and root went fine, the login of user-4 was blocked as expected.

In summary, I think this is a very useful script. Nevertheless, in larger production systems, it must be executed with caution, indeed, it would make a good sense to have a reverse all back to original state provision in the script - the error (for example putting in too low inactive days number, or having the named users list which should not be inactivated incorrectly) can cause a maintenance problem. At least there should be a Proceed:y/N prompt after the facts were gathered and displayed before the script executes the move. Definitely good to have the pre-run of the script captured first.

pwalczysko commented 2 years ago

@sbesson : When consulting the workflow with @will-moore , an important consideration came up. If we delete users, and the deleted user is logging in again using LDAP, this will re-create their account and they will have no detrimental user experience. This is not a given for the workflow of the graveyard group suggested here. The disabled user

pwalczysko commented 2 years ago

The test on a local OMERO and OMERO.web clearly shows that

The RHP load speed is:

This is an improvement of more than order of magnitude, but mainly, it also completely restores the feeling of responsiveness in webclient for a user using the UI to acceptable levels.

I think this shows that the script would be very useful, except for https://github.com/ome/prod-playbooks/issues/363#issuecomment-1224111856

pwalczysko commented 2 years ago

Note that the slowness is not perceived by an admin which is not a member of the group where the data are located. Unfortunately, unless we adjust the permissions of all the relevant users to be restricted admins on the learning system, we cannot exploit that quirk I guess...

pwalczysko commented 2 years ago

A new idea: the https://github.com/ome/prod-playbooks/issues/363#issuecomment-1224111856 could be solved by forcing the graveyard user to unhook from LDAP, and changing its loginname. In this way, when the same LDAP user logs in again, the account will be re-created ? What do you think @sbesson ?

sbesson commented 2 years ago

Interesting and the approach makes sense to me. I assume we will have to handle the scenario where an archived LDAP user is recreated and archived a second time as it might lead to name conflict.

Note the current maintenance process i.e. the deletion of old users with no associated data via a custom SQL script, meets the minimal requirements in terms of restoring performance of a system with a growing number of temporary users in a single group. The workflow discussed above would have the benefit of keeping the users in the DB which offers advantages mostly in terms of reporting as it allows to introspect historical data while collecting usage metrics.