search

CSCfi / rems

Resource Entitlement Management System is a tool for managing access rights to resources, such as research datasets.
MIT License
51 stars 21 forks source link

Unable to create organization with API #3248

Closed mathob closed 4 months ago

mathob commented 4 months ago

Describe the bug or behavior Unable to create organization with API

To Reproduce Steps to reproduce the behavior:

{
    "organization/id": "organization1",
    "organization/short-name": {
      "fi": "Oletus",
      "en": "Default",
      "sv": "Standard"
    },
    "organization/name": {
      "fi": "Oletusorganisaatio",
      "en": "The Default Organization",
      "sv": "Standardorganisationen"
    }
  }

This suggests that the input needs to include data for organization/modifier despite the CreateOrganizationCommand model indicating that only organization/id, organization/short-name and organization/name are required.

Is my input incorrect? or incomplete?

Expected behavior Expect organization to be created

Environment REMS "version": "v2.35-0-g32faf8c24", "revision": "32faf8c2435dba47474c61b9d9398667aad3481a" running in Docker container

Macroz commented 4 months ago

Hi,

Likely your input is correct, but the database has not seen a migration for a while. These columns were removed two years ago. Could it be so?

https://github.com/CSCfi/rems/pull/2830

On Tue, Jan 16, 2024, 03:53 Matthew Hobbs @.***> wrote:

Describe the bug or behavior Unable to create organization with API

To Reproduce Steps to reproduce the behavior:

  • In Swagger UI go to /swagger-ui/index.html#/organizations/post_api_organizations_create
  • click Try It Out
  • Enter API key, user ID
  • In the CreateOrganizationCommand input text box put:

{ "organization/id": "organization1", "organization/short-name": { "fi": "Oletus", "en": "Default", "sv": "Standard" }, "organization/name": { "fi": "Oletusorganisaatio", "en": "The Default Organization", "sv": "Standardorganisationen" } }

  • Click Execute
  • Observe a 500 error in browser
  • Observe that there are errors in container log the most relevant one being:

2024-01-16 12:42:47 Caused by: org.postgresql.util.PSQLException: ERROR: null value in column "modifieruserid" of relation "organization" violates not-null constraint 2024-01-16 12:42:47 Detail: Failing row contains (organization1, null, 2024-01-16 01:42:47.675842, {"enabled": true, "archived": false, "organization/name": {"en":...).

This suggests that the input needs to include data for organization/modifier despite the CreateOrganizationCommand model indicating that only organization/id, organization/short-name and organization/name are required.

Is my input incorrect? or incomplete?

Expected behavior Expect organization to be created

Environment REMS "version": "v2.35-0-g32faf8c24", "revision": "32faf8c2435dba47474c61b9d9398667aad3481a" running in Docker container

— Reply to this email directly, view it on GitHub https://github.com/CSCfi/rems/issues/3248, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGJC3L2JDSYGDBD5JRDUPLYOXMSNAVCNFSM6AAAAABB4BJVTCVHI2DSMVQWIX3LMV43ASLTON2WKOZSGA4DEOBYHE2DSOI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

mathob commented 4 months ago

Oh.

Yes I started with an elderly installation but ran this migration step which gave no errors:

java -Drems.config=simple-config.edn -jar rems.jar migrate

WARNING: abs already refers to: #'clojure.core/abs in namespace: taoensso.encore, being replaced by: #'taoensso.encore/abs
WARNING: abs already refers to: #'clojure.core/abs in namespace: garden.color, being replaced by: #'garden.color/abs
2024-01-12 17:16:13,737 [main rqc:- - -] INFO  rems.main - REMS {:version v2.35-0-g32faf8c24, :revision 32faf8c2435dba47474c61b9d9398667aad3481a}
2024-01-12 17:16:13,758 [main rqc:- - -] INFO  migratus.core - Starting migrations
2024-01-12 17:16:14,018 [main rqc:- - -] INFO  migratus.core - Running up for [20200213161515 20200316071545 20200325115459 20200421081751 20200421161639 20200427092452 20200519062723 20200904052101 20210129065826 20210201115807 20210317070506 20210326100846 20210810150445 20210913180133 20211104084242 20220114122744 20220120121117 20220421155417 20220524072351 20220615163311 20220621104646 20220711142631 20230912072334 20231026151640 20231103083019]
2024-01-12 17:16:14,018 [main rqc:- - -] INFO  migratus.core - Up 20200213161515-entitlement-add-actor-columns
2024-01-12 17:16:14,056 [main rqc:- - -] INFO  migratus.core - Up 20200316071545-multiple-forms
2024-01-12 17:16:14,389 [main rqc:- - -] INFO  migratus.core - Up 20200325115459-add-organizations
2024-01-12 17:16:14,401 [main rqc:- - -] INFO  migratus.core - Up 20200421081751-remove-api-key-roles
2024-01-12 17:16:14,412 [main rqc:- - -] INFO  migratus.core - Up 20200421161639-api-key-restrictions
2024-01-12 17:16:14,417 [main rqc:- - -] INFO  migratus.core - Up 20200427092452-audit-log
2024-01-12 17:16:14,431 [main rqc:- - -] INFO  migratus.core - Up 20200519062723-multiple-organizations
2024-01-12 17:16:14,560 [main rqc:- - -] INFO  migratus.core - Up 20200904052101-remove-organization-owner-role
2024-01-12 17:16:14,564 [main rqc:- - -] INFO  migratus.core - Up 20210129065826-remove-timestamp-defaults
2024-01-12 17:16:14,571 [main rqc:- - -] INFO  migratus.core - Up 20210201115807-add-form-data
2024-01-12 17:16:14,577 [main rqc:- - -] INFO  migratus.core - Up 20210317070506-user-secrets
2024-01-12 17:16:14,583 [main rqc:- - -] INFO  migratus.core - Up 20210326100846-catalogue-item-form-optional
2024-01-12 17:16:14,587 [main rqc:- - -] INFO  migratus.core - Up 20210810150445-invitations
2024-01-12 17:16:14,604 [main rqc:- - -] INFO  migratus.core - Up 20210913180133-add-resourcedata
2024-01-12 17:16:14,609 [main rqc:- - -] INFO  migratus.core - Up 20211104084242-categories
2024-01-12 17:16:14,617 [main rqc:- - -] INFO  migratus.core - Up 20220114122744-remove-owner-modifier
2024-01-12 17:16:14,623 [main rqc:- - -] INFO  migratus.core - Up 20220120121117-user-mappings
2024-01-12 17:16:14,629 [main rqc:- - -] INFO  migratus.core - Up 20220421155417-un-haka
2024-01-12 17:16:14,706 [main rqc:- - -] INFO  migratus.core - Up 20220524072351-slim-owners
2024-01-12 17:16:14,710 [main rqc:- - -] INFO  migratus.core - Up 20220615163311-remove-org-modifier
2024-01-12 17:16:14,713 [main rqc:- - -] INFO  migratus.core - Up 20220621104646-refactor-workflow-licenses
2024-01-12 17:16:14,717 [main rqc:- - -] INFO  migratus.core - Up 20220711142631-remove-workflow-licenses-table
2024-01-12 17:16:14,729 [main rqc:- - -] INFO  migratus.core - Up 20230912072334-event-public
2024-01-12 17:16:14,765 [main rqc:- - -] INFO  migratus.core - Up 20231026151640-change-expiration-events
2024-01-12 17:16:14,775 [main rqc:- - -] INFO  migratus.core - Up 20231103083019-change-expiration-events-1
2024-01-12 17:16:14,785 [main rqc:- - -] INFO  migratus.core - Ending migrations

In my db the fields owneruserid and modifeduserid are not in those other tables mentioned in #2830 (resource, form_template, etc.), only in the organization table. I guess I inherited a database in which a schema migration step failed or was rolled back for some reason.

Is there a way to check the structures of all tables?

Thanks for your prompt and helpful reply!

Macroz commented 4 months ago

I guess there is no table checker specifically, but REMS does some validation on startup for the basic stuff. If all the migrations have run then everything should be in order.

I can see in the log that this migration has been run, and as I'm looking at what it does, there shouldn't be a reason why the column isn't removed. https://github.com/CSCfi/rems/blob/master/resources/migrations/20220114122744-remove-owner-modifier.up.sql

Could that be case sensitivity? 🤔

I guess that's all I can imagine now. I would just check some of these later migrations and see if the database content seems to match the changes in (up) migrations. Or make sure that the database isn't wrong (just yesterday I was myself wondering why my code didn't work and the reason was that I was running against our dev server and not my local machine ... 😅)

In this case it's also safe to run the down migration and up migration for this specific migration again since it doesn't lose any data (the end result is that the data should have been removed anyway). I'm not sure that would make a difference because the column should have been dropped anyway. That should be something like.

lein migratus down 20220114122744
lein migratus up 20220114122744
mathob commented 4 months ago

I managed to re-run that migration and now with the correct table structure I am able to use post_api_organizations_create as expected.

Thanks for your help @Macroz