search

gbif / ipt

GBIF Integrated Publishing Toolkit (IPT)
https://www.gbif.org/ipt
Apache License 2.0
128 stars 58 forks source link

IPT can't update registration (wrong config/registration2.xml) #2555

Open abubelinha opened 1 month ago

abubelinha commented 1 month ago

Original issue title was: "can't update test mode IPT registration after IP address change"

We have migrated two IPT endpoints (/ipt + /iptest paths) to a different IP address (from 193.144.34.193 to 193.144.42.51)

Both production+test servers are still running in both IPs: the 2nd one is just a clone of the same VM (under a different IP), where we stopped tomcat 9, edited the relevant IP address references inside IPT config files (for both paths), and then restarted tomcat 9.

Then we clicked the UPDATE REGISTRATION button, with different success:

We have been getting this error for more than 10 days in the 2nd IP test mode IPT:

A bad request was issued. Please contact the GBIF Helpdesk (helpdesk@gbif.org) for help [https://gbrds.gbif-uat.org/registry/ipt/update/586ccccd-44a5-4c64-a414-43892f84f6f5]

IPT registration update failed! Update IPT registration failed: HTTP/1.1 400 Bad Request

I tried for several days at different hours. I also restarted tomcat. Logged out and in again with the admin ... But always get 400

Today I finally gave up and decided to update the test mode IPT of the new server. (old server /iptest was running IPT 2.6.3 and the one in the new VM is now running IPT 3.0.1) But we still keep getting 400. Any ideas?

Indeed we plan to update the production IPT, but not before everything goes well in the test one.

Thanks @abubelinha

mike-podolskiy90 commented 1 month ago

Thank you for reporting this

You need to update the tokens in Admin/Registration options/Tokens

abubelinha commented 1 month ago

Thanks @mike-podolskiy90

Why does this only happen to our test mode installation?
I didn't have any problems to update registry for the old production IPT 2.6.1

mike-podolskiy90 commented 1 month ago

It happens when the tokens are corrupted. Also, please make sure to use the latest version, which is the 3.1.0

abubelinha commented 1 month ago

Yes, in this test IPT we are using version 3.1.0

OK, I clicked the "update tokens" button and saw this green message:

IPT tokens update succeeded!

But then, I clicked the "update registration" button and I keep seeing the 400 error.

mike-podolskiy90 commented 1 month ago

Did you put actual token values in the inputs?

abubelinha commented 1 month ago

No. I don't recall having done this in the past. Is this something new in IPT 3?

Should I already have those tokens saved somewhere when I installed IPT 2.6.x? (I was expecting those boxes to auto fill when I clicked the update tokens button).

ManonGros commented 1 month ago

@abubelinha you can contact helpdesk@gbif.org, to ask for the tokens again

abubelinha commented 1 month ago

Thanks @ManonGros ... does your "again" mean we should have received those tokens in the past when installing 2.6.x or a previous release?

Then I will look for them before bothering. But I used to launch a script for upgrading our IPT releases up to 2.6.3 and the tokens were not employed or even commented in any step.

ManonGros commented 1 month ago

If the organisations are registered in your IPT, then you must have had the tokens at some point (likely in an email). Let us know if you would like them (re)sent.

abubelinha commented 1 month ago

Thanks @ManonGros

I have an email like this from registry@gbif.org (dated 2011):

You, or someone else, has requested the password for the organisation 'xxx' to be sent to your e-mail address ([yyy])

The information requested is:

Username: 8chars-4chars-4chars-4chars-12chars Password: 11chars

Is this what I need? I guess that email was related to our production IPT, not the test mode IPT.

I don't have any emails received from registry@gbif-uat.org Should I have one also or do the same credentials apply to both.

Anyway the emailed info and the IPT field names are somehow different:

Are we talking about the same things here? Thanks

ManonGros commented 1 month ago

Hi @abubelinha I think it should be the same thing (we used to call the tokens "password"). You can try it and see if it works. If not, please email us with the name of the organization. Thanks!

abubelinha commented 1 month ago

Thanks but this "update tokens" form is a bit confusing. The organization token I am using (from that old email) seems to be correct:

But what about the IPT password? I don't understand why the IPT doesn't complain about what I put in that field (as long as it has >=6 chars, I get a green banner as long as the token was correct ... so you may unconsciously submit a wrong IPT password but you see the same message as if it was correct).

And regarding to my original issue ... after "updating the token" I still cannot "update the registry" ... I keep getting either 400 or 401, depending on the "IPT password" string used above:

Side note: How I noticed about wrong IPT passwords: in my first try I was not sure about what the "IPT password" meant (its input box is next to the organization token input box, so I thought that was referring to the password received in that old email). So I introduced those ... as I said, the token was correct so the green banners didn't help to know if the IPT password was wrong or not.

As I was getting 401, I guessed the IPT password was probably wrong despite of green banners. I looked at registration2.xml and found wsPassword section. So I compared its value with the one in the original VM (old IP address) and realized I was being asked the old IPT master password. I tried that and it must be correct since that's the only string which does not lead me to a 401.

But anyway I keep getting 400 error when trying to update registry later on. Any hints @mike-podolskiy90 ?

mike-podolskiy90 commented 4 weeks ago

I've never seen it throws 400 with valid tokens. I have to see logs in the Registry to get more ideas, but I also need to know when requests were sent (at least 400 ones) and what env (test or prod)

abubelinha commented 4 weeks ago

I've never seen it throws 400 with valid tokens. I have to see logs in the Registry to get more ideas, but I also need to know when requests were sent (at least 400 ones) and what env (test or prod)

Thanks @mike-podolskiy90 As I said, this is all in test mode environment.

I reported the tokens and IPT password by email to helpdesk last friday, so I suppose you already have them. Do you want me to send them to any other email as well?

I'll retry registry update later on and annotate the date and time, so I can send this to you too.

abubelinha commented 4 weeks ago

OK, here it is.

1) I logged with an admin user at IPT 3.1.0 (test mode environment).

2) In ipt_url/admin/updateRegistration page, I selected "tokens" tab, then I filled correct organization token and IPT password (both submitted to helpdesk last yesterday), and then I clicked "update tokens". Then I got this green banner:

"IPT tokens update succeeded!"

Perhaps it is worth saying that this green banner always appears in ipt_url/home.do
(so, looks like there is a redirection: I have to navigate again to ipt_url/admin/updateRegistration in order to continue with step 3).

3) In ipt_url/admin/updateRegistration "Registration" tab, I clicked "update registration" button (2024.10.28, 18:42 GMT+1) Then I got two red banners (two lines):

A bad request was issued. Please contact the GBIF Help Desk (helpdesk@gbif.org) for help [https://gbrds.gbif-uat.org/registry/ipt/update/586ccccd-44a5-4c64-a414-43892f84f6f5]

IPT registration update failed! Update IPT registration failed: HTTP/1.1 400 Bad Request

This is it

mike-podolskiy90 commented 3 weeks ago

I've sent an update request manually with your credentials and it went fine (you can see that in the registry https://registry.gbif-uat.org/installation/586ccccd-44a5-4c64-a414-43892f84f6f5). Would it be possible to give me access to your test IPT?

abubelinha commented 3 weeks ago

Thanks @mike-podolskiy90

What do you mean with "fine"? Yes, in the registry link you send I see the old endpoint has changed: It does not point to the old IP address anymore. But it does not point to the new IP address either.
The endpoint is just empty now.

Yes, the portal now reflects that (an empty endpoint): https://www.gbif-uat.org/installation/586ccccd-44a5-4c64-a414-43892f84f6f5

Regarding giving access to the test IPT, do you mean creating an user for you? I think there is no problem in doing that. Which user level and which email should I send credentials to?

mike-podolskiy90 commented 3 weeks ago

I suppose the request was not quite complete since it was handcrafted. If possible, an admin account please

mike-podolskiy90 commented 3 weeks ago

By fine I mean the HTTP status of the request was "successful" and not 400/401

abubelinha commented 3 weeks ago

I suppose the request was not quite complete since it was handcrafted. If possible, an admin account please

OK. But to which email? I guess I should provide the same email when creating the account in ipt_url/admin/user.do

mike-podolskiy90 commented 3 weeks ago

mpodolskiy@gbif.org please

I've re-sent the request one more time - now endpoint looks correct

mike-podolskiy90 commented 3 weeks ago

let me know when you send it

abubelinha commented 3 weeks ago

I asked our staff to do that a couple of hours ago, so you should receive the email soon. Are you checking the reasons why the IPT was not able to update registry itself?

If the endpoint is now correct we should now be able to use it for publishing tests again. Please tell us whether we can do that or we should still wait for you to test something else.

Thanks!

mike-podolskiy90 commented 3 weeks ago

Thank you.

What I see in the registry logs when update fails:

Mandatory primary contact and/or hosting organization key missing or incomplete!
IPT installation update failed
mike-podolskiy90 commented 3 weeks ago

It is not necessary to update installation via IPT's "Update registration" one can just update the registry manually directly, just for your information. I was trying to figure out what is the issue and probably make it more clear for the future

abubelinha commented 3 weeks ago

It is not necessary to update installation via IPT's "Update registration" one can just update the registry manually directly, just for your information. I was trying to figure out what is the issue and probably make it more clear for the future

Yes, I suspected that manual possibility, but I also wanted to know what the issue was.

What I see in the registry logs when update fails:

Mandatory primary contact and/or hosting organization key missing or incomplete! IPT installation update failed

What does that mean in terms of IPT configuration?

Anyway, if you can see that info in registry logs, wouldn't it make sense if IPT receives that information back and shows it in a red banner?

abubelinha commented 3 weeks ago

Just in case, I have filled in "IPT administrator email" in admin/config.do

But I still got a 400 when I clicked "Update Registration" button in admin/updateRegistration (2024.10.29, 20:20 GMT +1)

mike-podolskiy90 commented 3 weeks ago

There might be something wrong (missing) with the config/registration2.xml file. Example of the IPT part:

    <ipt>
      <key>8949a635-0eb0-4bf1-aaf9-9af97a0ea949</key>
      <description>This instance showcases the latest release candidate for user acceptance testing (UAT).</description>
      <name>IPT UAT Instance</name>
      <primaryContactType>technical</primaryContactType>
      <primaryContactName>Helpdesk</primaryContactName>
      <primaryContactEmail>helpdesk@gbif.org</primaryContactEmail>
      <organisationKey>0a16da09-7719-40de-8d4f-56a15ed52fb6</organisationKey>
      <created>2015-02-25 18:34:59.847 UTC</created>
      <wsPassword>***</wsPassword>
    </ipt>

primaryContactType, primaryContactName, primaryContactEmail and organisationKey are required to perform the request. Maybe, organisationKey is missing

abubelinha commented 3 weeks ago

I understand config/registration2.xml is something you can't see in IPT using your admin account, and you are asking me to check it. Correct?

mike-podolskiy90 commented 3 weeks ago

Yes, I can't check it via UI. Logs state something of those is missing, so I guess you need to check that

abubelinha commented 3 weeks ago

We sent it to you by email.

There is nothing missing in the <IPT> section, but something looks odd to me in the <description>. It used to content some html code for making a break line, and also a link to our hosting provider. This was visible in the admin/registration.do "Registration" tab (under the "Description for this IPT installation" box, I think).

But now it is not visible anymore, despite it is still present in the registration2.xml file How is this possible?

mike-podolskiy90 commented 3 weeks ago

organisationKey is missing! You can see it should be present in the example above. So you should have there:

<organisationKey>def87a70-0837-11d9-acb2-b8a03c50a862</organisationKey>
abubelinha commented 3 weeks ago

You are completely right. Thanks a lot @mike-podolskiy90

I didn't notice the missing field because I saw a couple of <organisation><key> fields available in the next sections of registry2.xml (one for the default organization, another for our organisation). Also, in <ipt> section I was looking for empty tags <tagname></tagname> and didn't see anyone. So I overlooked the missing organisation.

...
    </ipt>
  </registry>
  <organisation>
    <key>625a5522-1886-4998-be46-52c66dd566c9</key>
     ...
  </organisation>
  <organisation>
    <key>def87a70-0837-11d9-acb2-b8a03c50a862</key>
     ...
  </organisation>
</registration>

But, why is <ipt><organisationKey> missing in our testing IPT registry2.xml?

Should I now add the <ipt><organisationKey> by hand, restart tomcat and try to update registration again?

BTW ... why IPT is not complaining about that missing tag, not even when we click "Update Registration" button?
Shouldn't this be reported somewhere (a banner, logs ... whatever)?

mike-podolskiy90 commented 3 weeks ago

Yes, please add the organisationKey manually. I have no idea right now why that happened, but I'll try to figure that out.

I'll also add IPT logs to make it more clear

abubelinha commented 3 weeks ago

I added <ipt>...<organisationKey>VALUE<organisationKey>...</ipt> to ipt_data_folder/config/registration2.xml

Finally ... "Update Registration" button is working well, and UAT registry is being updated. I will leave this issue open since I think it is important for the IPT to detect and give more information of bad config files in order to prevent issues like this to happen.

For my own record: I think some problems publishing/updating datasets from this testing IPT were related to this issue (#1859, #2360 portal #4394 and some emails to helpdesk 2021.03.05-17, 2022.10.25-2022.11.07).