search

SSHOC / sshoc-marketplace-backend

Code for the backend
Apache License 2.0
2 stars 0 forks source link

Upgrade of postgres and solr #395

Closed KlausIllmayer closed 10 months ago

KlausIllmayer commented 1 year ago

Our sysadmin asks for upgrading to newer versions of postgres and solr in the docker container setup. Currently, there are freeze versions in place that are already out of support (postgresql:9.6 and solr:8.2.0). The first question would be, if there are any general objections against upgrading to more newer versions - ideally the newest ones - and if not, how to introduce a workflow so that we are able to upgrade continuously. The second question is how to do the upgrade in detail, how to test and how to prevent unwanted side effects.

@tparkola What is your opinion on this?

tparkola commented 11 months ago

I upgraded to the versions that did not cause troubles when running the services or executing relevant tests. As for now postgres is in the newest version, Solr is in the newest version of major release 8. Please verify on your environments and remeber about dump / backup :)

KlausIllmayer commented 10 months ago

@tparkola When importing the database dump of the old postgres 9.6.16 version into the new postgres 15.4.0 version, I got two errors, did you also encounter these errors when you tested the new postgres version?

ERROR:  column "id" of relation "item_links" is an identity column
HINT:  Use ALTER TABLE ... ALTER COLUMN ... DROP IDENTITY instead.
[...]
ERROR:  cannot drop sequence public.item_links_id_seq because column id of table public.item_links requires it
HINT:  You can drop column id of table public.item_links instead.

the first refers to the line ALTER TABLE public.item_links ALTER COLUMN id DROP DEFAULT; and the second to the line DROP SEQUENCE public.item_links_id_seq;

One remark: I can fix the issue, but it is more about possible side effects that I'm not fully aware. I don't think, we will have a problem, but just out of curiosity, if you also had the problem (in case you tried to migrate from a postgres 9.x full dump).

KlausIllmayer commented 10 months ago

our sysadmins asked me if it is possible to upgrade the Solr to the version 9.x - @tparkola do you think that this is doable? I guess it is more troubling, as the major jump from current version 8.x to 9.x could be tricky. Can you maybe estimate, how easy this could be done?

tparkola commented 10 months ago

Hi,

  1. I cannot recall problems with importing data, however, I'm not sure if I did full import. If you will provide me with a dump I can try.
  2. In case of solr 9.x - I was able to reach this version, though with some additional re-configuration of both solr and backend. In general the problem is that spring data for solr has been deprecated and does not support newer versions of solr. I was able to overcome that, but this is not a long-term solution. Please review / test these changes thoroughly, as they are not a typical way to do so.
  3. We need to move from solr data to solrj at some point and this is a bigger task. I will add it to the backlog, so that we can plan it.
KlausIllmayer commented 10 months ago

Thanks for the feedback - regarding the import of data: when inserting the dump the first time, the mentioned errors showed up, when doing the import of the dump again, the errors disappeared (it occurred for every instance). I leave it as it is as the database looks correct. If we discover problems in the future, then this could be a cause but I think this is very implausible.

Thanks for looking into the solr issue, I will test it in the next weeks. And yes, a long-term solution is very welcomed ;)

KlausIllmayer commented 1 month ago

this is now finished and solr 9.3.0 is in place on all instances, especially also in the production instance.