Add webconfig to roleblock

[pwalczysko]

@sbesson this PR moves the cofig block directly under the web role, as discussed. Copied the block from from the learning.yml playbook, deleted everything which I did not recognize as useful or know, and added all the config as per the omero-web-outreach-webapps.omero file listings.

Nevertheless, the playbook with these changes still fails with

fatal: [ome-training-3.openmicroscopy.org]: FAILED! => {"changed": false, "msg": "Unable to start service omero-web: Job for omero-web.service failed because the control process exited with error code. See \"systemctl status omero-web.service\" and \"journalctl -xe\" for details.\n"}

[pwalczysko]

...
TASK [ome.omero_web : omero web | configuration 00-omero-web.omero] ************
--- before: /opt/omero/web/config/00-omero-web.omero
+++ after: /Users/pwalczysko/.ansible/tmp/ansible-local-90782m0302681/tmpr53bxzae/00-omero-web-omero.j2
@@ -3,7 +3,6 @@
 config drop default

 # Additional custom options
-config set -- omero.web.apps '["omero_iviewer", "omero_figure", "omero_fpbioimage", "omero_webtagging_autotag", "omero_webtagging_tagsearch", "omero_parade", "omero_mapr"]'
 config set -- omero.web.mapr.config '[[{"menu": "gene", "config": {"default": ["Gene Symbol"], "all": ["Gene Symbol", "Gene Identifier"], "ns": ["openmicroscopy.org/mapr/gene"], "label": "Gene"}}, {"menu": "keyvalue", "config": {"default": ["Any Value"], "all": [], "ns": ["openmicroscopy.org/omero/client/mapAnnotation"], "label": "KeyValue"}}]]'
 config set -- omero.web.open_with '[["omero_fpbioimage", "fpbioimage_index", {"supported_objects": ["image"], "script_url": "fpbioimage/openwith.js", "label": "FPBioimage"}], ["omero_iviewer", "omero_iviewer_index", {"supported_objects": ["images", "dataset", "well"], "script_url": "omero_iviewer/openwith.js", "label": "OMERO.iviewer"}], ["omero_figure", "new_figure", {"supported_objects": ["images"], "target": "_blank", "label": "OMERO.figure"}]]'
 config set -- omero.web.ui.center_plugins '[["Parade", "omero_parade/init.js.html", "omero_parade"], ["Auto Tag", "omero_webtagging_autotag/auto_tag_init.js.html", "auto_tag_panel"]]'

changed: [ome-training-3.openmicroscopy.org]
...

[pwalczysko]

with https://github.com/ome/prod-playbooks/pull/334/commits/276ed12a4f40215416e2b6fe3974e02379e3a6eb I have a successfull wet run of the playbook on ome-training-3. Web restarts and there are no errors. But the output bit of https://github.com/ome/prod-playbooks/pull/334#issuecomment-1072502699 is suggesting that the ....omero file is replaced by a ...j2 file as a source of config ? This might explain the duplication of the top links, the central pane links and of the open with links observed on ome-training-3. Also, the "Data" top link is gone, see screenshot.

ome-training-3

outreach (without this PR)

cc @will-moore

[pwalczysko]

not sure what is wrong. The top links are as expected. But they do not work, maybe I am not installing the apps ?

Note: I have deleted on the ome-training-3 server the file /opt/omero/web/config/.....omero . This removed the duplicate links. But the links now do not work.

[pwalczysko]

I think https://github.com/ome/prod-playbooks/pull/334/commits/024f1c243d2bdfa734ff98e16708248d8359ede7 should fix the apps installation - did not have them in the config set block, so it could not have worked. Will be able to verify once my ssh access is restored, see PRs on mngmt tools repo.

[pwalczysko]

@sbesson with the latest commit fixing the formatting of mapr and extra config, deployed on ome-training-3. Checked there and all works. The difference between the existing and the new config seems to me to be consisting only in the ordering of the apps, which should not bear any importance ?

diff -ur new.config outreach.config 
--- new.config  2022-03-21 13:12:02.837419621 +0000
+++ outreach.config 2022-03-21 13:12:12.488465336 +0000
@@ -1,4 +1,4 @@
-omero.web.apps=["omero_iviewer", "omero_figure", "omero_fpbioimage", "omero_webtagging_autotag", "omero_webtagging_tagsearch", "omero_parade", "omero_mapr", "django_prometheus"]
+omero.web.apps=["django_prometheus", "omero_figure", "omero_fpbioimage", "omero_iviewer", "omero_webtagging_autotag", "omero_webtagging_tagsearch", "omero_parade", "omero_mapr"]
 omero.web.mapr.config=[{"menu": "gene", "config": {"default": ["Gene Symbol"], "all": ["Gene Symbol", "Gene Identifier"], "ns": ["openmicroscopy.org/mapr/gene"], "label": "Gene"}}, {"menu": "keyvalue", "config": {"default": ["Any Value"], "all": [], "ns": ["openmicroscopy.org/omero/client/mapAnnotation"], "label": "KeyValue"}}]
 omero.web.middleware=[{"index": 1, "class": "django.middleware.common.BrokenLinkEmailsMiddleware"}, {"index": 2, "class": "django.middleware.common.CommonMiddleware"}, {"index": 3, "class": "django.contrib.sessions.middleware.SessionMiddleware"}, {"index": 4, "class": "django.middleware.csrf.CsrfViewMiddleware"}, {"index": 5, "class": "django.contrib.messages.middleware.MessageMiddleware"}, {"index": 6, "class": "django.middleware.clickjacking.XFrameOptionsMiddleware"}, {"index": 0, "class": "django_prometheus.middleware.PrometheusBeforeMiddleware"}, {"index": 1000, "class": "django_prometheus.middleware.PrometheusAfterMiddleware"}]
 omero.web.nginx_server_extra_config=["include /etc/nginx/conf.d-nested-includes/*.conf;", "location = /omero-ws {\n  proxy_pass https://127.0.0.1:4066;\n  proxy_http_version 1.1;\n  proxy_set_header Upgrade $http_upgrade;\n  proxy_set_header Connection $connection_upgrade;\n  proxy_read_timeout 86400;\n}\n"]

[pwalczysko]

puzzled by the molecule failures here - did not touch the notebooks pertaining to ome-dundeeomero and simlar... ?

[sbesson]

There are two types of failures here:

• the ome-pg-prod playbook fails with

    fatal: [ome-pg-prod1.openmicroscopy.org]: FAILED! => {"changed": false, "msg": "No package matching 'postgresql96' found available, installed or updated", "rc": 126, "results": ["No package matching 'postgresql96' found available, installed or updated"]}

which is related to https://github.com/ome/ansible-role-postgresql/pull/26 i.e. PSQL 9.6 is EOL and has been removed from the CentOS packages.

• the ome-dundeeomero, omero-training and nightshade-webclients tests all fail while linting the training server playbook with

 WARNING  Listing 1 violation(s) that are fatal
    [204] Lines should be no longer than 160 chars
    omero/training-server/playbook.yml:104
              - ["omero_iviewer", "omero_iviewer_index", {"supported_objects":["images", "dataset", "well"], "script_url": "omero_iviewer/openwith.js", "label": "OMERO.iviewer"}]

Warning: [E204] Lines should be no longer than 160 chars

I can handle the former. Do you want to look into the latter? I assume splitting the one-line string into a YML representation similar to 0fbf81c3bffa33288c19213a01e486f0258f73e7 would be an option

[pwalczysko]

I can handle the former. Do you want to look into the latter? I assume splitting the one-line string into a YML representation similar to https://github.com/ome/prod-playbooks/commit/0fbf81c3bffa33288c19213a01e486f0258f73e7 would be an option

unlike in the mapr config case, the iviewer item in the Open with list does not have nice headers everywhere. The list just starts with something which looks like a value of some (unnamed?) parameter

- ["omero_iviewer", "omero_iviewer_index", {"supported_objects":["images", "dataset", "well"], "script_url": "omero_iviewer/openwith.js", "label": "OMERO.iviewer"}]

I tried to approximate with

- omero_iviewer
- omero_iviewer_index
          - supported_objects:
              - "images"
              - "dataset"
              - "well"
          - script_url: 
              - "omero_iviewer/openwith.js"
          - label: "OMERO.iviewer"

which does not fail in ansible, but is not delivering the final result, because, obviously, the overarching syntax which would suggest "this is iviewer open with item" is not there. How to solve and create a nice "pen" for iviewer ?

[sbesson]

I think you're almost there. Using https://onlineyamltools.com/convert-json-to-yaml, I got

- omero_iviewer
- omero_iviewer_index
- supported_objects:
    - images
    - dataset
    - well
  script_url: omero_iviewer/openwith.js
  label: OMERO.iviewer

[pwalczysko]

I think you're almost there. Using https://onlineyamltools.com/convert-json-to-yaml, I got

- omero_iviewer
- omero_iviewer_index
- supported_objects:
    - images
    - dataset
    - well
  script_url: omero_iviewer/openwith.js
  label: OMERO.iviewer

Thank you @sbesson - after some trials and errors, I realized there must be, surprisingly, a double dash as per https://github.com/ome/prod-playbooks/pull/334/commits/31b3da186153c42f902697e047b33a2c1588d157 , otherwise the web fails with an error complaining about open with and iviewer. The comparison of the before and after showed that one encompassing square bracket is missing around the whole "open with iviewer" bit. Hence the additional starting dash. Tested on ome-training-3, seems to work fine.

[pwalczysko]

@sbesson this https://github.com/ome/prod-playbooks/pull/334/commits/31b3da186153c42f902697e047b33a2c1588d157 seems to have solved the milecule failures as well, except the ome-pg-prod and also

An error occurred during the test sequence action: 'converge'. Cleaning up.
--> Scenario: 'omero-training-server'
--> Action: 'cleanup'
Skipping, cleanup playbook not configured.
--> Scenario: 'omero-training-server'
--> Action: 'destroy'
[WARNING]: Invalid characters were found in group names but not replaced, use
-vvvv to see details

in the omero-training-server. Not sure why.

[pwalczysko]

Edit: deployed successfully also on ome-training-4 As with the ome-training-3, it was necessary and sufficient to

delete the file (manually) /opt/omero/web/config/omero-web-outreach-webapps.omero

and run the playbook here.

[pwalczysko]

This https://github.com/ome/prod-playbooks/pull/334/commits/70ad5af03200998c006e838cafecce0f571004ac seems to have fixed all except the ome-pg-prod ?

[sbesson]

Can you cherry-pick https://github.com/sbesson/prod-playbooks/commit/591472cce86d3aacaad131397ebe949a8bed4af1 ?

[pwalczysko]

yes, but unfortunately, hitting following error atm

To github.com:pwalczysko/prod-playbooks.git
 ! [remote rejected] HEAD -> add-weconfig-to-roleblock (Internal Server Error)
error: failed to push some refs to 'git@github.com:pwalczysko/prod-playbooks.git'

[pwalczysko]

@sbesson your commit is added. The Molecule does not seem to start ?

[sbesson]

@sbesson your commit is added. The Molecule does not seem to start ?

I think that's related to https://www.githubstatus.com/incidents/83lq7ftk19r5 rather than my commit ;)

[pwalczysko]

docker-prod is failing now

[pwalczysko]

Thank you @sbesson , merging.