search

ome / omero-server

Gradle project containing main server logic for OMERO
https://www.openmicroscopy.org/omero
GNU General Public License v2.0
4 stars 14 forks source link

Issue with image rendering #174

Open pwalczysko opened 5 months ago

pwalczysko commented 5 months ago

Observing an issue of slow service when images are being rendered.

  1. docker-compose run locally on M1 (first description of the general narrative):

jstacks will be delivered as I progress with them - see https://github.com/ome/omero-server/issues/174#issuecomment-1916705000

cc @jburel @joshmoore @will-moore @sbesson

pwalczysko commented 5 months ago

In order to be able to take jstacks inside the docker, need (from inside the docker as root)

docker exec --user 0 -i docker-example-omero-omeroserver-1 bash
sudo dnf install java-11-openjdk-devel.aarch64
pwalczysko commented 5 months ago

In order to take jstacks and store them (for now) inside the docker, develop a bash script

collect.sh

#!/bin/sh
while true
do
  echo "taking jstack"
  docker exec --user 0 -i docker-example-omero-omeroserver-1 bash  < test2.sh
  sleep 30
done

where test2.sh is

sudo -u omero-server jstack $(sudo -u omero-server /opt/omero/server/OMERO.server/bin/omero admin ice server pid Blitz-0) > /tmp/jstack-ro${n}-$(date -Iseconds).txt
pwalczysko commented 5 months ago

Docker M1 narrative - corresponding jstacks see

idr0-slot3:/uod/idr/scratch/jstacks-petr/jstacks-M1-docker-20240130-first

OMERO.server 5.6.10 OS Rockylinux9 db.poolsize=10

11.56 Start taking jstacks
11.56 Open an image in OMERO.figure
11.57 Refresh webclient - comes back immediately
11.57 Change rnd settings on that one image in figure
11.58 Refresh webclient - comes back immediately
11.59 Challenge the server with url opening 120 images in figure
11.59 Refresh webclient - comes back after 10 seconds
12.01 Change the rnd in figure on the 120 images
12.01 Refresh webclient
12.02 Webclient still did not come back
12.03 Webclient still did not come back
12.03 the webclient has vanished from the browser
12.04 the webclient is getting reloaded - slowly
12.04 the figure tab is still working on the rendering, aprox half of the panels done
12.05 very slowly, the webclient has the lhp almost rendered
12.06 webclient came back
12.06 around 90% of the panels in the figure are rendered, the rest still spinners
12.08 stop taking jstacks
pwalczysko commented 5 months ago

ome-training-3 VM narrative - corresponding jstacks see

idr0-slot3:/uod/idr/scratch/jstacks-petr/jstacks-ome-train-3-VM-poolsize60-20243001

OMERO.server 5.6.8 OS CentOS7 db.poolsize=60

16.41 start taking jstacks
16.43 open one image in OMERO.figure and change rnd on it
16.44 Refresh webclient - returns immediately
16.49 challenge server with url of 120 images in OMERO.figure https://ome-training-3.openmicroscopy.org/figure/new/?image=58007&image=58008&image=58009&image=58010&image=58011&image=58012&image=58013&image=58014&image=58015&image=58016&image=58017&image=58018&image=58019&image=58020&image=58021&image=58022&image=58023&image=58024&image=58025&image=58026&image=58027&image=58028&image=58029&image=58030&image=58031&image=58032&image=58033&image=58034&image=58035&image=58036&image=58037&image=58038&image=58039&image=58040&image=58041&image=58042&image=58043&image=58044&image=58045&image=58046&image=58047&image=58048&image=58049&image=58050&image=58051&image=58052&image=58053&image=58054&image=58055&image=58056&image=58057&image=58058&image=58059&image=58060&image=58061&image=58062&image=58063&image=58064&image=58065&image=58066&image=58067&image=58068&image=58069&image=58070&image=58071&image=58072&image=58073&image=58074&image=58075&image=58076&image=58077&image=58078&image=58079&image=58080&image=58081&image=58082&image=58083&image=58084&image=58085&image=58086&image=58087&image=58088&image=58089&image=58090&image=58091&image=58092&image=58093&image=58094&image=58095&image=58096&image=58097&image=58098&image=58099&image=58100&image=58101&image=58102&image=58103&image=58104&image=58105&image=58106&image=58107&image=58108&image=58109&image=58110&image=58111&image=58112&image=58113&image=58114&image=58115&image=58116&image=58117&image=58118&image=58119&image=58120&image=58121&image=58122&image=58123&image=58124&image=58125&image=58126&_=1706633063371
16.49 refresh webclient
16.49 webclient comes back after 10 seconds but the single folders are not resposive very fast 
16.50 change rnd settings in figure
16.50 refresh webclient
16.51 webclient still did not return
16.52 webclient still did not return
16.54 webclient still did not return
16.55 webclient returned
16.56 stop taking jstacks

cc @joshmoore

joshmoore commented 5 months ago

The hangs certainly make sense from the jstacks, but neither in them, nor locally in my own, am I seeing the type of deadlock that I was looking for. :/

imagesc-bot commented 4 months ago

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/omero-duplicate-weird-issues/68565/20