Rendering micro-service

[c0c0n3]

Implement a rendering micro-service for use in IDR, as documented in

• 2- Rendering, thumbnails, and other cache loading

in Josh's Queue Design document.

[joshmoore]

To summarize and add a few items: the goal would be to have an end point for calling bin/omero render test passing a single string like Image:12345. This is done on almost every IDR release to check all the filesets and could also be done periodically to check that all approx 10K. filesets are accessible.

In the one-off case, it would additionally be useful if anyone with access to the same shared account could ask:

• how many tasks are left?
• what's the error rate?
• for all errors, what was the stdout?

For the periodical case, it'd be important that the runs don't overlap and perhaps have errors per run.

cc: @mtbc

[joshmoore]

NB: this is my current setup using https://www.gnu.org/software/parallel/:

# first_image_per_fileset.sql
copy (select * from (select f.id, i.id, rank() over (partition by f.id order by i.id) as x
         from fileset f, image i
        where f.id = i.fileset
          )
        as r where r.x = 1) to stdout;

# script.sh
parallel -a images --eta --jobs 2 --joblog log --results rslt --files \
        /opt/omero/server/OMERO.server/bin/omero -q render test --force Image:{/}

$ psql -f first_image_per_fileset.sql | cut -f 2 > images
$ ./script.sh

cc: @manics

[joshmoore]

@c0c0n3 : do you have any suggestions on starting points for this now that ms-queue is done?

[c0c0n3]

I've just whipped together an example of scaffolding code for the rendering microservice and parked it my branch over here:

https://github.com/c0c0n3/omero-ms-queue/tree/rendering-ms/components/rnd-ms

NB this isn't meant to be prod code! it's just an example!! cc: @juliomateoslangerak could use this as a starting point for ideas about how to implement a deconvolution task.