search

miso-lims / runscanner

Monitors directories on the file system containing the output from sequencing instruments (e.g. Illumina) and reports their status and metrics
https://miso-lims.github.io/runscanner/
GNU General Public License v3.0
2 stars 1 forks source link

Containerized version #131

Open mattjmeier opened 2 years ago

mattjmeier commented 2 years ago

Hello,

I was wondering if you have any plans to release a containerized version of runscanner. Since MISO LIMS runs as a docker stack by default, it seems like it would be easier to integrate into the environment that way.

djcooke commented 2 years ago

We don't have plans to do this, but I agree that it would be ideal. We'll keep this issue open as a request

mattjmeier commented 2 years ago

Thanks!

In the mean time, if I attempt to make a dockerfile, do you have any suggestions about what image I should use as a base? If I get anywhere with it I will share it with you.

djcooke commented 2 years ago

That would be appreciated! tomcat:9-jdk11-openjdk-slim should be a good base - it's the same that the MISO webapp uses

mattjmeier commented 2 years ago

Hello,

I am putting something together here. I don't really have much experience building containers from scratch though. Probably better ways to do things... especially the Maven installation (just found an example online that I copied).

I'm not sure that it works yet though. The web server is running, and I have runs mounted to the /srv/sequencer folder. The problem is that it doesn't see existing runs... is it supposed to? I couldn't really tell from the documentation. When I run java -cp $RUN_SCANNER_HOME/WEB-INF/classes:$RUN_SCANNER_HOME/WEB-INF/lib/'*' -DplatformType=ILLUMINA -Dname=default -Dtz=America/Toronto -Dparameters={} ca.on.oicr.gsi.runscanner.scanner.FindRuns /srv/sequencer/NextSeq500/ it does indeed output run folder names, so I'm not sure where to go from there. I speculated that it could be a permissions issue but setting GID and UID for the container did not help. Here is what the site shows currently (and there is nothing listed in the other tabs):

Core

Uptime | PT27M32.207114S -- | -- Start Time | 2022-02-18T18:52:04.485158Z Run Scanner Version | 1.13.2-SNAPSHOT Is Configuration Good? | Yes Last Configuration Read | 2022-02-18T19:08:04.346811Z Scanning Enabled | Yes Currently Scanning | No Time Since Last Scan | PT11M32.341652S Processed Runs | 0 Waiting Runs | 0

/srv/sequencer/NextSeq500

Maybe I need to run a command to force import existing runs..? Let me know if that is the case...

Anyway, this could probably easily be added to the docker-compose for MISO LIMS but I haven't got that far yet. I suppose it might even be possible to build the Run Scanner software on top of that base image, but I don't really understand the staged build concept yet.

Thanks!

mattjmeier commented 2 years ago

As a follow up, I can also confirm that java -cp $RUN_SCANNER_HOME/WEB-INF/classes:$RUN_SCANNER_HOME/WEB-INF/lib/'*' -DplatformType=ILLUMINA -Dname=default -Dtz=America/Toronto -Dparameters={} ca.on.oicr.gsi.runscanner.scanner.ProcessRun /srv/sequencer/NextSeq500/${run_name} returns a huge JSON result with the expected results... so I am guessing I just need to somehow tell it to look for existing runs?

djcooke commented 2 years ago

The name in runscanner.json is meant to refer to one of the processors in Run Scanner. It should be "default" for all Illumina sequencers. Besides that, I think everything looks good.

It's probably a bug that the page shows config is good when an invalid processor was specified, so I created separate issue #133 to investigate that

I'm very curious... Is "Itchy" the name of your sequencer? 😆

mattjmeier commented 2 years ago

Ohhhhh haha. That's hilarious - yes, it's importing runs now that I changed it back to default. Thanks very much for the help!

Yes, we have two NextSeq 500s... Itchy, and of course, Scratchy.

djcooke commented 2 years ago

I was hoping there was a Scratchy to go with it! 💯

mattjmeier commented 2 years ago

One more thing I will need to do is figure out how to make the database persistent for this. Any ideas what I need to mount to make that work? Should I actually be running a database as a separate service?

djcooke commented 2 years ago

Run Scanner doesn't use a database. It holds the runs in memory and has to re-scan them all after a restart

mattjmeier commented 2 years ago

Oh, that makes sense. I did notice that it works pretty fast (we have less than a thousand runs in total) so obviously not a big deal to rescan.

Anyway, what I've put together in that repo seems to work at my end and communicate properly with the MISO LIMS detailed demo stack. I noticed that I had to actually add both instruments using their Illumina identifiers to actually get MISO to import the runs - it might be worth mentioning this in the documentation somewhere, like here (perhaps as a Step 5 under Deploying), since I didn't see that information explicitly laid out.

Feel free to close the issue if you're happy with the example Dockerfile I've posted. Use whatever you'd like from that repo obviously... if it's something that you guys want to pursue.

The way I am running it is basically as a separate stack right now, but like I suggested I'm sure it would be easy to integrate into the other compose file - then you wouldn't need to expose ports on the host machine since it could communicate in the docker overlay networks.

All the best, and thanks again for your help!