search

camicroscope / caMicroscope

Digital pathology image viewer with support for human/machine generated annotations and markups.
BSD 3-Clause "New" or "Revised" License
249 stars 289 forks source link

Integration in web application #641

Open ChristiaensBert opened 1 year ago

ChristiaensBert commented 1 year ago

Is your feature request related to a problem? Please describe. I want to use the caMicroscope within a frontend for a web application. I'm flexible about which framework to use, e.g. React, Javascript, HTML, others ... I want to use the Viewer and some of the backend, but integrate it with the Cloud and some other features.

Is this possible and how can this be done?

birm commented 1 year ago

Hello and thank you for reaching out. Can you share a bit more about what you're hoping to do? Are you planning on leaving the usual camicroscope backend stack (github.com/camicroscope/distro) but have the viewer embedded in some other page? Or are you looking for something else?

ChristiaensBert commented 1 year ago

@birm Hi! Yes of course, so I want to build a Machine Learning model for a client in a histopathology lab.

So first of all, I need to load Whole Slide Images (of a high resolution) into an image viewer (in some format like DICOM, JPG, TIFF, ...) and have some functionalities on this WSI image viewer. First of all, it needs the usual features such as zooming, measuring, maybe even rotating, ... Another important thing is that they need to be able to annotate multiple classes on the WSI image, in a segmentation manner. So choosing a certain label (eg type of tissue) and mark / colour the according regions.

And then a last important feature is that we will do predictions on these WSI images (segmentations, very similar to what they label in previous explained feature) and show these predictions on loaded images as some type of layer. This is probably a bit similar to the previous one, but with annotations already supplied by the backend.

And all of this should indeed be embedded into a Web Application that we will build, that will also have some other stuff, like reporting, metadata and more. So I already noticed that the CaMicroscope does a lot of these things (loading, zooming, measuring, some annotations) but I was wondering if we could transfer this functionality (and adapt if needed) into our frontend / backend. :D

So we also don't need the features that are present in this repo like authentication and overview pages etc, since we will build our own.

Thank you so much for your time!!

Maybe a last question, I saw that a lot of the image viewer functionality is from the OpenSeaDragon library. Do you still think this is the way to go, I don't think they released a lot of updates over the last few years.

birm commented 1 year ago

Apologies for my delayed response!

I'm not sure how we can best support your project, but I'll talk about a few things which caMicroscope use in case they're helpful to you. I guess the other option is trying to incorporate your application into caMicroscope.

Yes, the viewer is based on openseadragon, and we have some apps which use pretrained TF models ( see https://github.com/camicroscope/caMicroscope/tree/master/apps/model and https://github.com/camicroscope/caMicroscope/tree/master/apps/segment).

The backend is a customizable node app which currently uses mongo for storage. The annotation and heatmap formats are caMicroscope specific as of now. Try drawing or labeling a sample image and see what's in the database to get some sample data easily.

The loader and tile server are separate microservices, but both use openslide. The slideloader is really just a python metadata extractor at its core, with a few functions to make uploading easier on top of it. The iipserver is a tile server, which uses openslide to break a slide into pieces. it supports DZI and IIIF urls, which can be useful.

I hope this information is helpful, please let me know if I can help you with your project any further!