Complete code

[djones4487169]

I'm interested in the ML pipeline using OHIF and would like to look at the complete code to see how it is put together. I can see the server side code (SAKE) and the original Viewer code but how does this link to your new function OHIF viewer? For example, how did you add the Segment to the toolbar and get t to integrate the ML pipeline with the original OHIF viewer code?

I'm new to Flask etc so please bear with me with your reply so I can keep up.

Thanks David

[span314]

I'll try to find some time this week to rebase against the latest version on OHIF and test/document adding the plugin so I can give a better answer.

Off the top of my head, you'll need at least the following steps to get the tool working in the frontend. 1) The code for the segmentation plugin under Packages https://github.com/capstone-mgh/Viewers/tree/master/Packages/sake 2) Add the button to the appropriate toolbarSection.js (We've only tested this with the StandaloneViewer for the demo, but hopefully it will work for the full OHIFViewer and LesionTracker as well.)

        buttonData.push({
            id: 'sake',
            title: 'Segment',
            classes: 'imageViewerTool toolbarSectionButton',
            svgLink: 'packages/sake/assets/sakelogo.svg#icon-tools-segment'
        });

3) Add sake to the list of stack tools in cornerstoneTools.js (temporary solution--source file is stackSpecificStateManager.js in the Cornerstone project). The plugin needs to store data on the stack rather than individual image layers to handle 3D regions.

var stackTools = [ 'stack', 'stackPrefetch', 'playClip', 'volume', 'slab', 'referenceLines', 'crosshairs', 'sake' ];

The machine learning backend gets called (via a RESTful AJAX POST request) with the sendAnalyzeRequest function in segment.js. You would have to update the url to the appropriate server.

The data contains the array of polygons and some meta-data, and a new request is sent whenever the polygon is updated by the user. We used Flask as a lightweight Python REST endpoint, but you could pretty much use anything that can accept JSON POST requests for the ML. I'm not really an expert with Flask either. For our server we needed to install Python, flask, and flask-cors and add an entry to the Apache httpd.conf configuration file (I think the README has some of the details). I'm probably forgetting about some steps.

I'll give an update after I get around to testing some of these steps. Sorry things aren't documented or in a clean state right now.

[djones4487169]

Thanks for the detailed reply much appreciated an update of the code along with the OHIF viewer would be excellent.

Just to get me going how would you change, for example the logo and name top left, text on the GUI, text overlays on the image, colours etc. so I can begin to customise the OHIF viewer if possible?

I notice your version has a couple of built-in images. How would you go about linking the viewer to a PACS server (e.g. Orthanc) that can run in the background and take requests for images as and when? The original OHIF viewer does something similar I believe so you must have disabled that part?