Add "standard" video pipeline #8

Closed thompson318 closed 4 years ago

In GitLab by @MattClarkson on Oct 29, 2018, 16:11

Is it worth having a class to represent a pipeline:

1) Grab image 2) Scale in X, Y. For example upscale 1920x540 to 1920x1080 3) If intrinsics and distortion params are present - do distortion correction 4) Anything else?

Referring to Issue #3, how would this work with multiple feeds?

Also, how would we add additional image processing, e.g. thresholding, CAFFE, Tensor Flow? or do we just keep it simple for now, and let other developers implement their own additional pipelines to follow this one?

@StephenThompson @ThomasDowrick

In GitLab by @StephenThompson on Oct 29, 2018, 16:41

I would like to keep things simple. i.e. a separate module/package to do each thing, then let the user string pipelines together.

In GitLab by @MattClarkson on Oct 29, 2018, 16:43

But is this common enough to be worthwhile? Otherwise, we end up pushing more code to application layer?

In GitLab by @StephenThompson on Oct 29, 2018, 16:55

I'd rather have an extra line at the application layer than create unnecessarily large libraries.

In GitLab by @ThomasDowrick on Oct 29, 2018, 17:27

I think it is at least worth having some common usage examples for each package, such as the one described above. We might then find that some examples lend themselves to becoming a class/command line progam/gui etc.

We should also try to standardise some function naming conventions between different modules e.g. everything has a .DoProcessing() or .ModifyImage() method, so that we could pass in different modules as arugments to a class/pipeline, which can then run them all automatically without having to call them individually. A quick example:

useful_modules = [distortion, interlace, resize]

MyPipeline.set_input(camera)
MyPipeline.set_modules(useful_modules)

MyPipeline.Start()
# This would then grab data and call .DoProcessing() on each module in useful_modules

In GitLab by @ThomasDowrick on Oct 29, 2018, 18:33

Thinking more on my last point, we should ensure that modules stick to a common interface. Python doesn't support formal interfaces in the way Java (for example) does, but we can document it somewhere, or use an abstract class than can be extended.

In GitLab by @MattClarkson on Oct 29, 2018, 19:05

OK, I'll take all points on board - its obviously not a "no brainer" to go an implement something.

Referring to Steve's comment "a separate module/package to do each thing, then let the user string pipelines together", I agree, but lends itself to a discussion on granularity. For example there would be no point writing a library function to easily call distortion correction, when its pretty much one line of code in opencv-python anyway. Similarly, we can't argue that everything is done at application layer, or else there is no SNAPPY platform, and no jobs for us! :-)

In reference to Tom's comment: "common interface". This is hard to define, as some functions would naturally output 1,2..N outputs, some would want equivalent of call-by-reference, call-by-value (i.e. where do you put memcpys in, where do you use views, where do you use reference to pre-allocated data). So, I can see why Steve pushes it back to the application layer.

So, can someone (my wifi is failing, so I might lose this text) raise some combination of the following Issues on SNAPPY project, or just questions on the Wiki?

Can image processing ops be standardised? What would an interface be?
Can we modularise GUI components without creating too much complexity? i.e if 10 different apps needed video camera calibration, could we provide a standard window? If so, is it a plugin? or just a widget?
Or are we overthinking it for now, and we should just focus on integrating some devices for a bit. So, we raise the above points as issues for discussion later. Then, we crack on with device integration. For example, I still don't know how to make a Python wrapper for a C++ library nicely (e.g. Atracsys).

So, if somone could raise these issues (im approaching St. Pancras i think), then we can close this issue.

In GitLab by @MattClarkson on Oct 29, 2018, 22:01

closed

SciKit-Surgery / scikit-surgeryimage

Add "standard" video pipeline #8