Roadmap for JuliaImages

[SimonDanisch]

As discussed in JuliaLang/Color.jl#101 it seems like I can't get away anymore with silently prototyping on my own. JuliaLang/Color.jl#101 seems to be on a good way to be resolved quickly, now the remaining question is how to proceed with Images.jl.

@timholy, where would you like to define the image types and where would you like the algorithms? It seems like you're okay with ImageMagick.jl, does the same apply to e.g. NRRD.jl? I think ImageIO can stop existing, as the IO packages will directly interface with FileIO. Remains the question if Images.jl should become very lightweight including only conversions and constructors while the algorithms go into packages like ImageFilters.jl.

We have similar issues with Meshes.jl, and I think we came to the conclusion that we have the packages as modular as possible and then define convinience packages like MeshesBase, which imports and exports all the smaller packages, offering a quick entry for beginners and avoiding that you have to import 10 packages before you can start working. This addresses the concerns of working against monolithic packages, while keeping the user experience high. Also, in a package like ImageBase, we could make sure that everything is working well together.

Are there any plans for GPU accelerated filtering?

I'm also looking forward to packages like FeatureDetect, probably by @mehmetgunturkun.

CC: @tknopp

[timholy]

As discussed in JuliaLang/Color.jl#101 it seems like I can't get away anymore with silently prototyping on my own.

Not if you advertise it and ask for help with the design/implementation :smile:, as you did for FileIO. Obviously, playing around on one's own is fine.

Yeah, let's keep ImageMagick.jl and NRRD.jl in JuliaIO. I agree there is no need for ImageIO. I plan an ImageCore.jl (or ImageTypes.jl?) package so you can use the standard Image container without getting all the other stuff. I think for reasons of backwards compatibility, Images.jl should stay "big" but be a meta-package that just loads a bunch of more granular packages. Moreover, I think it makes sense to keep the short names for the "load everything" packages, and the longer names for people who want specific components---they've already gone to the effort to think through what they need, so a little more typing is fine.

Are there any plans for GPU accelerated filtering?

I agree this would be a good thing to have someday. Obviously as a separate package. BTW, regarding https://github.com/JuliaGPU/AbstractGPUArray.jl, I haven't had the time to look at it. And mostly I will wait for a PR against CUDArt.jl :smile:.

[SimonDanisch]

Not if you advertise it and ask for help with the design/implementation

:sweat_smile:

Moreover, I think it makes sense to keep the short names for the "load everything" packages, and the longer names for people who want specific components

That's very convincing! Lets do it this way.

Regarding GPUArrays: its tightly coupled to what I need for my work with OpenGL. It will require some more work to make it general and elegant. I hope to find a nice use case soon, to generalize it alongside. Maybe writing some first map/reduce/convolute kernels. I'll be mostly working with OpenCL, though. I really want to avoid NVidias little vendor lock-in scheme.

[timholy]

Regarding GPUArrays: ...I'll be mostly working with OpenCL, though.

Understood.

[rsrock]

First off, it's good to have a home for all of the image processing issues. It's a great idea to collect all of the images stuff in one place. I think a number of us have small packages that don't belong in Images.jl proper, but are useful and could fit in with the overall Images design.

With that, let the bikeshedding begin...

I'd like to discuss the rationale behind splitting off ImageIO into a larger FileIO package. Does it really make sense to separate image IO from image processing? How often will one want to do one without the other? Just to be clear, I think the same concerns apply to other domain-specific filetypes.

The specific concern is this: it's too easy for packages to get out of sync, even though there are (theoretical if not practical) mechanisms in place to prevent this. When something gets borked after Pkg.update(), I usually have to Pkg.status() to make sure I didn't leave anything dirty (happens surprisingly often). Then I have to check the Github pages to see what changes have been made in the the two coupled packages, figure out if one has been tagged without the other, and then decide if I want to checkout master. Often times some of the packages aren't tagged after updates, leading to breakage unless you want to live on master. I'm fine with living on master for Images.jl, but with most other stuff I'd prefer to be on the last tag.

Now multiply all of that that by all of our users.

With a single Images.jl repository that includes file IO, it's basically impossible for the two to get out of sync as long as we run the tests before pushing. (Now that I've written that down, someone will point out I said this the next time I forget :wink:). I know that Google, etc. have taken this idea to an extreme, where all of their code lives in a single repo. Sounds crazy, but I'm tempted to try it out for my lab code, at least.

Is there a happy middle ground, somewhere?

[timholy]

I'm a bit worried about that too. I contribute to so many packages that keeping everything in sync is a big job. Thanks for raising this.

Here are my thoughts, in random order:

• With the arrival of precompilation, development is even nicer with smaller packages, because when you make a change to source code you don't have recompile any of the dependent packages. So from a developer perspective there are new reasons to split.
• If you don't actually hack on a package, with precompilation load time is not the disincentive it once was. Presuming @SimonDanisch only wants to use Colors and Images, he has less reason now to care about the fact that he's loading some extra code he doesn't need. So we're in the situation now where "users" shouldn't care how packages are split, but "developers" might. (And I might be a user of Simon's code and a developer of my own, and for him vice versa.)
• One thing I wonder about is whether a "big" package is intimidating for new developers, and having smaller packages might be more approachable.
• One thing I've wondered is whether new developers will be confused by the zoo of dependencies of a particular package (i.e., the opposite concern of the previous point :wink:)

On balance, though, I want to experiment with splitting out ImageMagick mostly because I really dislike it---the binary dependency causes more problems than all the rest of Images put together. I think having a separate package might make it easier to replace some day. That said, I am not sure this experiment will work. Precompilation has introduced new demands on module dependencies, and there's some risk in "optional" dependencies. I think I have figured out how to manage that, but I will only know once I try.

[SimonDanisch]

@timholy says already most of it. Here are my motivations:

There are a lot of use cases, where you would just want IO without the processing (file explorer, visualizations, saving procedural images, your own processing that is not included in Images for whatever reason, etc). The other way around is probably rare, but definitely not unseen.

I think what influences if all the packages will work well together is the number of interested people with push access and how exhaustive the tests are. Putting this in a github group makes it possible to give the people that potentially can fix issues the push access. Also, nothing prohibits us from testing if all dependencies of Images.jl work well together. As a bonus, its easier to integrate the smaller packages you're talking about. They don't face the high hurdle of actually getting into Images.jl, but they still get exposed to all the developpers that can potentially straighten out incompatabilities.

On the upside, by splitting up the packages, people now have a better chance to only face errors that actually concern them. If you only work with NRRD files you shouldn't worry about if ImageMagick builds successfully. I on the other hand was annoyed by warnings for all the array operations defined in Images for a long time, even though I just used Images to load jpg's. Finally, splitting this in to smaller modules is a good test for modularity. Tim explains why this is a good thing: it makes it possible to more easily swap out functionality and it might be easier to concentrate development.

[timholy]

I still think @rsrock's concerns about the whole package management are important. Weekly I have the experience, "drat, that commit passes on my machine, why did it fail Travis?" only to dig and realize I have some commit in some other repository I forgot to push (or tag). The more repos there are to juggle, the worse it gets.

That said, I'm so used to this now that, for me, it's not entirely a deal-breaker.

I on the other hand was annoyed by warnings for all the array operations defined in Images for a long time, even though I just used Images to load jpg's.

Hmm, that's a good one. I bet you were using Gadfly/DataArrays at the same time. I keep hoping this issue will go away soon (IIUC Jeff plans to turn those warnings into stub functions that throw an error if you try to call them---but doing so reliably requires confidence in the type inference system, and he wants to do the redesign first).

[tknopp]

These concerns are, however, not specific to Images.jl but hold for the Pkg landscape in general.

Finding the right package size is not an easy task. I like the unix philosophy of: Do one thing and do it right. If packages get to big one will start to have redundant parts as one tries to have in these packages the "complete shop". A finer granularity helps here and one then first tries to improve an existing package before making ones own shop complete.

What we have not seen yet are Metapackages which would help giving the users a composed package. With reexport.jl this is actually pretty easy so I would welcome the images split could be combined with the first Julia "toolbox".

[SimonDanisch]

@tknopp, In the future I hope that the package manager changes a lot to make this simpler, but in the meantime I agree that the toolbox approach is a valuable alternative. On 10 Aug 2015 21:41, "Tobias Knopp" notifications@github.com wrote:

These concerns are, however, not specific to Images.jl but hold for the Pkg landscape in general.

Finding the right package size is not an easy task. I like the unix philosophy of: Do one thing and do it right. If packages get to big one will start to have redundant parts as one tries to have in these packages the "complete shop". A finer granularity helps here and one then first tries to improve an existing package before making ones own shop complete.

What we have not seen yet are Metapackages which would help giving the users a composed package. With reexport.jl this is actually pretty easy so I would welcome the images split could be combined with the first Julia "toolbox".

— Reply to this email directly or view it on GitHub https://github.com/JuliaImages/meta/issues/1#issuecomment-129579910.