search

JuliaImages / Images.jl

An image library for Julia
http://juliaimages.org/
Other
535 stars 142 forks source link

Countdown to the new Images.jl #542

Closed timholy closed 7 years ago

timholy commented 8 years ago

There have been justifiable questions about "what exactly is happening?" so let me post my TODO list here (including completed items, with an explanation of purpose):

Things that could be done before or after:

Longer-term future: implement lots of missing functions. Other BSD-licensed projects should be used as important inspiration.

Evizero commented 8 years ago

Absolutely! Right now I do the operation chaining for my image augmentation package unnecessary inefficient by creating temporary copies after each operation (input -> cropped image -> rotated image -> distorted image .... etc).

After watching your talk I am really sold on the idea of lazy views, which should - if I am not mistaken - allow for just nesting views instead; which would avoid my predicament.

That said, I am not an image processing or computer vision person, so bare with my newbie mistakes I am bound to make, but I am catching on.

To be a little less vague about my true intentions, I essentially want to create a bridge between julias nice Image functionality and machine learning needs, which diverge in some strange places. For example the "z" or "time" dimension does for us store observations with no intrinsic order/relation. Even stranger: in some of our usecases the color dimension of an image is actually farther apart in memory than the spatial!. so a 3x20x20x1000 image would often be represented in 20x20x3x1000 (this is just a consequence of how some of our algorithms work)

Anyway, thank you for taking the time to explain this!

timholy commented 8 years ago

With regards to your 20x20x3x1000 array, using ImagesCore you can already do this:

julia> using ImagesCore, Colors

julia> a = rand(UInt8, 20, 20, 3, 1000);

julia> summary(a)
"20×20×3×1000 Array{UInt8,4}"

julia> p = permuteddimsview(a, (3,1,2,4));

julia> c = colorview(RGB, ufixedview(p));

julia> summary(c)
"20×20×1000 ImagesCore.ColorView{ColorTypes.RGB{FixedPointNumbers.UFixed{UInt8,8}},3,MappedArrays.MappedArray{FixedPointNumbers.UFixed{UInt8,8},4,Base.PermutedDimsArrays.PermutedDimsArray{UInt8,4,(3,1,2,4),(2,3,1,4),Array{UInt8,4}},ImagesCore.##15#17{FixedPointNumbers.UFixed{UInt8,8}},ImagesCore.##16#18}}"

julia> col = c[5, 3, 80]
RGB{U8}(0.953,0.914,0.208)

julia> red(col).i
0xf3

julia> a[5,3,1,80]
0xf3

julia> green(col).i
0xe9

julia> a[5,3,2,80]
0xe9

julia> blue(col).i
0x35

julia> a[5,3,3,80]
0x35

julia> a[5,3,2,80] = 0
0

julia> c[5,3,80]
RGB{U8}(0.953,0.0,0.208)

That's the thing about these views...it takes a while to realize how something that seems so "boring" can be so insanely useful. In this case we put a "permuteddimsview" inside of a "ufixedview" inside of a "colorview" and should get darn good performance despite all those shennanigans.

Now the only thing left is to wrap it in an AxisArray (using ImagesAxes.jl) that indicates that the final dimension is time (if that's what you are using it for).

mronian commented 8 years ago

Maybe we could include corners(recently overhauled) and edges(probably needs changes) in ImageFeatures.jl since they come under feature detection.

:+1: for 100% test coverage ! Love to see the bright green coverage badge :smile:

Regarding the documentation, I was thinking of having one juliaimages.github.io which will have the docs for everything. I am still a bit confused how to use Documenter.jl to tap into each repository and get the docstring. I think one of the other GSoCers has done it and I'll take help from him. Does this seem like a good idea or should there be a separate documentation for each one?

mronian commented 8 years ago

Should TestImages.jl be moved to JuliaImages? We could also have some other package (ImageSets.jl ?) which will have the famous datasets in a easily importable format.

Evizero commented 8 years ago

I think it would make sense to keep the documentations separate and focused, but with a link section somewhere which point to each other.

mronian commented 8 years ago

Maybe we could have stuff like tutorials etc on the main website and if someone wants to dig into the function reference then they would be pointed to the individual websites.

rsrock commented 8 years ago

Before I read any of the above, let me say thanks for filling us in! I was wondering "what exactly is happening???"

rsrock commented 8 years ago

All of the above sounds great, I can't wait to kick the tires.

Here's one thing for my personal wishlist, a pretty printer for image types. When I see something like:

"20×20×1000 ImagesCore.ColorView{ColorTypes.RGB{FixedPointNumbers.UFixed{UInt8,8}},3,MappedArrays.MappedArray{FixedPointNumbers.UFixed{UInt8,8},4,Base.PermutedDimsArrays.PermutedDimsArray{UInt8,4,(3,1,2,4),(2,3,1,4),Array{UInt8,4}},ImagesCore.##15#17{FixedPointNumbers.UFixed{UInt8,8}},ImagesCore.##16#18}}"

I have to sit there and count curly braces to figure out where I am! A few spaces in there would help as well.

timholy commented 8 years ago

@mronian,

Maybe we could include corners(recently overhauled) and edges(probably needs changes) in ImageFeatures.jl since they come under feature detection.

I was wondering the same thing. I am in the middle of putting the gradient kernels into ImagesFiltering, but the edge-detection stuff doesn't quite fit. One option is to leave them in Images (src/algorithms won't entirely go away), but I tend to agree that ImageFeatures might be a better choice. CC @kmsquire to see if he has a preference.

:+1: for 100% test coverage ! Love to see the bright green coverage badge :smile:

Yes, that's important. The new repos are generally closer to that goal. ImagesAxes is much better than it looks, because some bug (probably in julia) incorrectly misses test coverage of traits-functions. See https://github.com/mauro3/SimpleTraits.jl/pull/6.

Regarding the documentation, I was thinking of having one juliaimages.github.io which will have the docs for everything.

I've had the same thought. I think it will be good to have the docs for Images.jl take much more of a tutorial style, and they can link out to the docs for the individual packages for details. But it would be good to also have one giant function reference. I think that will be fairly straightforward with Documenter, but I haven't tested yet.

Should TestImages.jl be moved to JuliaImages? We could also have some other package (ImageSets.jl ?) which will have the famous datasets in a easily importable format.

Yes, good idea.

@rsrock,

Here's one thing for my personal wishlist, a pretty printer for image types.

Yes, I agree. Ideally that would be something fixed in base julia, but perhaps we could pioneer something here. Jeff has been violently opposed to abbreviating the output of typeof(x), but seems fine with customizing the printing of x itself. So we could conceivably modify Base.summary. We'd have to do it for all array types, or perhaps all array types of Colorants, which is a little scary, because it would affect other packages too that were loaded at the same time as Images.

kmsquire commented 8 years ago

I think putting edge detection in ImageFeatures would be fine.

mronian commented 8 years ago

When we move an algorithm say A, how do we do so without affecting either of the packages? If I just directly copy, it gives a redefinition of A error. What is the correct way to do this so that A is ported and none of the packages depending on the older A are affected?

Evizero commented 8 years ago

I think it will be good to have the docs for Images.jl take much more of a tutorial style

@timholy What do you have in mind here? I wrote a beginner's tutorial on julia's approach to handling image/pixel data for a project of mine over here, and I'd happily contribute it if you want it (it is somewhat WIP still). But since it is mainly aimed at ML students (possibly new to Julia), and non-image-experts, I am guessing it might be a little too beginner oriented to be useful to you? (also, feel free to criticize me when I made a mistake or over/understated something)

timholy commented 8 years ago

When we move an algorithm say A, how do we do so without affecting either of the packages?

I think we have to do merges to master/tagging in synchrony. In other words, start a new branch in both packages that make the same change.

I wrote a beginner's tutorial

I haven't read it in detail, but this looks like exactly the kind of thing I was envisioning. Most will probably display images in the docs, too.

I haven't looked in detail, but http://scikit-image.org/docs/dev/user_guide.html seems like a promising model to follow.

Evizero commented 8 years ago

where would you want such tutorials to be located at?

timholy commented 8 years ago

I think Images.jl should be the home of all the "major" documentation. (Images.jl will move to JuliaImages at the time of the transition.)

mronian commented 8 years ago

We could have the docs in a separate repository -> https://github.com/JuliaImages/juliaimages.github.io so that juliaimages.github.io becomes the website.

Evizero commented 8 years ago

One thing to keep in mind is that if the docs should contain lots of images, then they have to be stored somewhere. That can be a hassle depending on where the docs are (since one wants to keep the julia package size small and the git history clean)

mronian commented 8 years ago

We faced this problem in TestImages.jl for the documentation. Turns out you can use the gh-pages branch to store all website related stuff. :)

timholy commented 8 years ago

All good suggestions. :+1: to putting it in the org.

ViralBShah commented 8 years ago

Is JuliaImages going to be the new github org for all this? I do see various repos initialized there, so seems like it. This is all looking quite good!

ViralBShah commented 8 years ago

We should certainly do a blog post on julialang.org once this is done.

tlnagy commented 8 years ago

Will the new Images be able to be installed and used with a simple using Images? Because right now it looks like it is broken into a huge number of smaller packages, which could be a nightmare to interface with.

timholy commented 8 years ago

Yes, it will. First, they will all be REQUIREd packages, so they'll be installed automatically. Second, their functions will all be @reexported, so unless you want just a subset of the functionality of Images (and therefore load specific sub-packages) you won't even have to care how the functionality is partitioned at the source-code level. Images will probably still have some "real" code in it, but in many ways it is evolving into a meta-package that just bundles a lot of related functionality together.

This split-up is partly designed to make them more accessible to people who want a subset of the functionality of Images...for example https://github.com/JuliaLang/julia/issues/18384. There is no way to satisfy the desire to "reuse code across different types of applications" unless we split things into relatively focused units of functionality. Reexporting is the key that allows you to have this without sacrificing on other equally-worthy goals, like "provide a comprehensive Images package."

tlnagy commented 8 years ago

Thanks for the clarification. That sounds promising, I haven't heard of the @reexport feature yet. Is there a framework for running integration tests in the meta package?

timholy commented 8 years ago

You can run any tests you want...but it sounds like you have something specific in mind with "framework", what do you have in mind?

tlnagy commented 8 years ago

So currently most Julia packages have a test/ folder with unit tests that gets run every time a new commit is pushed to master or a PR is updated. I guess each subpackage of Images will load all of Images every time and test all functionality to make sure that there are no regressions in some other Images subpackage. Now that I said it, it sounds reasonable.

On Wed, Sep 7, 2016, at 15:06, Tim Holy wrote:

You can run any tests you want...but it sounds like you have something specific in mind with "framework", what do you have in mind? — You are receiving this because you commented. Reply to this email directly, view it on GitHub[1], or mute the thread[2].

Links:

  1. https://github.com/timholy/Images.jl/issues/542#issuecomment-245434736
  2. https://github.com/notifications/unsubscribe-auth/ABlaL5F4fW-XkufoV_Kkc82LQqcRFnpIks5qnzV7gaJpZM4Jjs9f
mronian commented 8 years ago

We could define the runtests.jl in Images.jl in such a way that it provides a few options like running specific tests or the tests of each package.

timholy commented 8 years ago

I see your point. This is a valid issue to be concerned about.

In general I would go the other way: I would have the focused packages test their own functionality, and have the meta-package test everything. You're right, though, that this means it's possible to inadvertently break packages, because if you're updating one of the "focused" packages it won't automatically trigger the Images tests.

One model is to have extra tests that run only on Travis. For example, JLD tests JLDArchives (a repository of "outdated" *.jld files) to ensure backward-compatibility. But this doesn't run as part of Pkg.test("JLD") or include("runtests.jl") for just JLD on your local machine--it's something that is specially triggered in the .travis.yml file.

So using this model, each package could have its own focused tests, but then the .travis.yml script also forces the Images.jl tests to run too, to make sure the PR doesn't break other packages. The one downside is that CI will be slower, but perhaps that's the price we have to pay.

tlnagy commented 8 years ago

Agreed. I think it would still be nice to add an option to runtests like @mronian said because otherwise it would be difficult to test your changes locally prior to pushing the PR. It shouldn't be that much slower than the running the full test suite of the current Images.jl because it's the same number of tests, but split into smaller containers.

mronian commented 8 years ago

Will Images.jl have any functionality itself apart from being a meta package (Since ImageCore etc are already separated) ?

tlnagy commented 8 years ago

It's already on @timholy's todo list:

On a branch (let's call it "XYZ"), rebase Images.jl on these packages. Will make heavy use of @rexport.

mronian commented 8 years ago

If it itself doesn't have any functionality, will the tests be similar to those in individual packages? I mean, do we need separate tests again for Images.jl or will the individual package tests suffice?

tlnagy commented 8 years ago

I mean, do we need separate tests again for Images.jl or will the individual package tests suffice?

It would be nice to have a quick way to run all tests of Images* (ImagesFiltering etc) in one base. I envision something like this: make small tweak in ImagesFiltering and run Pkg.test("Images") and make sure you didn't cause any regressions in the other tightly linked Images* packages.

mronian commented 8 years ago

I hope we find a workaround :sweat_smile: as CI will quickly become very slow as these packages are very heavy (ImageFeatures.jl itself takes around 20-30 min now and with additional functionality like AGAST the tests will only bloat up ) :disappointed:

tlnagy commented 8 years ago

Here's one thing for my personal wishlist, a pretty printer for image types. When I see something like:

20×20×1000 ImagesCore.ColorView{ColorTypes.RGB{FixedPointNumbers.UFixed{UInt8,8}},3,MappedArrays.MappedArray{FixedPointNumbers.UFixed{UInt8,8},4,Base.PermutedDimsArrays.PermutedDimsArray{UInt8,4,(3,1,2,4),(2,3,1,4),Array{UInt8,4}},ImagesCore.##15#17{FixedPointNumbers.UFixed{UInt8,8}},ImagesCore.##16#18}}"

I have to sit there and count curly braces to figure out where I am! A few spaces in there would help as well.

I would go further than @rsrock and say that this is basically unreadable as is. I can only imagine what error messages look like with that type printed everywhere.

timholy commented 8 years ago

I also find these extremely difficult to read, and would love to change this. The key question is how to fix it.

Currently we rely on the generic Base.summary for printing that kind of type information. The impenetrably-nested braces comes from that typeof(a). Now, we're advised against changing how types are printed, which is bad news for addressing this:

I can only imagine what error messages look like with that type printed everywhere.

But it's legitimate to provide a specialized show or summary function, since that's used for value-printing. So we can at least fix some aspect of this.

I guess in my mind the key question is, how to specialize it? I see three options:

  1. override the Base definition for every AbstractArray
  2. specialize for arrays that have a Colorant eltype
  3. come up with a list of containers (at least ColorView, ChannelView, MappedArrays, ImageMeta) that deserve specializations of summary (if current plans hold, there will be no "one container to rule them all" when it comes to images).

Pros and cons:

  1. Pro: could perhaps really fix the issue. Current thought would be to compute the nesting depth and then do something immensely clever (just have to figure out what that is :wink:) depending on the answer. Con: this will cause a warning to be printed every time you say using Images. More subversively, it would affect the behavior of any other package that the user had co-loaded with Images---it could mess up printing for their (array) types. That's considered bad form.
  2. Pro: most images will presumably be loaded as Colorant types. In particular, IJulia will only show arrays that have Colorant eltypes as an image; the rest will fall back to text output. Con: leaves out ChannelView and any grayscale images that don't explicitly use Gray.
  3. Pro: probably the cleanest in terms of encapsulating for types we care about. Con: will miss important types like SubArray and PermutedDimsArray, which are also very useful containers for images.

To be honest, my favorite solution is 1. I think we could take something for a test drive here, knowing that we're committing a faux pas, with the excuse that if we all like it we'll submit it as a PR for Base.

This is the moment where somebody reading this chimes in with the brilliant suggestion that gets the best of all possible worlds.

timholy commented 8 years ago

While on the topic of how things are printed, perhaps it's worth pointing out https://github.com/JeffBezanson/FixedPointNumbers.jl/pull/51#issuecomment-247507435, which is a proposal to change how the fixed-point types are printed. A major motivation for this proposal is to make text-display of pixels values (or whole images) more compact.

timholy commented 8 years ago

I've periodically been updating the list above. It feels like it's getting close now.

There will be some delay, unfortunately. I did a fair amount of work updating ImageMagick/writemime, but then my laptop died (motherboard problem), and my old laptop wouldn't read the new ones' hard drive. So some items will likely wait until I get my laptop back. So far I haven't found myself short of other tasks in that list to keep me occupied :persevere: .

timholy commented 8 years ago

Locally I have a candidate solution to the printing-of-types problem mentioned above as being high on @rsrock's wish list, https://github.com/timholy/Images.jl/issues/542#issuecomment-239647238. I figured out a way to do it that will only affect types defined in the images-related packages (I think).

The idea is based on the observation that sometimes the type printing is more confusing than the sequence of calls you'd use to create the object. So I'm trying out a "show it like you build it" approach (I even have a nascent package with that name). Here is a demo:

julia> a = rand(UInt8, 5,5,3)
5×5×3 Array{UInt8,3}:
[:, :, 1] =
 0x63  0x44  0xe1  0x1e  0xba
 0x08  0x07  0x3f  0x4c  0x1f
 0x10  0xaf  0xf7  0xe7  0x87
 0x3c  0x37  0x49  0x6e  0xd4
 0xd9  0x05  0xf2  0x5b  0x0f

[:, :, 2] =
 0x8a  0xeb  0x34  0x6b  0x59
 0x0f  0xd6  0x71  0x8d  0x74
 0xe6  0xfa  0xb1  0xba  0xeb
 0x9b  0xb6  0xa4  0xe2  0x24
 0xe2  0x4f  0x62  0xc5  0x70

[:, :, 3] =
 0xfd  0x31  0x24  0x49  0xf3
 0x9a  0x2d  0x80  0x2c  0x10
 0xeb  0x0e  0x03  0x68  0x68
 0xbd  0x0e  0x39  0xf0  0x0a
 0xc1  0x0c  0x66  0x49  0x78

julia> p = permuteddimsview(a, (3,1,2))
3×5×5 permuteddimsview(::Array{UInt8,3}, (3,1,2)) with element type UInt8:
[:, :, 1] =
 0x63  0x08  0x10  0x3c  0xd9
 0x8a  0x0f  0xe6  0x9b  0xe2
 0xfd  0x9a  0xeb  0xbd  0xc1

[:, :, 2] =
 0x44  0x07  0xaf  0x37  0x05
 0xeb  0xd6  0xfa  0xb6  0x4f
 0x31  0x2d  0x0e  0x0e  0x0c

[:, :, 3] =
 0xe1  0x3f  0xf7  0x49  0xf2
 0x34  0x71  0xb1  0xa4  0x62
 0x24  0x80  0x03  0x39  0x66

[:, :, 4] =
 0x1e  0x4c  0xe7  0x6e  0x5b
 0x6b  0x8d  0xba  0xe2  0xc5
 0x49  0x2c  0x68  0xf0  0x49

[:, :, 5] =
 0xba  0x1f  0x87  0xd4  0x0f
 0x59  0x74  0xeb  0x24  0x70
 0xf3  0x10  0x68  0x0a  0x78

julia> u = ufixedview(p)
3×5×5 ufixedview(N0f8, permuteddimsview(::Array{UInt8,3}, (3,1,2))) with element type FixedPointNumbers.UFixed{UInt8,8}:
[:, :, 1] =
 0.388N0f8  0.031N0f8  0.063N0f8  0.235N0f8  0.851N0f8
 0.541N0f8  0.059N0f8  0.902N0f8  0.608N0f8  0.886N0f8
 0.992N0f8  0.604N0f8  0.922N0f8  0.741N0f8  0.757N0f8

[:, :, 2] =
 0.267N0f8  0.027N0f8  0.686N0f8  0.216N0f8  0.02N0f8 
 0.922N0f8  0.839N0f8  0.98N0f8   0.714N0f8  0.31N0f8 
 0.192N0f8  0.176N0f8  0.055N0f8  0.055N0f8  0.047N0f8

[:, :, 3] =
 0.882N0f8  0.247N0f8  0.969N0f8  0.286N0f8  0.949N0f8
 0.204N0f8  0.443N0f8  0.694N0f8  0.643N0f8  0.384N0f8
 0.141N0f8  0.502N0f8  0.012N0f8  0.224N0f8  0.4N0f8  

[:, :, 4] =
 0.118N0f8  0.298N0f8  0.906N0f8  0.431N0f8  0.357N0f8
 0.42N0f8   0.553N0f8  0.729N0f8  0.886N0f8  0.773N0f8
 0.286N0f8  0.173N0f8  0.408N0f8  0.941N0f8  0.286N0f8

[:, :, 5] =
 0.729N0f8  0.122N0f8  0.529N0f8  0.831N0f8  0.059N0f8
 0.349N0f8  0.455N0f8  0.922N0f8  0.141N0f8  0.439N0f8
 0.953N0f8  0.063N0f8  0.408N0f8  0.039N0f8  0.471N0f8

julia> c = colorview(RGB, u)
5×5 ColorView{RGB}(ufixedview(N0f8, permuteddimsview(::Array{UInt8,3}, (3,1,2)))) with element type ColorTypes.RGB{FixedPointNumbers.UFixed{UInt8,8}}:
 RGB{U8}(0.388,0.541,0.992)  RGB{U8}(0.267,0.922,0.192)  RGB{U8}(0.882,0.204,0.141)  RGB{U8}(0.118,0.42,0.286)   RGB{U8}(0.729,0.349,0.953)
 RGB{U8}(0.031,0.059,0.604)  RGB{U8}(0.027,0.839,0.176)  RGB{U8}(0.247,0.443,0.502)  RGB{U8}(0.298,0.553,0.173)  RGB{U8}(0.122,0.455,0.063)
 RGB{U8}(0.063,0.902,0.922)  RGB{U8}(0.686,0.98,0.055)   RGB{U8}(0.969,0.694,0.012)  RGB{U8}(0.906,0.729,0.408)  RGB{U8}(0.529,0.922,0.408)
 RGB{U8}(0.235,0.608,0.741)  RGB{U8}(0.216,0.714,0.055)  RGB{U8}(0.286,0.643,0.224)  RGB{U8}(0.431,0.886,0.941)  RGB{U8}(0.831,0.141,0.039)
 RGB{U8}(0.851,0.886,0.757)  RGB{U8}(0.02,0.31,0.047)    RGB{U8}(0.949,0.384,0.4)    RGB{U8}(0.357,0.773,0.286)  RGB{U8}(0.059,0.439,0.471)

So this still shows as nested, but I think

ColorView{RGB}(ufixedview(N0f8, permuteddimsview(::Array{UInt8,3}, (3,1,2)))) with element type ColorTypes.RGB{FixedPointNumbers.UFixed{UInt8,8}}

is easier to read than

julia> typeof(c)
ImageCore.ColorView{ColorTypes.RGB{FixedPointNumbers.UFixed{UInt8,8}},2,MappedArrays.MappedArray{FixedPointNumbers.UFixed{UInt8,8},3,Base.PermutedDimsArrays.PermutedDimsArray{UInt8,3,(3,1,2),(2,3,1),Array{UInt8,3}},ImageCore.##29#30{FixedPointNumbers.UFixed{UInt8,8}},Base.#reinterpret}}

Thoughts? Is this a good direction? If so, is it sufficient, or are there adjustments needed?

Evizero commented 8 years ago

I like it. certainly easier than the type. I am not quite sure I fully follow the pattern yet. For example why write ColorView{RGB}, but unfixedview(N0f8, when both originated from a similar looking method call

timholy commented 8 years ago

To explain: it's because ColorView{C}(A) always creates a ColorView, whereas colorview(C, A) might create a ColorView. It just returns a plain array when the combination of C and A is amenable to reinterpret. This strategy is an attempt to reduce the amount of nesting in the types themselves.

I agree that's a little confusing, though, so I'm happy to entertain alternatives.

Evizero commented 8 years ago

Isn't that kind of dispatch true for ufixedview as well https://github.com/JuliaImages/ImageCore.jl/blob/master/src/ImageCore.jl#L90? I guess I am just unclear when a function-style printing is chosen (such as for ufixedview) vs when a type-style printing is chosen (such as for colorview)

Regardless of my confusions (which tend to be numerous no matter what anyways), I do like the change. 👍

timholy commented 8 years ago

I would call this "productive confusion," in the sense that this is 100% valid. So no apologies needed; to the contrary, thanks for raising these concerns.

In the case of ufixedview, I'm not certain I see a way to resolve the issue without introducing problems elsewhere:

ChristianKurz commented 8 years ago

As there are alternative IO-solutions available now, is it still mandatory to install ImageMagick? I'm asking as IM is blocked at my company due to serious security flaws. It would be nice to have a query at installation to select one of the IO-packages.

timholy commented 8 years ago

There is a query on OSX because of QuartzImageIO, but there isn't a comprehensive alternative on any other platform. Netpbm is probably closest.

This is a great area for motivated folks to contribute! If you have another library in mind, you can see from ImageMagick.jl that it's not an outrageously large undertaking to write a wrapper. (Not a trivial amount of work, either.) Or more ambitiously one could write a parser in pure julia (like the Netpbm and NRRD parsers). But that would be one format at a time, and a much larger undertaking.

Evizero commented 8 years ago

http://www.graphicsmagick.org/index.html

timholy commented 8 years ago

See also previous discussion in https://github.com/timholy/Images.jl/issues/24

ChristianKurz commented 8 years ago

Actually, GraphicsMagick is known to have inherited some of the security flaws from ImageMagick. For that reason, it is also blocked at our company. I will see if i can write a little wrapper for some basic IO library.

timholy commented 8 years ago

Near the end of a long journey

I'm pleased to announce that the "new Images" is ready for testing. It's not that absolutely everything is done (see below), but I think it would be appropriate for people to start playing with this. For a list of some of the main changes, please see https://github.com/timholy/Images.jl/blob/images-next/NEWS.md.

Documentation

I've put a lot of work into documentation so far, although that effort isn't completely done. The new documentation will be hosted mostly at http://juliaimages.github.io/latest/, which is intended to be "organizational-level" documentation for all of JuliaImages. Before working on that documentation, I'd also written documentation for ImageCore, ImageFiltering, ImageAxes, and ImageMetadata---I'm wondering if this should just be moved to JuliaImages?

I am hoping this will become very polished documentation, competitive with other image processing suites like skimage, OpenCV, etc., but it is not there yet. What is there is supposed to be good, so do read it with a critical eye and please fix/report problems. In my mind the "tutorial" sections are mostly complete, but it's missing important components (like API documentation) and most especially a bunch of examples (e.g., see http://scikit-image.org/docs/dev/auto_examples/index.html). I really hope that users will contribute to the documentation especially by providing examples.

Trying it out

Pkg.update()

Then download this gist (or just copy/paste the file), and say

include("imagebranch.jl")
imagebranch("fixed-renaming")

If you have dirty packages or are on branches, this will almost certainly fail. I have verified that it does work with a fresh pkg folder, e.g.,

tim@cannon:~/src/julia-0.5$ JULIA_PKGDIR=/tmp/tim/pkgs2 ./julia -q
julia> Pkg.init()
INFO: Initializing package repository /tmp/tim/pkgs2/v0.5
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl

julia> Pkg.add("PkgDev")
<some output>

julia> Pkg.add("Images")
<some output>

julia> Pkg.add("ImageMagick")  # plus any others you might need
<some output>

julia> include("/tmp/tim/imagebranch.jl")
INFO: Precompiling module PkgDev.
imagebranch (generic function with 1 method)

julia> imagebranch("fixed-renaming")
<lots of output>

If you find yourself unhappy and want to go back to the original, use

imagebranch("release")

and if you want to try the "intermediate" set (one that doesn't include https://github.com/JuliaMath/FixedPointNumbers.jl/pull/51), use

imagebranch("images-next")

Stuff I'd like feedback on

Please report problems as separate issues (don't just add to this one), but mark it clearly as to which "overarching branch" you're using ("release", "images-next", or "fixed-renaming").

Tips while running old scripts with new code

First, I urge you to use a branch when making any changes to your own internal code; who knows how long it will be before this merges to master, so you might want to be able to easily go back and forth.

Second, don't forget that you can launch julia as julia --depwarn=no and suppress a whole sea of deprecation warnings. That said, the point of the deprecation warnings is to teach you how to migrate your code, so you'll want to do at least enough of this to start discovering problem points, things you like or dislike, etc.

Evizero commented 8 years ago

Thanks! I was able to update using your gist, although I did have to visit all affected package folders and do git fetch --all (which may be an overkill, I know). Otherwise the command imagebranch("fixed-renaming") did not infer which corresponding remote branch to track and threw an error for each package I did not fetch

INFO: Checking out Colors fixed-renaming...
ERROR: GitError(Code:ERROR, Class:Merge, There is no tracking information for the current branch.)
[...]