search

SimonDanisch / GLPlot.jl

Plotting for Julia with OpenGL
Other
74 stars 8 forks source link

volume.nii #40

Open timholy opened 7 years ago

timholy commented 7 years ago

At one point you mentioned a replacement for ImageView based on GLVisualize; this is of considerable interest to me. I'm guessing that the volume.jl and related examples are what you mean?

One immediate issue: there doesn't appear to be a downloader for volume.nii. I tried searching for it, but the file I found isn't of the right size and consequently I get

ERROR: LoadError: BoundsError: attempt to access 1024×1024×4 Array{Float64,3} at index [Colon(),Colon(),100]
 in throw_boundserror(::Array{Float64,3}, ::Tuple{Colon,Colon,Int64}) at ./abstractarray.jl:363
 in checkbounds at ./abstractarray.jl:292 [inlined]
 in _getindex at ./multidimensional.jl:270 [inlined]
 in getindex(::Array{Float64,3}, ::Colon, ::Colon, ::Int64) at ./abstractarray.jl:760
 in include_from_node1(::String) at ./loading.jl:488
while loading /home/tim/.julia/v0.5/GLPlot/example/volume.jl, in expression starting on line 17

If you can tell me where that file comes from, I can contribute an installer and tweak the paths in the various examples.

timholy commented 7 years ago

BTW, did you see this? http://julialang.org/soc/projects/images.html

I took the liberty of listing you, hope that's OK.

SimonDanisch commented 7 years ago

Excellent!! :) I wanted to do the same, but didn't get to it!

SimonDanisch commented 7 years ago

Are you talking about? https://github.com/SimonDanisch/GLPlot.jl/blob/68c9b217d2cc9b5814ecfd91f60f36f28bb6031f/example/volume.jl ? That's just some random file from a friend who scanned her brain ;) If you have any demo file we can use and automatically download it, that would obviously much better!

timholy commented 7 years ago

I got one from http://www.humanconnectome.org/data/ (a project run out of my home institution). It's 68M, so if we use it there seems to be a hosting issue we'd have to solve. Conceivably we could store it as Float16 (it's now Float32) and at lower resolution (it's now (260,311,260)), but there are of course costs to pay especially in reducing the resolution.

But without dealing with that, I don't even understand what the buttons on the left mean. Aside from rotating the head, how does one interact with the image? Suppose I just want to "play through" each 2d slice? Finally, whatever the code after the gui command is doing, it actually messes up the slice images.

timholy commented 7 years ago

(You need a free account to access data on HCP, so it's not great for automatic downloads.)

timholy commented 7 years ago

Pinging @vanessen in case he has any suggestions. David, this repository is a high-performance OpenGL based visualization suite for Julia. Main page is at https://github.com/JuliaGL/GLVisualize.jl.

SimonDanisch commented 7 years ago

Yeah, that example is a bit buggy, admittedly! I should probably rewrite it to make it work better! But this is how it should look: https://vimeo.com/180307247

timholy commented 7 years ago

That explains it, here's the window I see right after running the example script (switching to the dataset I downloaded from HPC):

image

But prompted by this, I noticed the little scrollbar to the right; once I grab it and move it, the window suddenly reshapes and I see this:

image

The sliders work very well, although I note a few problems (of which some might come from using a different dataset than you):

image

which seems quite different than what you show in that movie.

Despite the list of problems, I am very intrigued; it's quite a functional GUI for ~20 lines of user code! Happy to work with you to try to fix the bugs, but I might need some assistance/guidance since I'm not entirely sure what's intended and what's not.

SimonDanisch commented 7 years ago

Oh yeah please bug me with these before you try to debug something I very well know about! ;) Most of what you describe is the side effect of Plots.jl not working nicely with GLPlot. I think what I will do is to port most of the functionality of Plots to GLPlot, and make it all natively available! Let me see if I can update the example to make it behave a bit better!

SimonDanisch commented 7 years ago

Here you go: 92ffcb6f6884ed322d14e2ad8ddd6950bf155cfc The axes are gone because Plots.jl created them. I need to write up some API to nicely add one (problem: axis should be camera and maybe even content aware!)

trying to expand the topmost control gives me an error in the REPL:

I think this is an old error... Try checking out GLVisualize!

timholy commented 7 years ago

I think what I will do is to port most of the functionality of Plots to GLPlot, and make it all natively available!

I think that's a good idea. Plots is amazing, but since it has to be a wrapper around many toolkits it might (in the long run) hold you back.

And checking out master on GLVisualize and using your new commit: that's a lot better! All of those bugs I mentioned above are gone.

The main remaining problem is (at least for this dataset) the perspective changes in the top-left panel (when I click-drag with mouse) are relative to a corner of the image rather than the center, so the head pivots crazily around the image. I'm also struggling a bit with how to get what I want from the render settings; any documentation on the meaning of the parameters and how they affect the image?

SimonDanisch commented 7 years ago

I hope it helps to play around with it in the edit menu!

I also half heartedly implemented this function:

"""
you can call get_docs exactly like glplot, but get back a printed representation of all parameters.
If you're lucky, they will even be accompanied by some explanation! 
"""
get_docs(rand(Float32, 3,3,3)) # e.g. for getting help with a volume
SimonDanisch commented 7 years ago

as a reference: https://github.com/JuliaGL/GLVisualize.jl/blob/master/src/documentation.jl#L43

SimonDanisch commented 7 years ago

This should loosely map to this macro: https://github.com/JuliaGL/GLVisualize.jl/blob/master/src/visualize/image_like.jl#L47 That macro makes up the default values and as you can see, also allow me to connect the parameter with a doc string.

timholy commented 7 years ago

The main remaining problem is (at least for this dataset) the perspective changes in the top-left panel (when I click-drag with mouse) are relative to a corner of the image rather than the center, so the head pivots crazily around the image.

I should have pointed out that a good way to specify the origin is with "non-1" indexing. In ImageFiltering I now have a centered function that provides default indices:

julia> x = (-9:9)/3;

julia> y = ImageFiltering.centered(exp(-x.^2/2))
OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}} with indices -9:9:
 0.011109 
 0.0285655
 0.0657285
 0.135335 
 0.249352 
 0.411112 
 0.606531 
 0.800737 
 0.945959 
 1.0      
 0.945959 
 0.800737 
 0.606531 
 0.411112 
 0.249352 
 0.135335 
 0.0657285
 0.0285655
 0.011109

One could allow the user to communicate the intended center that way.

I hope it helps to play around with it in the edit menu!

It does, but with the data set I have I'm noting the following:

This is still really cool. How hard would it be to set up perpendicular viewing axes and have the head rotation affect the slice orientation? Would it help to have an AbstractArray type like the SlicedPlane type I mentioned in my JuliaCon talk?

timholy commented 7 years ago

This can be closed, right?