search

neurodata / ndstore

code for storing neurodata images and image annotations
http://neurodata.io
Apache License 2.0
38 stars 12 forks source link

can we propagate bloss16? #254

Closed jovo closed 8 years ago

jovo commented 8 years ago

i'd like to zoom out and look at it.

@kunallillaney @Aeusman @alexbaden

j6k4m8 commented 8 years ago 
import ndio.remote.neurodata as nd

n = nd()
n.propagate_token('bloss16', 'image')

...possibly?

[edit]: suggesting instead of running in case @kunallillaney or @alexbaden have a good reason why we shouldn't run that

jovo commented 8 years ago

is propagate status currently correct?

alexbaden commented 8 years ago

Yes. Currently all three channels are not propagated. I kicked off propagate for channel0 using the rest interface. It is currently propagating: http://openconnecto.me/ocp/ca/bloss16/channel0/getPropagate/. I believe it's working, based on watching the tables for each resolution fill up as the job progresses. Once it's done and confirmed, we can do the rest.

jovo commented 8 years ago

thank you!!!

"watching the tables" requires logging into server, yah? we have no other way of knowing more detailed information, right?

On Wed, Mar 23, 2016 at 5:04 PM, Alex Baden notifications@github.com wrote:

Yes. Currently all three channels are not propagated. I kicked off propagate for channel0 using the rest interface. It is currently propagating: http://openconnecto.me/ocp/ca/bloss16/channel0/getPropagate/. I believe it's working, based on watching the tables for each resolution fill up as the job progresses. Once it's done and confirmed, we can do the rest.

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/neurodata/ndstore/issues/254#issuecomment-200542668

the glass is all full: half water, half air. neurodata.io

alexbaden commented 8 years ago

Correct. The log rotate is still broken on dsp029 so all the log files are empty. @randalburns and I had talked about implementing a webpage that watches the celery queue and gives status of each job -- I'm still planning on doing that at some point for histograms, propagate, etc.

jovo commented 8 years ago

when i zoom out from http://ix.neurodata.io/project/bloss16/0/5749/3749/0/# i still see nothing :(

kunallillaney commented 8 years ago

There is data at res1. Here is the xy cutout. http://dsp029.pha.jhu.edu/ocp/ca/bloss16/channel0/xy/1/4000,4500/2500,3000/1/ and data at res2 http://dsp029.pha.jhu.edu/ocp/ca/bloss16/channel0/xy/2/2000,2500/1000,1400/1/

jovo commented 8 years ago

@alexbaden can you explain what is going on?

alexbaden commented 8 years ago

Of course -- the black tiles from the prepropagated state were cached in both the tilecache and memcached. I cleared both tilecache and memcached and you can see we're in much better shape now: http://viz.neurodata.io/project/bloss16/xy/1/2898/2039/0/blend:screen/ (that's res 1).

This is all still going to be a little slow because ndv v0.4 pulls the 32-bit mcfc tiles from the tilecache instead of 8-bit "standard" tiles. This means each color is cached separately, and ndv pulls 4x more data than it really needs. A fix for that is coming real soon...

jovo commented 8 years ago

so cool! you can now add blending to the URL!

On Sun, Apr 3, 2016 at 10:21 AM, Alex Baden notifications@github.com wrote:

Of course -- the black tiles from the prepropagated state were cached in both the tilecache and memcached. I cleared both tilecache and memcached and you can see we're in much better shape now: http://viz.neurodata.io/project/bloss16/xy/1/2898/2039/0/blend:screen/ (that's res 1).

This is all still going to be a little slow because ndv v0.4 pulls the 32-bit mcfc tiles from the tilecache instead of 8-bit "standard" tiles. This means each color is cached separately, and ndv pulls 4x more data than it really needs. A fix for that is coming real soon...

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/neurodata/ndstore/issues/254#issuecomment-204984753

the glass is all full: half water, half air. neurodata.io

alexbaden commented 8 years ago

Yah! But, it's still not really public... The names are going to change slightly, but I'll keep screen around and just alias it to what it "should" be (plain old additive blending).

On Sun, Apr 3, 2016 at 19:32 joshua vogelstein notifications@github.com wrote:

so cool! you can now add blending to the URL!

On Sun, Apr 3, 2016 at 10:21 AM, Alex Baden notifications@github.com wrote:

Of course -- the black tiles from the prepropagated state were cached in both the tilecache and memcached. I cleared both tilecache and memcached and you can see we're in much better shape now: http://viz.neurodata.io/project/bloss16/xy/1/2898/2039/0/blend:screen/ (that's res 1).

This is all still going to be a little slow because ndv v0.4 pulls the 32-bit mcfc tiles from the tilecache instead of 8-bit "standard" tiles. This means each color is cached separately, and ndv pulls 4x more data than it really needs. A fix for that is coming real soon...

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/neurodata/ndstore/issues/254#issuecomment-204984753

the glass is all full: half water, half air. neurodata.io

—

You are receiving this because you were mentioned.

Reply to this email directly or view it on GitHub https://github.com/neurodata/ndstore/issues/254#issuecomment-205084509

alexbaden commented 8 years ago

I think this has been resolved. Closing!