search

neurodata / neurodata.io

Homepage for NeuroData
https://neurodata.io
Other
12 stars 17 forks source link

some connectomes missing #91

Closed jovo closed 6 years ago

jovo commented 6 years ago

eg, 2 of the worm ones. the 3rd rat one.

if they are gone, that is ok, we can probably get nearly all of them back, but it would be nice to know which we are missing.

of note, we can remove all the human ones, and provide a link to m2g.io (but keep the description of the brain graph attributes please).

falkben commented 6 years ago

For my own clarity this is on awesomer. @disa-mhembere do you have any idea where these missing graphs are located?

disa-mhembere commented 6 years ago

All the data I know of is here: http://awesome.cs.jhu.edu/graph-services/download/. Do you @jovo see what you're looking for there?

disa-mhembere commented 6 years ago

We had a DOS attack that filled all the messaging buffers taking down the machine -- I've resolved this now, but the download links to data don't work because of the reverse proxy issue from openconnecto.me/graph-services.

The reverse proxy from awsome.cs.jhu.edu/graph-services <-> openconnecto.me/graph-services going down also broke the download links that the services provides users. Is there a reason http://openconnecto.me/graph-services isn't up @falkben @jovo?

jovo commented 6 years ago

@floki - several of links go to a 404 page. for example, the 2nd graph under "Worm". any idea what is going on there?

On Tue, May 1, 2018 at 9:58 AM, Disa Mhembere notifications@github.com wrote:

We had a DOS attack that filled all the messaging buffers taking down the machine -- I've resolved this now, but the download links to data.

The reverse proxy from awsome.cs.jhu.edu/graph-services <-> openconnecto.me/graph-services going down also broke the download links that the services provides users. Is there a reason http://openconnecto.me/graph-services isn't up @falkben https://github.com/falkben @jovo https://github.com/jovo?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/neurodata/neurodata.io/issues/91#issuecomment-385676935, or mute the thread https://github.com/notifications/unsubscribe-auth/AACjcjgqc95lSAdqdsM6SDEDn9rCA48wks5tuGoYgaJpZM4TbIGy .

-- the glass is all full: half water, half air. neurodata.io ps - i am committed to responding to my emails, it often takes about a week. thank you for understanding.

falkben commented 6 years ago

openconnecto.me went down when we prematurely returned one of the DSP nodes.

looks like the page you linked is returning 400 errors for me now (and is also what is linked to on neurodata.io/connectomes -- at least until I transfer the graphs and create a new download page).

disa-mhembere commented 6 years ago

sorry i turned off something to try to stop rabbitmq buffers from filling & that had adverse effects. It all works as expected now other than download location for the result which points to openconnecto.me

falkben commented 6 years ago

It looks like the graphs are present on awesomer on /mnt/nfs/mrocp/data/static/ (even for the ones that have URLs to openconnecto.me). The entire directory is about 6TB (624 files).

I'm planning on copying the ones that are linked on that page to s3 for hosting on neurodata.io.

Should I also backup the other files (ones that aren't linked from the download page)? It would be somewhat pricey to keep them all on s3 if they aren't needed.

jovo commented 6 years ago

i don't know what all the files are. can you send a list or something?

On Tue, May 1, 2018 at 4:38 PM, Ben Falk notifications@github.com wrote:

It looks like the graphs are present on awesomer on /mnt/nfs/mrocp/data/static/ (even for the ones that have URLs to openconnecto.me). The entire directory is about 6TB (624 files).

I'm planning on copying the ones that are linked on that page to s3 for hosting on neurodata.io.

Should I also backup the other files (ones that aren't linked from the download page)? It would be somewhat pricey to keep them all on s3 if they aren't needed.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/neurodata/neurodata.io/issues/91#issuecomment-385782677, or mute the thread https://github.com/notifications/unsubscribe-auth/AACjcnBw0CBD7BWyWVxNEopmpl3m9sB0ks5tuMevgaJpZM4TbIGy .

-- the glass is all full: half water, half air. neurodata.io ps - i am committed to responding to my emails, it often takes about a week. thank you for understanding.

disa-mhembere commented 6 years ago

Let's sit down for 30min on Thursday and take action after? I think I know what needs to go and perhaps we can shut down this last local service and reinstate awesome as a dev machine too ...

jovo commented 6 years ago

ok, lemme know when you think the download links are all working again?

On Tue, May 1, 2018 at 11:26 PM Disa Mhembere notifications@github.com wrote:

Let's sit down for 30min on Thursday and take action after? I think I know what needs to go and perhaps we can shut down this last local service and reinstate awesome as a dev machine too ...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/neurodata/neurodata.io/issues/91#issuecomment-385854411, or mute the thread https://github.com/notifications/unsubscribe-auth/AACjciSHiK9Hvth3na0IsnWbdG4FJcWDks5tuSc1gaJpZM4TbIGy .