October 26th Deliverables

[mlee156]

Ultimate goal: By Wednesday, have a docker image that launches our web app and displays Ailey's data.

Key To Deliverables: Bold is for Jovo suggested deliverables Italics is for "extra" deliverables if we have time

@lukezhu1 and @alee156 - Try to finish by Monday http://104.196.108.174/

• [x] Import data directly from NdStore NOT local files
• [x] Use Ailey brain that is propagated
• [?] Run our pipeline on the Ailey brain that is propagated and try to interpret the results
  • Sent to Jovo, need to run pipeline on more brains to understand our results
• [ ] Make sure to include a README that points to our website documentation for interpreting the graphs

By Wednesday

• [ ] Eliminate all redundant conversions (like to nifti for example) in order to increase speed

@jonl1096

• [ ] Convert Cherrypy code to Django https://github.com/NeuroDataDesign/seelviz/tree/gh-pages/django/seelviz
• [x] Make front-end for web app look nice and professional (aka not the black and white html garbage we currently have) https://github.com/NeuroDataDesign/seelviz/blob/gh-pages/django/website.png
• [x] Finalize website design for presentation to Ailey
• [x] Resolve naming issues and bugs in clarityviz API
• [ ] Increase speed of the pipeline (time before and after)
• [x] Standardize the plotly style that is used for all graphs
• [ ] Have nice documentation style that Plotly uses to explain our Plotly plots and results

Use https://plot.ly/python/3d-scatter-plots/ as inspiration

@toekneesunshine

• [x] Help John with the webapp frontend
• [x] Convert Nifti images into 3D tiff volumes
  • Script here: https://github.com/NeuroDataDesign/seelviz/blob/gh-pages/Jupyter/NIFTI%20(.nii)%20Ilastik%20Membrane%20Detection.ipynb generates TIFF slices of each Z plane and stores as separate slice. With slices, we can import in ImageJ by importing as a sequence (3D TIFF Volume)
  • These 3D TIFF volumes can be analyzed in Ilastik by importing as a sequence.
• [x] Run Ilastik and analysis on converted 3D tiff images
  • https://github.com/NeuroDataDesign/seelviz/blob/gh-pages/Jupyter/3D%20TIFF%20stack.ipynb
  • Need a better way to select features: currently using pixel based object classifier, which would mean I'd have to manually select every single unique object in the entire 3D TIFF stack.
• [ ] Code.md for the Gulio paper CNN code that we can implement realistically
• [ ] Run above code on one of the Tokens
• [x] Downsample all plotly plots to make them run easier on the website
  1) Finalized style for Plotly by removing ugly export link
  2) Created new downsampled plotlys (5k nodes) on one brain for consistency (used Control 258)
  3) Fixed bug on atlas region HTML improperly rendering a static link for one of the hover texts
  4) Load time cut from around 1 minute to 12 seconds
• [x] Finalize website design for presentation to Ailey
• [x] Make free trial account on readme.io and make our documentation with it
  • Made account, ported our documentation into it
  • https://seelviz.readme.io/docs/clarityviz-tutorial
• [?] Host our website on CIS server
  • Unable to load with proper CSS. Contacted Greg, who gave us access to group billing under Jovo.
  • [ ] \ Host our website on Amazon EC2 and make sure it links to our web app **
  • Since we're still working out the last couple kinks, we haven't deployed to EC2 yet (since it costs money)
  • Using the last remnants of our GoogleCloud compute, we effectively have the Docker running: http://104.196.108.174/, http://104.196.239.231/
  • Once we iron out the last problem, just need to commit the Docker, make appropiately sized EC2, then run the Docker image.
• [x] Cost analysis of running our web app on the ec2 instance
  • https://github.com/NeuroDataDesign/seelviz/blob/gh-pages/reveal/october26stuff/EC2CostAnalysis.pdf

@gur509 and @ykim160

• [?] Modify our pre-existing docker container to launch our web server automatically
  • Luke completed this deliverable
• [?] Copy our pre-existing docker container to another container such that it is able to launch a jupyter server we can connect to with localhost and easily develop with
  • Luke completed this deliverable
• [ ] In Latex organize notes on different convolutional neural network algorithms we can use for our Clarity brain images
• [ ] After web service is finished cross check results for all 15 brains between the web service and the python package
• [ ] After web service is finished cross check results for the propagated Ailey brain between the web service and the python package

[mlee156]

@gkiar @jovo

[jovo]

this is a ton! i recommend prioritizing. in particular, i recommend not doing any CNN work yet.

[mlee156]

@jovo @gkiar

I've updated the deliverables with a different kind of prioritization system where I put stuff in bold if you suggested the deliverable in class (the ones we will prioritize), and stuff in italics that is on our product backlog that is pertinent to our week's activities.

[jovo]

:ok_hand:

On Thu, Oct 20, 2016 at 2:52 AM, alee156 notifications@github.com wrote:

@jovo https://github.com/jovo @gkiar https://github.com/gkiar

I've updated the deliverables with a different kind of prioritization system where I put stuff in bold if you suggested the deliverable in class (the ones we will prioritize), and stuff in italics that is on our product backlog that is pertinent to our week's activities.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NeuroDataDesign/seelviz/issues/25#issuecomment-255023767, or mute the thread https://github.com/notifications/unsubscribe-auth/AACjcnV17SlZuylpRcZrrpGZVKkVhpBVks5q1w-VgaJpZM4KboAq .

the glass is all full: half water, half air. neurodata.io, jovo calendar https://calendar.google.com/calendar/embed?src=joshuav%40gmail.com&ctz=America/New_York

[mlee156]

@jovo I think we're going to need to pivot on the host website on the CIS server: cis.jhu.edu/~alee/ is the website hosted on the CIS server

I think there's a restriction on the website memory because the website's not loading. I tried hosting the website without any CSS styling and it loaded, however a) it looked terrible obviously and b) it was extremely slow, even slower than github hosting.

[jovo]

@gkiar can you give them access to NeuroData AWS account.

@seelviz - use it wisely please, that is, deploy, not test, using real money :)

On Thu, Oct 20, 2016 at 1:06 PM, alee156 notifications@github.com wrote:

@jovo https://github.com/jovo I think we're going to need to pivot on the host website on the CIS server: cis.jhu.edu/~alee/ is the website hosted on the CIS server

I think there's a restriction on the website memory because the website's not loading. I tried hosting the website without any CSS styling and it loaded, however a) it looked terrible obviously and b) it was extremely slow, even slower than github hosting.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NeuroDataDesign/seelviz/issues/25#issuecomment-255167014, or mute the thread https://github.com/notifications/unsubscribe-auth/AACjcvjVkt4CmyMpQKadMfancOmNDTdqks5q15-PgaJpZM4KboAq .

the glass is all full: half water, half air. neurodata.io, jovo calendar https://calendar.google.com/calendar/embed?src=joshuav%40gmail.com&ctz=America/New_York

[mlee156]

@jovo

Greg already gave us access! We're just waiting for Cortex to get set up so that we can test before deploying.

[jovo]

• [ ] why is ilastik working on a cigar shaped data?
• [ ] price analysis?
• [ ] condense tasks
• [x] why is albert wearing a suit?
• [ ] document that it takes 20 minutes (and what all is in pipeline)
• [ ] get people to provide orientations
• [ ] might consider making saving/converting as a nifti image
• [ ] look at more raw data
• [ ] albert sent me some link, what is that about?