NF: Process TRACULA data into a nodes table.

[arokem]

Addresses #174

Partially addresses #163

Still remaining: streamline representation.

[arokem]

OK - I believe that this is almost ready here, but there are a couple of things we need to fix (possibly on the JS side) to make this work, and I wanted to ask you all to weigh in, before we merge this:

• When processing a TRACULA data-set, I now create a streamlines.json file. I think that this should replace the afq_streamlines.json file (i.e. here: https://github.com/yeatmanlab/AFQ-Browser/blob/master/afqbrowser/site/client/js/3d-brain.js#L222), but I wanted to make sure that I am not missing any implications of that.

• I couldn't get the website to work with the streamlines.json output produced here. I suspect this has to do with the fact that each streamline only has one key ("coreFiber") and only one streamline. The error produced in the JS console is:

Uncaught TypeError: Cannot read property 'x' of undefined
    at q.subVectors (three.min.js:303)
    at R.computeFaceNormals (three.min.js:497)
    at R.fromBufferGeometry (three.min.js:495)
    at 3d-brain.js:296
    at Array.forEach (<anonymous>)
    at Object.success (3d-brain.js:223)
    at fire (jquery-1.12.4.js:3232)
    at Object.fireWith [as resolveWith] (jquery-1.12.4.js:3362)
    at done (jquery-1.12.4.js:9840)
    at XMLHttpRequest.callback (jquery-1.12.4.js:10311)

It could also have to do with the nodes.csv file. I will double-check that one as well.

[richford]

@arokem, Did you test this change against master or against PR #200. It should break master but it should work with #200.

[arokem]

Oh - good call. Against master.

Looks like #200 is ready to merge anyway?

I am still working on this (found a bug in the way that nodes.csv was constructed...), but will test again once we merge #200.

[richford]

Yeah, I think #200 is ready.

[arokem]

OK - I think that this is done on my side. Website still doesn't work, but I think that we will have to sit together and look at the interaction between the data that gets generated from TRACULA and the way that the JS code reads and processes the data to understand why it's not working.

[richford]

I pulled this branch and it worked on my end. We'll can troubleshoot at our meeting next week.

[arokem]

Did you try it with TRACULA data, by any chance? That's where things don't work for me.

On Sat, Dec 2, 2017 at 3:14 AM, Adam Richie-Halford < notifications@github.com> wrote:

I pulled this branch and it worked on my end. We'll can troubleshoot at our meeting next week.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/yeatmanlab/AFQ-Browser/pull/181#issuecomment-348685184, or mute the thread https://github.com/notifications/unsubscribe-auth/AAHPNnYvozLdaRIHe2VPRtMMjvj25QZrks5s8TEEgaJpZM4QRyDp .

[richford]

@arokem, Ahh, now I see. I tested by replacing streamlines.json with tracula_streamlines.json in 3d-brain.js and reproduced your error. nodes.csv and streamlines.json should have the same bundle names but for me, nodes.csv still had the old AFQ-generated names.

[arokem]

That should be handled by afqbrowser-assemble. Here

[arokem]

Sorry. Misfired there...

[arokem]

As I was saying... that should be handled by afqbrowser-assemble. I am thinking of a way to share that data with you.

[arokem]

OK - there it is. I add the nodes/subjects/streamlines data that get generated from TRACULA's stats folder by afqbrowser-assemble on this branch, as a sub-folder in the data folder.

[richford]

@arokem, I submitted a PR to your branch that fixes some of these issues. We still have #196 to contend with, but I think my PR will solve all the other problems. I believe @anotherjoshsmith has a fix for #196 as well.

[arokem]

Getting closer! Linking between the bundle name list, the 3D viz and the tract details plots seems to work well now.

Things that still don't seem to work:

• I can't highlight individuals subjects' data: neither in the plots, nor in the table.

• Apart from the x axis issue, related to #196, there are also some additional weirdnesses with the tract profile plots. In particular, there are some additional lines, that I don't believe should be there. See diagonal straight lines in these images:

[richford]

Just submitted another PR to your branch that fixes the selection issue. I'm at a loss for the double lines. One thing I did notice is that the tracula nodes.csv files has redundant lines. It looks like there are 8 copies of each data point in there. I was hoping that some of these were different and that was causing the plotting error, but they all appear to be the same. Regardless, does this indicate that the assemble command is appending to nodes.csv instead of writing?

[arokem]

Ah yes. I see that as well.

As far as I can tell, that is a bug in Pandas (!), but I am still trying to sort that one out.

[arokem]

Pfft. "Bug in Pandas". I should eat my hat.

[arokem]

And those weird additional lines are now also gone! I propose that we merge this, and then solve #196. But it can certainly wait until our meeting tomorrow.