Import neuron-neuron connectivity from FAFB

[dosumis]

Dependency: Add new ROI individuals for FAFB for this?

@jefferis @mmc46: What is the origin of the FAFB region painting? Do they come from ROIs we already have on other templates, or were they independently painted?

[mmc46]

The FAFB neuropils are bridged from JFRC2.

On 8 Dec 2020, at 15:21, David Osumi-Sutherland notifications@github.com wrote:

Dependency: Add new ROI individuals for FAFB for this?

@jefferis https://github.com/jefferis @mmc46 https://github.com/mmc46: What is the origin of the FAFB region painting? Do they come from ROIs we already have on other templates, or were they independently painted?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/VirtualFlyBrain/VFB_connectomics_import/issues/7, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOJBUZRCNBA3UPBHB5GI2TSTZABPANCNFSM4USDXDFA.

[dosumis]

Cool. So we should use those. @admclachlan - you can get these here:

-> download CSV

[admclachlan]

• neuron-neuron through integration layer

[admclachlan]

neuron-neuron connectivity (total weights only) robot template can now be generated c44f03e

[dosumis]

• Alex to work with Clare to specify ROBOT template run in Makefile. This should:
  • generate template
  • use template to generate owl + annotate with iri: http://virtualflybrain.org/data/VFB/OWL/{filename}
  • gzip
• Robbie will use this Makefile for a Jenkins job that generates the file and puts it in the right place to resolve from http://virtualflybrain.org/data/VFB/OWL/
• Manually add the file name to this config: https://github.com/VirtualFlyBrain/vfb-pipeline-collectdata/blob/master/vfb_fullontologies.txt

Clone Jenkins job for each data source - so these can be run only when sources update. This should be written up as an SOP for future reference.

[dosumis]

@mmc46 - currently we only have synapse counts for connections between published neurons. We want to rapidly move to giving some sense of how this relates to total connectivity for a neuron. We plan to do this buy adding columns expressing number outgoing/incoming edges as percentage total edges. Does this sound reasonable to you?

[mmc46]

You would be calculating this from the data you have for those FAFB neurons? There are cases in which not all connectivity is shared, it depends on what the authors want to do. So for those, unless mentioned in the paper, there might not a value of total edges.

[mmc46]

The connectivity options are defined by annotations, but I see those are not being exported, so you can't see them in VFB CATMAID.

[admclachlan]

This would be calculated from the data we have for those FAFB neurons in VFB CATMAID. It looked like at least all of the neurons I looked at had complete total synapse counts so thanks for pointing out that that this may not always be the case. As there are only 16 papers currently loaded into VFB CATMAID, would it be reasonable to manually check if the connectivity provided includes all edges? I suppose this could still be an issue for data from later papers if there is no automatic way of checking.

Alternative would be to make it very clear to users that the connectivity shown is only for the neurons we have.

[mmc46]

How much connectivity is shared is in a meta-annotation for the “Paper: ” annotation. There are other useful ones like doi. I thought they were getting exported but they are not. They will on the next export which will happen soon.

On 25 Jan 2021, at 06:18, Alex McLachlan notifications@github.com wrote:

This would be calculated from the data we have for those FAFB neurons in VFB CATMAID. It looked like at least all of the neurons I looked at had complete total synapse counts so thanks for pointing out that that is not always the case. As there are only 16 papers currently loaded into VFB CATMAID, would it be reasonable to manually check if the connectivity provided includes all edges? I suppose this could still be an issue for data from later papers if there is no automatic way of checking.

Alternative would be to make it very clear to users that the connectivity shown is only for the neurons we have.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/VirtualFlyBrain/VFB_connectomics_import/issues/7#issuecomment-766573559, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOJBUYK2I3653GNPGMAM53S3UENHANCNFSM4USDXDFA.

[mmc46]

@admclachlan once they're in VFB CATMAID you can query for the meta-annotations, something like this for example.

library(catmaid)
library(tidyverse)

pub = catmaid_query_meta_annotations("Published")
pub_annot = mapply(catmaid_get_meta_annotations, annotations = pub$name)

pub_annot.df <- as.data.frame(t(pub_annot))

pub_annot.df %>%
  select(-entities) %>% 
  rownames_to_column(var = "pub_name") %>%
  unnest("annotations") -> pub_annot.df

The connectivity options are: export: intra-connectors-only (option 1) export: intra-connectors-and-placeholders (option 2) export: intra-connectors-and-original-placeholders (option 3) export: no-connectors (only skeletons)

1) Original connectors only exported if within the set of neurons. All others ignored. 2) Original connectors exported within the set of neurons + new placeholder connectors with new IDs and new targets if outside the set of neurons 3) Original connectors exported within the set of neurons + new placeholder connectors with original IDs and original targets (node ID, skeleton ID, neuron ID, neuron name).

[mmc46]

In the next CATMAID import, you'll see each 'Paper: \<citation>' annotation, meta-annotated with annotations that specify the export options for tags, annotations, connectors, timestamp and doi.

Example:

export: annotations
export: tags
export: intra-connectors-only
publication_timestamp: YYYY-MM-DD
publication_link: https://doi.org/<doi>

'publication_timestamp' is usually the publication date, but not always. It is the date of the last change to the data in CATMAID.

In R

catmaid::catmaid_get_meta_annotations("Paper: Hampel and Eichler et al 2020")$annotations
$`16239056`
[1] "export: intra-connectors-only"

$`16858354`
[1] "publication_link: https://doi.org/10.7554/eLife.59976"

$`16858348`
[1] "publication_timestamp: 2020-10-26"

$`15983322`
[1] "export: annotations"

$`16036180`
[1] "export: tags"

$`11476309`
[1] "Published"

$`16830760`
[1] "papers"

FYI: Zheng et al 2020 does not have a doi or timestamp as it is still a preprint. This will be added when it's published.

[admclachlan]

Progress

• [x] ROBOT tsv>OWL
• [x] Python script runners (CATMAID and neuPrint)

[admclachlan]

Data Sprint TODOs

• [x] @admclachlan to post .owl file to robbie for now: File is here.
• [x] @admclachlan + @Robbie1977 to work on setting up Jenkins jobs

[dosumis]

Assume this can now be closed. Please reopen if not.