Finding a Genetic Map Viewer

[mestato]

We need a genetic map viewer. Lets try to install CMap JS (https://github.com/LegumeFederation/cmap-js, https://legumefederation.org/cmap-js/) and see how it goes, is it generic enough to be used on a different site and production ready?

Testing could start with this chestnut map: https://link.springer.com/article/10.1007/s11295-012-0579-3

[almasaeed2010]

From tests so far we have a few issues with CMap-js: 1- UI interactions are not handled well (glitchy) and errors block the view making the browser hang. 2- Documentation does not include any info on how to configure, add data, data formats that it can accept, etc. Currently we are trying to find a way to convert FPC files to cmap format to see if it'll work with this browser. Even if that works, however, the map doesn't look or feel production ready at this point and there is a concerning warning that they might change a lot of how it currently works.

I am currently researching alternatives

[almasaeed2010]

Configuration documentation: https://github.com/LegumeFederation/cmap-js/blob/master/Configuration.md

[almasaeed2010]

@bradfordcondon found this other map viewer by Legume Federation https://github.com/LegumeFederation/cvitjs

[bradfordcondon]

I dont think this is the right issue but i cant find hte right one:

we tried converting nad loading the physical map: the genetic map is out there, and should be much easier to load in. Find the genetic map instead.

[mestato]

Genetic map publication: https://link.springer.com/article/10.1007/s11295-012-0579-3

Supplemental file with the final integrated map: https://static-content.springer.com/esm/art%3A10.1007%2Fs11295-012-0579-3/MediaObjects/11295_2012_579_MOESM5_ESM.xls

The basic map information for cmap is in the sheet called "Consensus map positions", columns A-C. Any marker named "CmSNP" is a SNP type marker and "CmSI" is an SSR type marker.

On Thu, Apr 5, 2018 at 5:45 PM, Bradford Condon notifications@github.com wrote:

I dont think this is the right issue but i cant find hte right one:

we tried converting nad loading the physical map: the genetic map is out there, and should be much easier to load in. Find the genetic map instead.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/statonlab/hardwoods_site/issues/234#issuecomment-379019699, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfA2gAJFnvVGXnBD1teeDaIL3s_rbz4ks5tllhPgaJpZM4TFKcG .

-- Margaret Staton Assistant Professor Department of Entomology and Plant Pathology 370 PBB, 2505 EJ Chapman Drive Knoxville, TN 37996-4560

864-506-4515 Mobile mstaton1@utk.edu

[bradfordcondon]

Thanks! Additional info for cmap format: as before, the anchor start will always be 0 and the map end will be the max value for that map (LG). The feature start and end will always be equal.

[bradfordcondon]

Here is my first draft of the chestnut map in cmap format. for loading into chado, i think that we'll have accession be for the feature uniquename, and name for feature name

map_acc	map_name	map_start	map_stop	feature_name	feature_accession	feature_aliases	feature_start	feature_stop	feature_type_acc	is_landmark
C_mollisima_A	A	0	90.4	CmSI0385	CmSI0385		0	0	SSR	0

So for example, the map feature is I (linkage group I), its uniquename will be C_mollisima_I. The feature is CmSNP01167 , uniquename C_mollisima_I_CmSNP01167

Note: last column is used to calculate something, its not part of cmap...

https://docs.google.com/spreadsheets/d/1smdWoqjqsls2itByeSkaS1PHap_qd11iGN1Wmo7pZT0/edit?usp=sharing

[bradfordcondon]

Here's the general state of the cmap viewer.

We can load our map, displaying snps/SSRs. We can configure linkouts for those markers back to chado when we're ready for that.

I think for the UI, it's kind of clunky. How would we want to display the multiple chromosomes?

Right now we add them all pre-loaded, and the user can scroll left/right to see them. Or user can click "add/remove map" buttons to add/remove chromosomes from view.

Navigating around is pretty buggy right now.

I am thinking what might be best is if we could design the field to have a dropdown to select a isngle chromosome. Then ti loads a locked cmap interface showing that chromosome only, with no option to add/remove map (they do that via drupal dropdown instead). Alternatively we could rename the buttons to say "add/remove linkage group" etc...

[bradfordcondon]

preview of cmap map viewer is up at http://160.36.205.61:8095/content/cmaptest we stuck it in an iframe for now because CSS is not namespaced.

[mestato]

I agree the UI is clunky (well downright unuseable in some ways). I cant' figure out how to scroll left or right with a mouse instead of a trackpad, somehow I'm doing crazy things to the whole image, not each chromosome.

Unless you all have some insights into this behavior I can post to cmap-js issue queue.

[almasaeed2010]

Yeah the Ui needs work but to scroll you can drag left and right (just above the boxes that say A, B, etc)

There obviously needs to be some sort of UI indicator that you can drag

[bradfordcondon]

scroll left right: click drag in JUST the riht spot.

Zoom issue : known bug, we have an issue.

we could probably hold a pow wow to talk more but you can see the suggestions/requests i made here https://github.com/LegumeFederation/cmap-js/issues/91

[bradfordcondon]

reviewed in meeting. We will not be loading into chado right now unless theres another reason to do so.

We will not be using cmap-js yet until the interface improves a bit.

[bradfordcondon]

let's test the dev branch, they've changed the UI https://github.com/LegumeFederation/cmap-js/tree/develop

[almasaeed2010]

I tested the develop branch today and it seems that it didn't get any UI updates yet. Also updating with npm breaks the site so couldn't test much.

[bradfordcondon]

ok, i've got the first mview working. no additional module changes were necessary. heres the field on the MAP bundle. (featuremap). We would likely want to attach it to ORGANISM instead, which we could contribute easily.

clicking on a chromosome brings us to this page

[almasaeed2010]

Submitted a PR to fix linkouts to unpublished features: https://gitlab.com/statonlab/TripalMap/merge_requests/2

[bradfordcondon]

https://gitlab.com/statonlab/TripalMap/merge_requests/7

PR (to ours, not to kathryn's) that will attach the featuremap field to organism. With this PR, we are clear to install and load onto the dev site.

Possible outstanding issues:

• QTL data/mview - we dont have it, dont use it. for now, we shuold just leave the mview unchanged (or not install it?)
• tripal_ds creates multiple 'x' boxes on the organism attached field when you use the dropdown.

[bradfordcondon]

Steps to load:

dev

• [x] Update tripal_cmap_loader

• [x] attach featuremap_type field chadoprop, set to required, enable

• [x] create featuremap

  • https://hardwoods.ag.utk.edu/Map/2192028
  • ~~also add publication? ~~
  • if i do that, also please update info on cmap_loader about the source data at https://raw.githubusercontent.com/statonlab/tripal_cmap_loader/master/example/c_mollisima_example.cmap

• [x] load cmap data (chinese chestnut).

  • job at https://hardwoods.ag.utk.edu/admin/tripal/tripal_jobs/view/597964

• [x] download and enable tripalmap

• [x] populate mview

• [ ] attach the map field to organism, configure display

[bradfordcondon]

job status is error because of tirpal bug. (see log below).

However, the job did in fact add the data: see here

select * from chado.feature where type_id = 507;
feature_id  dbxref_id   organism_id name    uniquename  residues    seqlen  md5checksum type_id is_analysis is_obsolete timeaccessioned timelastmodified
4881467     37  L   C_mollisima_L               507 f   f   2018-08-30 10:36:56.210779  2018-08-30 10:36:56.210779
4881292     37  K   C_mollisima_K               507 f   f   2018-08-30 10:36:56.210779  2018-08-30 10:36:56.210779
4881097     37  J   C_mollisima_J               507 f   f   2018-08-30 10:36:56.210779  2018-08-30 10:36:56.210779
4880908     37  I   C_mollisima_I               507 f   f   2018-08-30 10:36:56.210779  2018-08-30 10:36:56.210779
4880677     37  H   C_mollisima_H               507 f   f   2018-08-30 10:36:56.210779  2018-08-30 10:36:56.210779
4880526     37  G   C_mollisima_G               507 f   f   2018-08-30 10:36:56.210779  2018-08-30 10:36:56.210779
4880407     37  F   C_mollisima_F               507 f   f   2018-08-30 10:36:56.210779  2018-08-30 10:36:56.210779
4880180     37  E   C_mollisima_E               507 f   f   2018-08-30 10:36:56.210779  2018-08-30 10:36:56.210779
4879979     37  D   C_mollisima_D               507 f   f   2018-08-30 10:36:56.210779  2018-08-30 10:36:56.210779
4879804     37  C   C_mollisima_C               507 f   f   2018-08-30 10:36:56.210779  2018-08-30 10:36:56.210779
4879631     37  B   C_mollisima_B               507 f   f   2018-08-30 10:36:56.210779  2018-08-30 10:36:56.210779
4879322     37  A   C_mollisima_A               507 f   f   2018-08-30 10:36:56.210779  2018-08-30 10:36:56.210779
(12 rows)

Running 'Chado Cmap loader' importer
NOTE: Loading of file is performed using a database transaction.
If it fails or is terminated prematurely then all insertions and
updates are rolled back and will not be found in the database

processing marker 0processing marker 1000
Done.

Remapping Chado Controlled vocabularies to Tripal Terms...
Done.
WD php: Error: Call to a member function fetchObject() on null in tripal_chado_bundle_get_properties() [error]
(line 2820 of /home/www/sites/all/modules/tripal/tripal_chado/includes/tripal_chado.fields.inc).
Error: Call to a member function fetchObject() on null in tripal_chado_bundle_get_properties() (line 2820 of /home/www/sites/all/modules/tripal/tripal_chado/includes/tripal_chado.fields.inc).
Drush command terminated abnormally due to an unrecoverable error.

[bradfordcondon]

mview populated:

[bradfordcondon]

need to decide waht to do about the old fields:

simple field that links to the featuremap entry.

(its above the chromosome)

[bradfordcondon]

its up!

https://hardwoods.ag.utk.edu/mapviewer/1/E https://hardwoods.ag.utk.edu/bio_data/21?tripal_pane=group_physical_and_genetic_map

Issues resolved (see below)

• The map detail button links here, which crashes it: https://hardwoods.ag.utk.edu/tripalmap_featuremap/1

Why wouldnt it instead link to the node or entity?

• The "Disease resistant QTL" organism shows up in the dropdown. why?

I used the existing featuremap record, so i'd bet i linked both organisms to the featuremap (since thats what this map is, i think?).

[bradfordcondon]

fixed the map button crashing. it just reroutes you to https://hardwoods.ag.utk.edu/Map/2192028 anyway

Chinese chestnut QTL is, indeed, set via the featuremap_organism linker. Probably fine that way.

[bradfordcondon]

• For now, wait until our fork is integrated back into the main one before we go live with this module.

[bradfordcondon]

this issue is waiting on the changes made to fix the tripalmap module to be re-applied to the second repo. If this doesnt happen in a few weeks ill reconnect with Katheryn and offer to do it myself.

[bradfordcondon]

    $sql = "
  SELECT F.uniquename as marker_locus_name, F.feature_id as marker_locus_id, F2.uniquename as genetic_marker_name,
  C1.name as map_unit_type, C2.name as marker_type, FM.name as map_name, FM.featuremap_id as map_id, FMP.value as map_type,
  F3.name as linkage_group, F3.feature_id as linkage_group_id, FP.mappos as marker_pos, FPP.value as marker_pos_type,
  O.organism_id as organism_id, O.genus as genus, O.species as species, O.common_name as common_name
  FROM {feature} F
  INNER JOIN {feature_relationship} FR  ON FR.subject_id = F.feature_id AND
    F.type_id = (SELECT cvterm_id  FROM {cvterm} WHERE name = 'biological_region' AND
    cv_id = (SELECT cv_id FROM {cv} WHERE name = 'sequence'))
     AND
    FR.type_id = (SELECT cvterm_id  FROM {cvterm} WHERE {cvterm}.name = 'instance_of' AND
    cv_id = (SELECT cv_id FROM {cv} WHERE name = 'OBO_REL'))

  INNER JOIN {feature} F2                   ON FR.object_id = F2.feature_id 
     AND
    FR.type_id = (SELECT cvterm_id FROM {cvterm} WHERE name = 'instance_of' AND
    cv_id = (SELECT cv_id FROM {cv} WHERE name = 'OBO_REL'))

  INNER JOIN {featurepos} FP                ON F2.feature_id = FP.feature_id

  INNER JOIN {featuremap} FM            ON FM.featuremap_id = FP.featuremap_id
  INNER JOIN {cvterm} C1                ON C1.cvterm_id = FM.unittype_id
    INNER JOIN {cvterm} C2 ON C2.cvterm_id = F2.type_id
  INNER JOIN {featuremapprop} FMP       ON FMP.featuremap_id = FP.featuremap_id AND
   FMP.type_id = (SELECT cvterm_id FROM {cvterm} WHERE name = 'featuremap_type' AND
   cv_id = (SELECT cv_id FROM {cv} WHERE name = 'local'))
  INNER JOIN {featuremap_organism} FMO  ON FMO.featuremap_id = FM.featuremap_id
  INNER JOIN {feature} F3               ON FP.map_feature_id = F3.feature_id
  INNER JOIN {featureposprop} FPP       ON FPP.featurepos_id = FP.featurepos_id
  INNER JOIN {cvterm} C                 ON C.cvterm_id = FPP.type_id
  INNER JOIN {organism} O               ON FMO.organism_id = O.organism_id
  ";  }

heres the modified cmap.js mview. reproducing it here an elsewhere so it wont be lost as time marches on.

[almasaeed2010]

@jwest60 here is where our fork of tripal map lives: https://gitlab.com/statonlab/TripalMap/merge_requests/7

Let's go live with this fork. Of course test it on main dev first.

[bradfordcondon]

I have merged my pending PRs into our fork. We will use that fork on live and dev.

https://gitlab.com/statonlab/tripal_map

I suggest having that be the origin remote to keep ti simple until we decide we can use the main fork.

[bradfordcondon]

i have mapped the QTL mview.

QTLs do NOT show up as ranges on the map.

The start/stop is NOT in the mview. we therefore need to change the code to get the start/stop populated correctly

[almasaeed2010]

And we are live!

[almasaeed2010]

Live and dev sites have been updated to use the new QTL mview