base pair out of range - Githubissues

cshukai commented 7 years ago

Hello! Thanks so much for this d3-based visualization software. Our group loves it. I downloaded the karyotype (Ensemble) and centromere (UCSC) information for Maize and process the karyotype file in the same way implemented in convert_band_data.py.

But I continue to get the following error message: core.js:991 Uncaught Error: Base pair out of range. bp: 11588371304284; length of chr1: 301433382 at Ideogram.value (core.js:991) at Ideogram.value (core.js:1316) at n (core.js:2657) at core.js:2671

Chromosome length is correctly calculated based the file I provided. Since core.js seems to be loaded from somewhere outside of my server, it is a little difficult to print messages to console to find how it comes up with this huge amount of base pair.

eweitz commented 7 years ago

Hi @cshukai, interesting issue.

I downloaded the karyotype (Ensemble) and centromere (UCSC) for Maize and process the karyotype file in the same way implemented in convert_band_data.py.

Could you attach that processed data? I'd like to be able to reproduce the problem.

Since core.js seems to be loaded from somewhere outside of my server, it is a little difficult to print messages to console to find how it comes up with this huge amount of base pair.

The core.js file is loaded from the same server as ideogram.min.js, but it's loaded via source maps with a non-standard protocol (webpack://).

And although you can't easily print messages from core.js to console, you can set breakpoints in the source mapped code to debug it. Here's a screenshot showing how you can do that with Chrome:

It's worth noting that core.js was recently overhauled; it was renamed to ideogram.js and most of its code is now in smaller modules like coordinate-converters.js. The screenshot above uses the newest release, Ideogram.js v0.11.0.

cshukai commented 7 years ago

I appreciate the help , @eweitz. I am attaching the files including the karyotype, centromere and the array in js file as the input for ideogram program. Archive.zip

Thanks for explaining illustrating webpack! it is very nice to know that!

eweitz commented 7 years ago

@cshukai, I haven't been able to reproduce your problem. I unzipped Archive.zip, copied maize.js into /dist/data/bands/native, and put the following HTML in a file named maize.html in the examples folder.

<html>
<head>
  <script type="text/javascript" src="../dist/js/ideogram.min.js"></script>
</head>
<body>
<script type="text/javascript">
  var config = {
    organism: "maize"
  };
  var ideogram = new Ideogram(config);
</script>
</body>
</html>

The rendering of the maize genome that I see has some peculiarities -- e.g. extremely long chromosomes, no visible banding -- but I don't see the "Base pair out of range" error you reported.

Could you attach a minimal example of the HTML you're embedding your ideogram into, and the Ideogram configuration object you pass into new Ideogram(config)?

I downloaded the karyotype (Ensemble) and centromere (UCSC) information for Maize

Could you elaborate on that? I'm unaware of how to get such data from Ensembl or UCSC for maize. What steps did you take?

If your methodology for getting karyotype and centromere data is applicable beyond maize, then I could likely expand Ideogram.js's first-class support to more organisms, and solve your problem in a generic manner.

cshukai commented 7 years ago

Thanks for looking into that. Here is the html file (modified from annotation.histogram.html in example):

<head>
  <link type="text/css" rel="stylesheet" href="../src/css/examples.css"/>
  <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/cro
ssfilter/1.3.12/crossfilter.min.js"></script>
  <script type="text/javascript" src="../dist/js/ideogram.min.latest.js"></scrip
t>
  <script type="text/javascript" src="../src/js/filter.js"></script>

  <style>
    ul {
      list-style: none;
      padding-left: 10px;
      float: left;
    }

    #gene-type {
      padding-left: 30px;
    }

    .note {
      font-style: italic;
      padding-left: 10px;
      clear: left;
    }
  </style>
</head>
<body>
  <div id="container"></div>
  <script type="text/javascript">

  var counts, key, count;
  var d3 = Ideogram.d3;

  d3.selectAll("input").on("click", function() {
    var filterMap,
        tmp, facet,
        checkedFilter,
        checkedFilters,
        selections = {},
        i, facet, counts, count,
        filterID;

    filterMap = {
      "expression-level": {
        "extremely-high": 7,
        "very-high": 6,
        "high": 5,
        "moderately-high": 4,
        "moderate": 3,
        "low": 2,
        "very-low": 1
      },
      "gene-type": {
        "mrna": 1,
        "misc-rna": 2,
        "mirna": 3,
        "trna": 4,
        "lncrna": 5
      },
      "tissue-type": {
        "cerebral-cortex": 1,
        "heart": 2,
        "liver": 3,
        "skin": 4,
        "skeletal-muscle": 5
      }
    };

    checkedFilters = d3.selectAll("input:checked").nodes();

    for (i = 0; i < checkedFilters.length; i++) {
      tmp = checkedFilters[i].id.split("_");
      facet = tmp[1];
      checkedFilter = tmp[2];

      filterID = filterMap[facet][checkedFilter];
      if (facet in selections == false) {
        selections[facet] = {};
      }
      selections[facet][filterID] = 1;
    }

    counts = ideogram.filterAnnots(selections);

    for (facet in counts) {
      for (i = 0; i < counts[facet].length; i++) {
        count = counts[facet][i];
        key = count.key - 1;
        value = "(" + count.value + ")";

        // document.querySelectorAll("#" + facet + " .count")[key].innerHTML = v
alue;
      }
    }
  });

  var config = {
    container: "#container",
    orientation: "vertical",
    organism: 4577,
    //assembly: "GRCh37",
    chrHeight: 275,
    annotationsPath: "../data/annotations/example.gff.json",
    annotationsLayout: "histogram",
    barWidth: 3
  };

  var ideogram = new Ideogram(config);
  </script>
</body>

Initially, I renamed banana.js to maize.js and the process didn't run into the issue of out of range and chromosome lengths seemed fine.

Maize Karyotype was downloaded from Ensemble'sFTP site(http is used here for markdown to display hyperlink ) and centromere from USCS's table browser. After I have the two files , I labelled each region in karyotype file with either p or q based on their locations relative to corresponding centromere regions.

Thanks again for your prompt replies!

eweitz commented 7 years ago

Maize Karyotype was downloaded from Ensemble'sFTP site(http is used here for markdown to display hyperlink ) and centromere from USCS's table browser.

Very helpful! I will resolve this particular issue with maize as part of generally expanded support for cytogenetic bands.

cshukai commented 7 years ago

Thanks so much !! @eweitz

If there is anything I can help with the expansion, please let me know.

eweitz commented 6 years ago

@cshukai, can you still access the centromere data from Genomaize? I implemented support for maize karyotype and centromere data, and had it working locally, but it has since regressed.

It seems Genomaize updated their backend some time in November, and now I can no longer retrieve centromere data from here. For example, the "centromere" table in Genomaize disappeared, and there is now only a "cytoBandIdeo" table in its place -- but the latter lacks centromere data.

You can see banded Zea mays chromosomes without centromeres in the newly released Ideogram.js 0.12.0: https://eweitz.github.io/ideogram/eukaryotes.html?org=zea-mays.

eweitz commented 6 years ago

@cshukai, your original request has been implemented in Ideogram.js 0.13.0. Because Genomaize centromere data seems to have vanished from their server, I ended up splicing the centromere data that you attached as "Archive.zip" into the band data for Zea mays genome assembly B73 v2.

Screenshot from https://eweitz.github.io/ideogram/eukaryotes.html?org=zea-mays:

eweitz / ideogram

base pair out of range #82