interactive highlighting + peak model display?

[tdhock]

Hi @cmdcolin this is great work! Two questions:

(1) is it possible for the user to interactively add/update highlights? (as in apollo)

(2) is it possible to display a peak model on top of the data as well?

I would like to do something like this:

For context I am a machine learning researcher, and I would like to be able to use this kind of track to (1) gather labels (highlights) from genomic researchers, and (2) display a machine learning model that is trained based on those labels.

I would imagine that such a track would send the interactively created labels (highlights) to my central machine learning server, which would also host bigWig/bigBed files for displaying the labels and models.

[tdhock]

Sample data files to display: Labels: https://rcdata.nau.edu/genomic-ml/PeakSegFPOP/labels/H3K4me3_TDH_immune/samples/bcell/McGill0091/labels.bigBed Data: https://rcdata.nau.edu/genomic-ml/PeakSegFPOP/labels/H3K4me3_TDH_immune/samples/bcell/McGill0091/coverage.bigWig Peak model: https://rcdata.nau.edu/genomic-ml/PeakSegFPOP/labels/H3K4me3_TDH_immune/samples/bcell/McGill0091/joint_peaks.bigWig

in chr11:118092641-118124175 the track should look similar to this plot which I made in R/ggplot2

FYI this is one data set from this track hub, https://rcdata.nau.edu/genomic-ml/PeakSegFPOP/labels/H3K4me3_TDH_immune/hub.txt which is defined for hg19

[cmdcolin]

Hi there, I just came back to this issue. I thought I'd just say I think it's a very cool application. I don't necessarily have any updates but I can try and keep this type of thing in my head. I am imagining (a) displaying two bigwigs on the same track is probably the domain of multibigwig for jbrowse 1 at least, so this plugin could be made compatible with multibigwig (b) it appears making dynamic coloring would be useful (c) it appears making interactive highlights would be useful

[tdhock]

hey @cmdcolin thanks for the update. Glad to hear that you think its a cool application. I actually have a team of undergraduates working on this project, so maybe they can help with development / you can help them by telling them what parts of the code need to be modified in order to get this to work?

[cmdcolin]

@tdhock very cool. I'd be happy to help and get them setup and oriented with the code. if you want to arrange a meeting let me know

[cmdcolin]

first three subsections of https://jbrowse.org/docs/faq.html will get you setup with the "development environment" but I'm sure I could provide some other helpful tips too

[tdhock]

my students @81698 @yf6666 @John-C-Jackson

[tdhock]

maybe look how they implement editing in https://github.com/GMOD/Apollo

[cmdcolin]

Glad to see continued interest in this! I would just say, probably that the Apollo system is quite complex as a reference example. I worked on it for my past job and I always wanted apollo to be simple but it's a very heavy tech stack :) If you want to integrate with apollo specifically as a goal, let me know, but I'd recommend to make it a sort of lightweight approach

To start, design what you want your "backend" to look like.

In javascript land, this would probably involve making a express.js type server. For python you can use django maybe, or another microserver type thing. For other languages, just look at things that can make a little microservice probably.

Then your microservice can respond to requests like

POST http://yourserver/add_new_highlight/?refname=chr1&start=1000&end=2000

Your simple server would then get that and save it in a database e.g. postgres or mysql

Then you could also say

GET http://yourserver/get_highlights/?refname=chr1&start=1000&end=2000 and it would query give you any highlights that overlap this region

After this exists, then you can start to integrate jbrowse and the wigglehighlighter into the picture. JBrowse has a native little highlight tool and you can listen to the events that it produces, and then make a highlight post to your server in response.

If you'd like to schedule a little meeting let me know :)

[yf6666]

@Jacob-D-Christiansen

[tdhock]

thanks for the info @cmdcolin ! it is helpful to know that apollo is relatively large/complex system that would be difficult to modify. it may still be helpful to look at apollo just to see how the interactive editing / display is implemented.

also super helpful to know that we can listen to the events in the native jbrowse highlight tool, and produce a POST in response. that is exactly what we want to do I think!

So it is not necessary to use bigBed/bigWig for display on jbrowse? I was under the impression that we needed to put the data in bigBed/bigWig in order to display something on Jbrowse. If the answer is that we don't need bigBed/bigWig files then I would suggest using bigWig only for the raw coverage data, and then using these custom GET/POST requests for the other two data types we want to display (labels/highlights and peak models).

the GET/POST functionality you describe is already implemented in my previous https://github.com/tdhock/SegAnnDB system using Python/Pyramid/BerkeleyDB. (you may consider using that as a starting point for the server)

• add_region view (equivalent of your POST) adds a label to the database https://github.com/tdhock/SegAnnDB/blob/master/plotter/views.py#L258
• initial view is equivalent of your GET https://github.com/tdhock/SegAnnDB/blob/master/plotter/views.py#L470 Note that these two views send updated segmentation models as well as labels. And they do it for a given user/sample/chromosome combination (not chrom sub-region) so you would have to modify it for requesting chrom sub-regions.

[cmdcolin]

So it is not necessary to use bigBed/bigWig for display on jbrowse? I was under the impression that we needed to put the data in bigBed/bigWig in order to display something on Jbrowse. If the answer is that we don't need bigBed/bigWig files then I would suggest using bigWig only for the raw coverage data, and then using these custom GET/POST requests for the other two data types we want to display (labels/highlights and peak models).

I went ahead and updated this plugin so it can get highlights from a generic backend, not just bigbed. It was a silly limitation to just be bigbed, so now highlights can come from any store backend. If you match the JBrowse REST API format your storeClass could be "JBrowse/Store/SeqFeature/REST"

The minimal return for JBrowse/Store/SeqFeature/REST is something like

{ "features": [ {"start": 100, "end": 200, "seq_id": "chr1"}] }

So if your REST API returned data in that format, you can use that storeClass. Otherwise, might make another data adapter for your custom REST API (not too hard, many plugins make their own custom data adapters)

[cmdcolin]

The README has some updates on how to use generic backend

[cmdcolin]

As far as some of the specifics of the app, if it's not giant data, it's probably fine to design a system that just requests the whole chromosome at once for those annotations, it doesn't strictly have to request the subregions. Then there will need to be some work on the interactivity. Here is a kind of dense page on the event system http://jbrowse.org/docs/events.html

You can subscribe to highlight changes, or you can do some more involved code to make a custom highlighter code. But the point I wanted to make is that after you receive a highlight event, you can post to the database and then probably just re-get all the annotations and force a track redraw.

[tdhock]

thanks for the info @cmdcolin ! that is super helpful.

so about "re-get all the annotations and force a track redraw" I guess that is one (inefficient) option. would it be possible to do selective redrawing? for example after the highlight event we don't need to ask for an updated set of annotations/labels. we would just need to ask the server for an updated model to display. in particular it would definitely be more efficient if we did not have to re-draw the coverage/bigWig part of the track.

also on second thought we should consider bigBed/bigWig as storage backends for the labels/models. after the add label highlight event we could store the new label in a bigBed file, and update the corresponding model in the bigWig file. This would probably be more efficient for zoomed out display (e.g. an entire chromosome, when the number of pixels on screen is less than the number of features/labels/peaks to display), right?

[Jacob-D-Christiansen]

Hello @cmdcolin sorry for the pause in contact we just got back to school from our winter break and we are excited to continue work on this project. I have not posted here before I am Jacob Christiansen and I am the undergraduate team lead for the group working with Dr Hocking on this project. I have two quick questions for you. The first is when I input a bed file that has color information it shows all of the highlights as the same color and not the information of the file. The second question is we are planning on trying to combine this plugin with another plugin called MultiBigWig. This is so we can show the original data as a bigwig file, the model we would calculate as a bigwig file, and the highlights that the user gives, all on the same track. I was wondering what thoughts you might have on that, as well as if you have any resources you could point us towards for working with these plugins. If you don't That is alright just wondering

Thank you please respond at your earliest convenience

[tdhock]

hey @Jacob-D-Christiansen can you please use a question mark (?) when you ask a question? (so that it is easier to understand what needs to be responded to)

[cmdcolin]

The first is when I input a bed file that has color information it shows all of the highlights as the same color and not the information of the file.

JBrowse actually doesn't have the method of reading the color from a BED file. Generally this is done via color callbacks but this actually isn't implemented in this plugin yet. This plugin could add both color callback and color from BED file support

The second question is we are planning on trying to combine this plugin with another plugin called MultiBigWig

The functionality of this plugin actually depends on overriding a function called postDraw that only exists on some track types. Wiggle type is one of them. We can most likely add it to MultiBigWig I think!

[cmdcolin]

I can try to help guide investigation into this and/or just implement a couple features. You can let me know if you're interested in digging into the guts of it?

[Jacob-D-Christiansen]

sorry for the delay in a response, but yes any help that you are willing to give is greatly appreciated. As of right now we are digging in to the documentation for the jbrowse library commands and also Dojo so excuse us if some of these questions are a bit surface level.

I do have some more questions for you about some of what we want to implement in to jbrowse. The first couple are for allowing big bed color informaiton to be red in to this plugin. The second is about joining this plugin with multibigwig on to one track.

For reading in the color information in the big bed file could you explain a bit more on what you mean by color callbacks? if you could implement the feature to read in the color information for this plugin that would be fantastic. However if you don't have the time or simply don't want to, we are happy to do that, how would you recommend going about doing that?

For implementing this kind of highlighting on to multibigwig you say we need to override a function called post draw. Could you quickly go over what this function does and how you use it to get this effect?

[Jacob-D-Christiansen]

Also we are working on the highlight tool currently as well. You spoke of having a listener for the built in highlight tool of jbrowse. Would this be done in a similar way as changing the left click behavior that is talked about in the documentation page or were you pictureing something else?

[cmdcolin]

I did a couple updates

1) I added callback functionality for the highlightColor and indicatorColor 2) I added an example to make this plugin work with multibigwig 3) Here is how some of the events work https://github.com/GMOD/jbrowse/blob/master/plugins/DebugEvents/js/main.js the globalHighlightChanged event can give you the highlight events

[cmdcolin]

Please see https://github.com/cmdcolin/wigglehighlighter/blob/master/test/data/tracks.conf for some configurations, the callback exists on highlightColor and shows how you can use a function to return the color of a particular highlight. You could use for example

return feature.get('type')==='PEAK'?'red':'blue'

I can try and advise more on the color callback if it helps

[Jacob-D-Christiansen]

My goodness thank you very much you work very quickly. We will start going over everything you went over in your last reply and Im sure we will have a question or two. Thank you very much for your help

[cmdcolin]

Any updates? Happy to answer questions

[tdhock]

yesterday we were talking about highlighting in the context of multiple tracks, say 10-20 samples on screen at once. we want to be able to draw a rectangle and then quickly assign labels to several samples, perhaps of different label type for each sample, depending on whether or not that sample has a peak in the highlighted region. any ideas about how to implement that?

[cmdcolin]

Ah that sounds cool. I guess what I am imagining is that you could have a event handler on the globalHighlightChanged event, which would make a popup that has a little box like

[x]------------------
| Region selected: chr1:200,000-210,000
| Track 1: [ ] peak?
| Track 2: [ ] peak?
| Track 3: [ ] peak?
|   [ok]   [cancel]
--------------------------------

You can grab the list of currently visible tracks from window.JBrowse.view.tracks (that is a global object)

[tdhock]

that indirect manipulation interface would be ok, but I think it would be even better if there was some way we could get an direct manipulation interface, e.g. click on the tracks themselves and then you immediately see the color of the rectangle on that track change (as well as a textual label which says the label type).. possible?

[cmdcolin]

Oh right. No textual labels exist on the highlights right now. Prolly gotta add that

[cmdcolin]

As far as direct manipulation I guess I don't see the exact work flow,would it be highlight and then label them individually with a click action? What state are they in before they are labelled in that case

[tdhock]

yes textual labels would be helpful. yes I think highlight and then label them individually with a click action. I imagine that state of each sample should be unlabeled by default.

1. click labeled sample -> change to next possible label type (noPeaks, peakStart, peakEnd).
2. click unlabeled sample -> assign last label type that was assigned. (that way you can quickly and easily assign lots of the same label to multiple samples) i'm open to other options/suggestions but in my opinion a simple direct manipulation interface like this would be simplest/easiest/quickest.

[Jacob-D-Christiansen]

Hello,

Very sorry about the delayed response, our school scheduled has picked up and we were busy with other things. However we have gotten a basic highlighter tool working, as Dr Hocking said, that allows the user to be able to put in two kinds of labels as of right now. We wanted to get something working so that we could start the conversation between the browser and server. Now we have a couple of questions.

The first question is that we are getting an odd error. Essentially it is saying it can not find a JSON file in the names directory in the data folder with the conf file of our tracks. Do you know anything about this kind of error? My first guess, and this may be way off, but you have implemented a new feature for the wiggle highlighter track. From what I have read it seems we may need to run the "flatline-to-json.pl" program, to get jbrowse to understand the new feature of the wiggle track. Do you think that that sounds reasonable?

The next question is for the next step of the highlight tool. Dr Hocking is wondering if there is a way to have a popup sub window with the the highlight options show up on the user using the highlight tool. I know this is possible with creating a function inside of the conf file for the track with onClick() or something but currently the listener for the highlight tool is in the main file of wiggle highlighter. My question is is there away to make an iFrame from inside of the main.js file of wiggle highlighter or is there some way to move the highlight listener to the conf file?

The last question is for the color callback system for the highlights. We need to have each highlight be able to be a one of four colors independent of one another. I have not had a chance to play around with the feature you implemented but is that something that it would be able to do as of right now? additionally will we have to talk to the server in order to get any of the information from the file itself, ie how many labels there are and what each label is, or is there some way to do this completely on the browser?

We are just looking for some general impressions for these questions. We have already began work with experimenting with each of these we were just seeing if you could point us in the right direction. Thank you so much for your help so far and your continued interest in the project.

[cmdcolin]

The first question is that we are getting an odd error. Essentially it is saying it can not find a JSON file in the names directory in the data folder with the conf file of our tracks.

If it is an error for root.json that just means you have not run generate-names.pl. Our documentation says "Note that if you are getting 404 errors for names/root.json then JBrowse is falling back to the legacy names store (and failing) so it is likely that you need to retry generate-names." A tracks.conf file 404 can also be ignored probably.

My question is is there away to make an iFrame from inside of the main.js file of wiggle highlighter or is there some way to move the highlight listener to the conf file?

I think that this logic will be custom and requires some custom code, e.g. using dojo to make a popup, or something like this. I am not exactly seeing the workflow. The workflow that was mentioned by Dr. Hocking in the above comments seems pretty cool and didn't seem like it would require iframes.

The last question is for the color callback system for the highlights. We need to have each highlight be able to be a one of four colors independent of one another. I have not had a chance to play around with the feature you implemented but is that something that it would be able to do as of right now? additionally will we have to talk to the server in order to get any of the information from the file itself, ie how many labels there are and what each label is, or is there some way to do this completely on the browser?

This seems pretty much in the realm of custom logic for your app. The code for wigglehighlighter allows for color callbacks, but this is a little difficult to make a conclusive technical answer because it relies on some unknowns about your system

If you want any more help let me know and we can schedule a call sometime :)

[tdhock]

please avoid popups/iframes and other indirect manipulation interfaces if at all possible.

[cmdcolin]

Adding labels to each highlight may be complicated. It may be helpful to just have a legend or something if that works. Let me know if there are any other thoughts

[tdhock]

A legend is OK if text for each label rectangle is too complicated (but I think text above/below/inside each colored rectangle would be more user-friendly).

[cmdcolin]

I created a demo that allows clicking on wigglehighlighter peaks to toggle through some "states"

https://github.com/cmdcolin/interactivepeakannotator

It is pretty bare bones and uses browser localstorage for storing the "state" of the peaks

[cmdcolin]

The main body of the code is this store which returns features from localStorage as the highlight features https://github.com/cmdcolin/interactivepeakannotator/blob/master/js/Store/SeqFeature/Features.js

Then this track type which colors and has an onHighlightClick method for toggling states https://github.com/cmdcolin/interactivepeakannotator/blob/master/js/View/Track/XYPlot.js

And then this data directory configuration https://github.com/cmdcolin/interactivepeakannotator/blob/master/test/data/tracks.conf

[cmdcolin]

And https://github.com/cmdcolin/interactivepeakannotator/blob/master/js/main.js for the highlight event listener

[Jacob-D-Christiansen]

Hello,

we have now tried to install and run your demo for iteractivePeakAnnotator, however we keep getting the error that "feature" is not defined. What do you recommend to do about this?

[cmdcolin]

Can you provide the exact error details e.g. screenshot, or stack trace

[tdhock]

@Jacob-D-Christiansen please read this https://stackoverflow.com/help/minimal-reproducible-example which explains how to ask questions with enough detail in order to get a helpful response.

[cmdcolin]

Indeed. Probably what I need in this case is a stack trace of the error (e.g. copy the actual error about feature not being defined), and then maybe info like what steps you took to install it and such

[cmdcolin]

Any updates?

[Jacob-D-Christiansen]

Hello, Very sorry about the spacing between posts, as well as the the vague-ness of the last question. To install the plugin I copied the git repo in to the plugins folder as well as renamed it to match what was supposed to be written in to the "jbrowse_conf.json" ie "InteractivePeakAnnotator". I then went to "http://localhost/?data=plugins/InteractivePeakAnnotator/test/data" to view the demo that is provided. Here it displays the tracks but as I try to add highlights it causes many errors as it draws the highlights back to the screen. Below I posted a picture of one of these errors, they all seem to be the same error from what I can see. At least all of them have the first line of "feature is not defined". There are two pictures as the error is quite long. The error starts on the first one and is contitued on the second untill it reads "feature is not defined" again.

[cmdcolin]

@Jacob-D-Christiansen can you check that you have the latest version of the wigglehighlighter plugin?

[Jacob-D-Christiansen]

Hello, That did it thank you very much. I will be in contact if there is any other issues

[Jacob-D-Christiansen]

Hello, Some quick questions that I have had while working with the peak annotator.

I have added 2 more kinds of labels and when adding what color those labels would be I thought that you used hex for the color codes. However when I plugged in some hex in to the highlight color callback it showed a non transparent block over the graph. I was wondering what color codes do you use to have them be transparent?

The second question is that we want to be able to delete a label on a specific track . I have a system in place that currently that can remove a label once it is place. However how it does this is that it removes the clicked on feature from the feature array stored in local Storage. This causes an issue as all tracks use the same feature array to draw its highlights so removing this removes the highlight from all tracks. What are your initial thoughts on this? do you have any specific things you would do in order to accomplish this?

Thank you for your help

[cmdcolin]

1) there are 3 places referring to "state" here https://github.com/cmdcolin/interactivepeakannotator/blob/master/js/View/Track/XYPlot.js did you update all of them? hex or any type of color would be allowed ( feel free to show me the code when you ask these questions)

2) I guess my feeling is that deleting could be a good idea. maybe a right click menu added to the highlight. but why would you want to delete the label on a specific track? wouldn't it be better to have a "nopeak" label rather than a deleted label?

[Jacob-D-Christiansen]

I have the highlights showing different colors just when I cycle through to the new colors they do not have any transparency. and I suppose you are right about the deletion thing. For now at least it will be fine to just not have that ability and instead delete all of that label instead.

[cmdcolin]

You could certainly add a transparent state

I have also added now onHighlightRightClick configuration to this repo, similar to the onHighlightClick that was there. There is no proper custom right-click context menu but hopefully it could help as is