Improve visuals and speed of single-tissue queries

amkwong commented 4 years ago

Ticket

Addresses #33 and #34

Purpose

Restyled the single-tissue range view page using Bootstrap 4 elements.
Use tissue-specific data files when processing single-tissue requests to speed up single-tissue range-based queries

Deployment / configuration notes

I have already uploaded the necessary files to the deployment server's data directory.

The updated query requires tissue-specific data files when querying from a single tissue. You can find test files at the location below. Note that these files only have data for chr19:448506-528506 for testing purposes.

/net/amd/amkwong/browseQTL/all_chr/data/test.data/Adipose_Subcutaneous.allpairs.sorted.txt.gz /net/amd/amkwong/browseQTL/all_chr/data/test.data/Adipose_Subcutaneous.allpairs.sorted.txt.gz.tbi /net/amd/amkwong/browseQTL/all_chr/data/test.data/Brain_Cortex.allpairs.sorted.txt.gz /net/amd/amkwong/browseQTL/all_chr/data/test.data/Brain_Cortex.allpairs.sorted.txt.gz.tbi

The full data set can be found at the following location: /net/amd/amkwong/browseQTL/all_chr/data/.allpairs.sorted.txt.gz

These files can be generated by the data processing pipeline if you start with raw data from GTEx v8.

Things to fix

Dynamically generate the list of genes (this is the more important one!) and tissues (can use the same 49 every time) in the dropdown buttons in the single-tissue range view. For example: https://github.com/abought/pheget/blob/2bd614bc32c584547b012102298f7c665d7e7e6d/pheget/frontend/templates/frontend/region.html#L49-L52
Create a suggestion system to choose which gene and/or tissue to use as the default anchors when searching for a gene or tissue in an arbitrary range
Maybe rely less on omnisearch (which is currently down)

amkwong commented 4 years ago

I added small test files for two tissues (Adipose_Subcutaneous and Brain_Cortex), along with their index files, to the data directory. They contain data in the region chr19:448506-528506, and are less than 1 MB each. They will be useful for testing region views on a local machine.

amkwong commented 4 years ago

I've done all the updates I can at the moment. There are still some issues I want to tackle (making searchbox.js better, for example), but I'll do it in a new branch. I may still update the buttons in the region view here, but I'm going to leave everything else alone for now. I'm ready for this pull request to be finalized, merged, and updated on the server.

If you're still reading, a quick question to see if you have any good ideas: Is there an easy way to convert the long (ENSG) names to the shorter common gene names in JavaScript that doesn't involve relying on omnisearch or some other outside service? I'm specifically thinking about this: https://github.com/abought/pheget/blob/b3a8594141e5c7f056e0f6f2a69f242d46b79196/pheget/static/js/region.js#L78 JavaScript doesn't seem to like the idea of me loading a file, or else this would be much easier. Is the most straightforward solution to create an API that serves gene symbols when I submit a long gene name?

abought commented 4 years ago

If you're still reading, a quick question to see if you have any good ideas:

I clearly have work life balance issues this week, so yeah.

You'll generally have issues reading files from JS, and certainly you don't want users to download a big lookup (like a database) on every page view. The network boundary is pretty rough that way.

As for options- I assume this can't be done at page load because it's a dynamic menu. But if the menu already has one name, can we give it two? Check in with me in the AM and we can take a closer look together; I need to look at the data. I'll merge start of the day; I should never merge so near to zonking.

amkwong commented 4 years ago

The json file uploaded contains the full conversion table for all genes in GTEx v8. This means we no longer need the python pickle file for gene name lookups, so I removed it, and edited the parts of the code that depends on the pickle.

statgen / fivex