Tells me to use Chrome instead of Safari...but I'm using Chrome

[jamestwebber]

Just installed the viewer and started it up, and while the page loads I get this error message:

Note: fetching is currently broken on Safari, use Chrome or Firefox instead

This is pretty confusing because I actually am using Chrome (Version 61.0.3163.100). Couldn't tell if this was a known issue already so I thought I'd open one.

edit: I should say that this is a very interesting project and I'm interested in using it!

[freeman-lab]

Yeah super cool project! Getting the same error as @jamestwebber in Chrome...

[JobLeonard]

Hey, thanks for being interested!

So uh.. this is embarassing. In-house this only broke on Safari before, and seemed to be related to the polyfill we used for fetch. So this was just a message for the Linnarsson group and not meant to be seen by the public, but I didn't get around to finding the real source of the problem before the conference! (maybe we should just ditch the fetch polyfill altogether, it's fairly well-supported and we're relying pretty heavily on other ES16 features anyway)

I just got home after a day-long flight from Israel to Stockholm, but I'll look into this first thing in the morning! :)

Few questions:

• did you install from pip or from source?
• which operating systems are you using?
• which version of Chrome?
• Any add-ons that might interfere with Fetch? (not very likely, unless you have some very strict white-listing policy)
• Does the problem persist in other browsers?

[jamestwebber]

Installed from pip on OS X 10.12.6, Chrome version as above (61.0.3163.100). No weird add-ons. As for the last question, I only have Chrome and Safari installed...

[jamestwebber]

And no worries about having bugs, better to release early with bugs than too late. 👍

[JobLeonard]

First of all: I just uploaded the latest version to PyPi - the previous update was early summer, and a lot of things have changed. Including specific fixes related to this, right now. So please run pip install -U loom-viewer before proceeding.

I have a suspicion what the problem might be: you don't have any Loom files in the loom-datasets folder.

My apologies, this wasn't mentioned in the docs yet. However, the loom command itself does have an up-to-date --help flag, and each subcommand has one too. It might still be a bit cryptic at times though (and I don't think this particular thing is mentioned in there either).

There also another thing that you haven't run into yet, but will, because I forgot to document it. I'll write that down below as well.

Where to store .loom files for the loom-viewer

By default, the loom command looks in loom-datasets (created the first time it is run) for projects. It should be structured as follows:

loom-datasets/
├── #Project 1 (folder)
│   ├── # dataset1.loom (loom file)
│   └── # dataset2.loom (loom file)
└── #Project 2 (folder)
    ├── # dataset1.loom (loom file)
    └── # dataset2.loom (loom file)

In the root data folder, each folder is treated as a project, and anything that is not a folder is ignored. So if we have a cortex.loom file containing data about some paper, and store it in a Published project folder like so:

loom-datasets/
└── Published
    └── cortex.loom

The resulting view should be something like:

I hope this was the source of your problem, if not let me know!

Viewing the heatmap and genes

Currently, only list and attribute metadata is generated on the fly; extracting data from a loom file through h5py was such a bottleneck that it made our loom-viewer server server crash on loads as high as a few dozen genes being requested at once. It also could not cope with multiple people accessing the same loom file.

So instead, this data needs to be pre-generated from the loom file for quick static serving. Heat map tiles, and gene expression values are "expanded" using CLI (we're planning to add this expansion to the web-interface, to be more accessible to people less familiar with the command line - see #114).

After putting a loom file in the appropriate project folder as explained above, you can run the following commands from anywhere:

loom tile cortex.loom
loom expand -r cortex.loom

The output should look similar to this:

> loom tile cortex.loom

2017-10-20 21:39:31,680 - INFO - Found 1 projects
2017-10-20 21:39:31,680 - INFO - Entering project /home/job/loom-datasets/Published
2017-10-20 21:39:31,680 - INFO -   Connecting to cortex.loom at full path
2017-10-20 21:39:31,714 - INFO -     Precomputing heatmap tiles (stored in cortex.loom.tiles subfolder)

> loom expand -r cortex.loom

2017-10-20 21:15:02,688 - INFO - Found 1 projects
2017-10-20 21:15:02,688 - INFO - Entering project /home/job/loom-datasets/Published
2017-10-20 21:15:02,688 - INFO -   Connecting to cortex.loom at full path
2017-10-20 21:15:03,335 - INFO -     Expanding rows (stored in /home/job/loom-datasets/Published/cortex.loom.rows subfolder)

As you might have guessed, the loom tile and loom expand commands will automatically search all projects for all matching file names, and expand each. Unique file names are therefore encouraged!

Because expansion can be slow for larger Loom files, the command checks if the relevant subfolder already exists and skips expansion if it does. Meaning that if you abort gene expansion halfway, the unexpanded genes will not be added if you try again. To force that, run: loom expand -rt cortex.loom (t for "truncate"), which generates newly expanded files for all genes. Alternatively, delete the subfolder in question.

[JobLeonard]

@jamestwebber, can you tell me if this was the cause of the problem or if I should look elsewhere for bugs? Thanks :)

[jamestwebber]

I tried again and I now have a nice "you don't have any loom files" message, so I think you got it.

[JobLeonard]

Great, I'll keep this open until we're sure that once you do have a loom file in there, it all works.

Please keep the bug reports coming! Well, I'd rather not see more bugs but you get the idea

[jamestwebber]

Is there an example file I can download just to try things out? I didn't see one linked in the docs.

[JobLeonard]

This is not really my department (I'm not a biologist and generally don't know what is and isn't ready for public viewing), but I'm pretty sure I can share the cortex.loom file we've been using internally for small tests, since it's based on an already published paper. Perhaps it should be added the Linarrsson Lab website as a static example file...

Here's a quick dropbox link:

https://www.dropbox.com/s/xjk5nnyw8ymjjck/cortex.loom?dl=0

[jamestwebber]

I'd say that if the raw data is already published and this is just a different format, you're in the clear. 👍

And now that I have a file the viewer is working nicely. Thanks!

[JobLeonard]

Good to hear :)