search

linnarsson-lab / loom-viewer

Tool for sharing, browsing and visualizing single-cell data stored in the Loom file format
BSD 2-Clause "Simplified" License
35 stars 6 forks source link

Package for pip, command line tool #37

Closed slinnarsson closed 8 years ago

slinnarsson commented 8 years ago

Create setup.py and register with Pypi to enable "pip install"

slinnarsson commented 8 years ago

Make it a command that can be run from the Terminal. The command would also have subcommands for managing loom files:

loom: start the server, launch user's default browser loom put: upload a loom file to a remote server loom clone: download a remote loom file to the local dataset folder loom prepare: prepare a loom file for browsing (tSNE, 2D projection, etc.) loom tsne: t-SNE projection loom tile: prepare heatmap tile pyramid loom stats: prepare standard stats (mean, cv etc) loom make: Linnarsson lab-specific, make loom from SQL server

These commands are essentially the old pipeline but made available to command-line users. The previous "upload" functionality would be replaced by loom make, loom prepare and loom put.

This also means we need an endpoint to upload files, with proper authorization. Probably PUT /loom/{project}/{filename} makes sense.

JobLeonard commented 8 years ago

The build process doesn't quite get rid of old builds now:

creating build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.0b80efca10453db2e7c3.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.e4da1ada290b9b4911ed.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.d54cd66797c031098437.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.f43728d0560134400e61.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.041e85a2226a2246bff2.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.a1446e51ac3ed28e8b5f.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.0eeff4f89ffabbcd28b5.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.c48257cdf53e0d35b6cf.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.855e53fee59303c0e234.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.3701b14c83296ad01d6b.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.45c52ee1902f254eaec5.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.559e265394fb8c468593.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.16240de0eac4ec259d8b.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.287cac99b079aed08e34.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.19c5bfe6824b2e7893b2.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.1fdea5475c0bc1b1d87d.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.a423a0bdbdca631fc243.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.d20f5cbaede9d58efefe.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.e3b07c47d0a8bc54ffc1.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.1526b0744fe8bf4ea465.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.0127b87ba1a9eff14af3.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.feba36df0993cec39528.js -> build/bdist.linux-x86_64/egg/loompy/static/js
copying build/lib/loompy/static/js/bundle.f4729fc696ad9b540bfd.js -> build/bdist.linux-x86_64/egg/loompy/static/js
creating build/bdist.linux-x86_64/egg/loompy/static/css

I'm not sure if it's related, but I'm also having issues with the website loading older, cached versions of the JS bundle, which gets in the way debugging obviously.

JobLeonard commented 8 years ago

I tried python setup.py clean but that didn't do anything. Adding the following to the build file works though:

if [ -a ./python/build ]; then
  rm -r ./python/build
fi
JobLeonard commented 8 years ago

So I uncluttered the dataset-list and tried to upload, but then I got this message:

removing 'loompy-0.9.3' (and everything under it)
running upload
Traceback (most recent call last):
  File "setup.py", line 30, in <module>
    url = "https://github.com/linnarsson-lab/Loom",
  File "/home/job/Dev/anaconda2/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/home/job/Dev/anaconda2/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/home/job/Dev/anaconda2/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/home/job/Dev/anaconda2/lib/python2.7/distutils/command/upload.py", line 60, in run
    self.upload_file(command, pyversion, filename)
  File "/home/job/Dev/anaconda2/lib/python2.7/distutils/command/upload.py", line 70, in upload_file
    raise AssertionError("unsupported schema " + schema)
AssertionError: unsupported schema

Since I didn't touch setup.py I'm not sure what the problem is. Anyway, I pushed the fix to git so you can upload to the server yourself for now. It defaults to genescape, since that one works properly right now.

slinnarsson commented 8 years ago

Ah. You need a .pypirc file with the correct credentials. See http://peterdowns.com/posts/first-time-with-pypi.html

I will give it to you separately so it doesn’t end up in the repository.

/Sten

Sten Linnarsson, PhD Professor of Molecular Systems Biology Karolinska Institutet Unit of Molecular Neurobiology Department of Medical Biochemistry and Biophysics Scheeles väg 1, 171 77 Stockholm, Sweden +46 8 52 48 75 77 (office) +46 70 399 32 06 (mobile)

On 15 Aug 2016, at 15:40, Job van der Zwan notifications@github.com<mailto:notifications@github.com> wrote:

So I uncluttered the dataset-list and tried to upload, but then I got this message:

removing 'loompy-0.9.3' (and everything under it) running upload Traceback (most recent call last): File "setup.py", line 30, in url = "https://github.com/linnarsson-lab/Loom", File "/home/job/Dev/anaconda2/lib/python2.7/distutils/core.py", line 151, in setup dist.run_commands() File "/home/job/Dev/anaconda2/lib/python2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "/home/job/Dev/anaconda2/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "/home/job/Dev/anaconda2/lib/python2.7/distutils/command/upload.py", line 60, in run self.upload_file(command, pyversion, filename) File "/home/job/Dev/anaconda2/lib/python2.7/distutils/command/upload.py", line 70, in upload_file raise AssertionError("unsupported schema " + schema) AssertionError: unsupported schema

Since I didn't touch setup.py I'm not sure what the problem is. Anyway, I pushed the fix to git so you can upload to the server yourself for now. It defaults to genescape, since that one works properly right now.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://github.com/linnarsson-lab/Loom/issues/37#issuecomment-239804049, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKKagx4aMGrMGGRIfoff3DgYAPr93ajtks5qgGw3gaJpZM4JgJxG.

slinnarsson commented 8 years ago

Ok clearly something is not working. I pull the latest version, build and check locally. I verify that I see the latest version with less clutter on the dataset view. Then I deploy, and confirm that the server indeed got upgraded to the latest version from PyPi. But still the server shows the previous version of the web page.

JobLeonard commented 8 years ago

Could it be that is has the same issue I had, with old .js files not being cleaned?

On 15 August 2016 at 17:55, Sten Linnarsson notifications@github.com wrote:

Ok clearly something is not working. I pull the latest version, build and check locally. I verify that I see the latest version with less clutter on the dataset view. Then I deploy, and confirm that the server indeed got upgraded to the latest version from PyPi. But still the server shows the previous version of the web page.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/linnarsson-lab/Loom/issues/37#issuecomment-239842608, or mute the thread https://github.com/notifications/unsubscribe-auth/AAP3AKQG9bbEKtm19d2RSBqzDbcoMFO8ks5qgIvYgaJpZM4JgJxG .

JobLeonard commented 8 years ago

Although that wouldn't really make sense, given that I also updated the build script to fix that locally for me.

slinnarsson commented 8 years ago

Ok fixed it: