Closed saeedfc closed 2 years ago
Absolutely, I've written a bit of documentation about this here: https://cellbrowser.readthedocs.io/internet.html
And yes, closing the terminal shuts down the webserver and so you can't access it anymore. It's really preferable to have it all copied to some webserver. What is your University? Don't you have a lab website?
Otherwise: we can host your dataset. All you have to do is send us the cellbrowser directory (not the -o html directory, but the directory where you ran cbBuild) and we'll take care of the rest for you.
On Thu, Feb 27, 2020 at 5:48 PM Saeed notifications@github.com wrote:
Dear,
Thanks for the tool. I was wondering, if it would be possible for me to deposit the output of cbBuild on a shared folder (on university server storage which is accessible by everyone at the lab) in the lab and everyone can access it by clicking on index.html file or some other URL? Maybe a silly question, But I don't know how this works. Also, At the moment,now, when I build with -p 8888 option, I am able to open the browser. But after I close the terminal, if I click on index.html, I get the error saying "Could not find a dataset at dataset.json?701c98b551. If you are sure that the link is correct, please contact the administrator of this server, or cells@ucsc.edu if this is running at UCSC. "
$cbImportSeurat -i SO_Human_Fibrosis_ALL_7_Jan_2020.rds -o fibrosis_2HC_1CD --htmlDir=~/public_html/cb -p 8888 INFO:root:inFname: SO_Human_Fibrosis_ALL_7_Jan_2020.rds, outDir: fibrosis_2HC_1CD, datasetName: fibrosis_2HC_1CD INFO:root:running fibrosis_2HC_1CD/runSeurat.R through Rscript Loading required package: Seurat Reading SO_Human_Fibrosis_ALL_7_Jan_2020.rds Exporting Seurat data to fibrosis_2HC_1CD Seurat Version installed: 3.1.3 Object was created with Seurat version 3.1.1.9020 Writing expression matrix to fibrosis_2HC_1CD/exprMatrix.tsv.gz Embedding tsne does not exist in Seurat object. Skipping. Writing embeddings to fibrosis_2HC_1CD/pca.coords.tsv Writing embeddings to fibrosis_2HC_1CD/umap.coords.tsv Writing meta data to fibrosis_2HC_1CD/meta.tsv Found precomputed markers in obj@misc['markers'] Writing top 100, cluster markers to fibrosis_2HC_1CD/markers.tsv Writing cellbrowser config to fibrosis_2HC_1CD/cellbrowser.conf Prepared cellbrowser directory fibrosis_2HC_1CD Warning messages: 1: In ExportToCellbrowser(sobj, "fibrosis_2HC_1CD", "fibrosis_2HC_1CD", : Embedding pca has more than 2 coordinates, taking only the first 2 2: In x[i] <- value[[j]] : number of items to replace is not a multiple of replacement length 3: In x[i] <- value[[j]] : number of items to replace is not a multiple of replacement length 314.13user 7.43system 5:21.62elapsed 99%CPU (0avgtext+0avgdata 13248796maxresident)k 0inputs+99848outputs (0major+4233026minor)pagefaults 0swaps INFO:root:Wrote logfile of R run to fibrosis_2HC_1CD/analysisLog.txt INFO:root:Copying SO_Human_Fibrosis_ALL_7_Jan_2020.rds to fibrosis_2HC_1CD/seurat.rds INFO:root:Not writing fibrosis_2HC_1CD/desc.conf, already exists INFO:root:dataRoot is not set in ~/.cellbrowser.conf. Dataset hierarchies are not supported. INFO:root:Determining if /home/luna.kuleuven.be/u0119129/public_html/cb/fibrosis_2HC_1CD/exprMatrix.tsv.gz needs to be created INFO:root:Reading headers of file /home/luna.kuleuven.be/u0119129/public_html/cb/fibrosis_2HC_1CD/exprMatrix.tsv.gz INFO:root:current input matrix looks identical to previously processed matrix, same file size, same sample names INFO:root:/home/luna.kuleuven.be/u0119129/public_html/cb/fibrosis_2HC_1CD/meta.tsv has the same md5 as in /home/luna.kuleuven.be/u0119129/public_html/cb/fibrosis_2HC_1CD/dataset.json, no need to rebuild meta data INFO:root:Reading sample names from /home/luna.kuleuven.be/u0119129/public_html/cb/fibrosis_2HC_1CD/meta.tsv INFO:root:Reading sample names from /home/luna.kuleuven.be/u0119129/public_html/cb/fibrosis_2HC_1CD/meta.tsv INFO:root:Loading old config from /home/luna.kuleuven.be/u0119129/public_html/cb/fibrosis_2HC_1CD/dataset.json INFO:root:Matrix and meta sample names have not changed, not indexing matrix again INFO:root:Parsing column Cluster from /home/luna.kuleuven.be/u0119129/public_html/cb/fibrosis_2HC_1CD/meta.tsv INFO:root:Parsing coordinates for Seurat pca INFO:root:Writing coordinates for Seurat pca INFO:root:Parsing coordinates for Seurat umap INFO:root:Writing coordinates for Seurat umap INFO:root:Wrote /home/luna.kuleuven.be/u0119129/public_html/cb/fibrosis_2HC_1CD/desc.json INFO:root:Auto-detecting http://luna.kuleuven.be/u0119129/public_html/cb/fibrosis_2HC_1CD/desc.jsonINFO:root:Auto-detecting number type of /media/u0119129/J-drive/GBW-0446_Targid-Mucosal_Immunology/Matteoli's Lab/Data/scRNA seq Analysis/Human Fibrosis/fibrosis_2HC_1CD/exprMatrix.tsv.gz INFO:root:Auto-detect: Numbers in matrix are of type 'int' INFO:root:Auto-detected gene IDs type: symbols INFO:root:Wrote 28 .tsv.gz files into directory /home/luna.kuleuven.be/u0119129/public_html/cb/fibrosis_2HC_1CD/markers/markers_0 INFO:root:dataRoot not set in ~/.cellbrowser.conf, no need to rebuild hierarchy INFO:root:Wrote /home/luna.kuleuven.be/u0119129/public_html/cb/fibrosis_2HC_1CD/cellbrowser.json.bak INFO:root:Wrote /home/luna.kuleuven.be/u0119129/public_html/cb/fibrosis_2HC_1CD/dataset.json INFO:root:Wrote main config /home/luna.kuleuven.be/u0119129/public_html/cb/fibrosis_2HC_1CD/dataset.json INFO:root:Rebuilding flat list of datasets, without hierarchies INFO:root:Found 1 datasets in subdirectories of directory /home/luna.kuleuven.be/u0119129/public_html/cb INFO:root:Wrote /home/luna.kuleuven.be/u0119129/public_html/cb/dataset.json Interrupt this process, e.g. with Ctrl-C, to stop the webserver Serving /home/luna.kuleuven.be/u0119129/public_html/cb on port 8888 Point your internet browser to http://127.0.0.1:8888 (or the IP address of this server)
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/maximilianh/cellBrowser/issues/159?email_source=notifications&email_token=AACL4TLSZJRJWYAXF4EMLFLRE7VHDA5CNFSM4K46ZZC2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IQ3IXAA, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACL4TNOKNO37L35RBUWHTDRE7VHDANCNFSM4K46ZZCQ .
Dear Max,
Thanks for the response. I am at KU Leuven university, Belgium. Unfortunately, we don't have an independent lab website at the moment. What we have is a shared storage which is accessible by an y lab member. What I would like to do is to put he directory there and give my colleagues a web interface to browse the data.
Once I keep the output folder there, how can people open the data in chrome(or any other browser)? Is it by clicking on the index file? This part is not clear to me.
I tried
sudo cbBuild -o /var/www/
[sudo] password for u0119129:
INFO:root:dataRoot is not set in ~/.cellbrowser.conf. Dataset hierarchies are not supported.
INFO:root:Creating /var/www/fibrosis_2HC_1CD
INFO:root:Determining if /var/www/fibrosis_2HC_1CD/exprMatrix.tsv.gz needs to be created
INFO:root:/var/www/fibrosis_2HC_1CD/exprMatrix.tsv.gz does not exist.
INFO:root:Creating /var/www/fibrosis_2HC_1CD/metaFields
INFO:root:Checking and reordering meta data to /var/www/fibrosis_2HC_1CD/meta.tsv
INFO:root:Reading sample names from /media/u0119129/J-drive/GBW-0446_Targid-Mucosal_Immunology/Matteoli's Lab/Data/scRNA seq Analysis/Human Fibrosis/fibrosis_2HC_1CD/meta.tsv
INFO:root:Reading headers of file /media/u0119129/J-drive/GBW-0446_Targid-Mucosal_Immunology/Matteoli's Lab/Data/scRNA seq Analysis/Human Fibrosis/fibrosis_2HC_1CD/exprMatrix.tsv.gz
INFO:root:Data contains 18373 samples/cells
INFO:root:Field 4, 'Location', has only a single value. Removing this field from meta data.
INFO:root:Field 5, 'Segment', has only a single value. Removing this field from meta data.
INFO:root:Field 7, 'Sampling_Type', has only a single value. Removing this field from meta data.
INFO:root:Converting to numbers and compressing meta data fields
INFO:root:Field Cell: type uniqueString, 18373 different values
INFO:root:Field origident: type enum, 4 different values
INFO:root:Field nCountRNA: type int, 11330 different values
INFO:root:Field nFeatureRNA: type int, 4743 different values
INFO:root:Field PatientClass: type enum, 2 different values
INFO:root:Field Age: type int, 3 different values
INFO:root:Field Patientno: type enum, 3 different values
INFO:root:Field percentmt: type float, 18048 different values
INFO:root:Field nCountSCT: type int, 7190 different values
INFO:root:Field nFeatureSCT: type int, 3393 different values
INFO:root:Field integratedsnnres06: type int, 28 different values
INFO:root:Field seuratclusters: type enum, 28 different values
INFO:root:Field Cluster: type enum, 28 different values
INFO:root:Indexing meta file /var/www/fibrosis_2HC_1CD/meta.tsv to /var/www/fibrosis_2HC_1CD/meta.index
INFO:root:Kept 18373 cells present in both meta data file and expression matrix
INFO:root:Auto-detecting number type of /media/u0119129/J-drive/GBW-0446_Targid-Mucosal_Immunology/Matteoli's Lab/Data/scRNA seq Analysis/Human Fibrosis/fibrosis_2HC_1CD/exprMatrix.tsv.gz
INFO:root:Auto-detect: Numbers in matrix are of type 'int'
INFO:root:Auto-detected gene IDs type: symbols
INFO:root:Copying/compressing /media/u0119129/J-drive/GBW-0446_Targid-Mucosal_Immunology/Matteoli's Lab/Data/scRNA seq Analysis/Human Fibrosis/fibrosis_2HC_1CD/exprMatrix.tsv.gz to /var/www/fibrosis_2HC_1CD/exprMatrix.tsv.gz
INFO:root:converting /var/www/fibrosis_2HC_1CD/exprMatrix.tsv.gz to /var/www/fibrosis_2HC_1CD/exprMatrix.bin and writing index to /var/www/fibrosis_2HC_1CD/exprMatrix.json, type None
INFO:root:Compressing gene expression vectors...
INFO:root:Auto-detecting number type of /var/www/fibrosis_2HC_1CD/exprMatrix.tsv.gz
INFO:root:Auto-detect: Numbers in matrix are of type 'int'
INFO:root:Wrote compressed expression values for 1000 genes
INFO:root:Wrote compressed expression values for 2000 genes
INFO:root:Wrote compressed expression values for 3000 genes
INFO:root:Wrote compressed expression values for 4000 genes
INFO:root:Wrote compressed expression values for 5000 genes
INFO:root:Wrote compressed expression values for 6000 genes
INFO:root:Wrote compressed expression values for 7000 genes
INFO:root:Wrote compressed expression values for 8000 genes
INFO:root:Wrote compressed expression values for 9000 genes
INFO:root:Wrote compressed expression values for 10000 genes
INFO:root:Wrote compressed expression values for 11000 genes
INFO:root:Wrote compressed expression values for 12000 genes
INFO:root:Wrote compressed expression values for 13000 genes
INFO:root:Wrote compressed expression values for 14000 genes
INFO:root:Wrote compressed expression values for 15000 genes
INFO:root:Wrote compressed expression values for 16000 genes
INFO:root:Wrote compressed expression values for 17000 genes
INFO:root:Wrote compressed expression values for 18000 genes
INFO:root:Wrote compressed expression values for 19000 genes
INFO:root:Wrote compressed expression values for 20000 genes
INFO:root:Wrote compressed expression values for 21000 genes
INFO:root:Wrote compressed expression values for 22000 genes
INFO:root:Wrote compressed expression values for 23000 genes
INFO:root:Global minimum in matrix is: 0.000000
INFO:root:Wrote /var/www/fibrosis_2HC_1CD/cellbrowser.json.bak
INFO:root:Wrote /var/www/fibrosis_2HC_1CD/dataset.json
INFO:root:Wrote main config /var/www/fibrosis_2HC_1CD/dataset.json
INFO:root:Parsing column Cluster from /var/www/fibrosis_2HC_1CD/meta.tsv
INFO:root:Parsing coordinates for Seurat pca
INFO:root:Creating /var/www/fibrosis_2HC_1CD/coords/coords_0
INFO:root:Writing coordinates for Seurat pca
INFO:root:Parsing coordinates for Seurat umap
INFO:root:Creating /var/www/fibrosis_2HC_1CD/coords/coords_1
INFO:root:Writing coordinates for Seurat umap
INFO:root:Wrote /var/www/fibrosis_2HC_1CD/desc.json
INFO:root:Creating /var/www/fibrosis_2HC_1CD/markers/markers_0
INFO:root:Wrote 28 .tsv.gz files into directory /var/www/fibrosis_2HC_1CD/markers/markers_0
INFO:root:dataRoot not set in ~/.cellbrowser.conf, no need to rebuild hierarchy
INFO:root:Wrote /var/www/fibrosis_2HC_1CD/cellbrowser.json.bak
INFO:root:Wrote /var/www/fibrosis_2HC_1CD/dataset.json
INFO:root:Wrote main config /var/www/fibrosis_2HC_1CD/dataset.json
INFO:root:Rebuilding flat list of datasets, without hierarchies
INFO:root:Found 1 datasets in subdirectories of directory /var/www/
INFO:root:Wrote /var/www/dataset.json
INFO:root:/var/www/index.html does not exist: running cbUpgrade now to make sure there is an index.html
INFO:root:Copying js, css and img files to /var/www/
INFO:root:Creating /var/www/ext/images
INFO:root:Creating /var/www/img
INFO:root:Creating /var/www/js
INFO:root:Creating /var/www/css
INFO:root:Wrote /var/www/index.html (devMode: False)
And then I clicked on the index.html file in /var/www/index.html . But it returned the following error
"Could not find a dataset at dataset.json?701c98b551. If you are sure that the link is correct, please contact the administrator of this server, or cells@ucsc.edu if this is running at UCSC. "
Thanks and kind regards, Saeed
Dear Max,
Thanks for the offer of hosting the data. We will definitely think about it once we publish. At the moment, we are constantly revising the datasets by adding more data and also I have multiple projects with different datasets.
Hi,
I have now made a webpage with github pages. https://saeedfc.github.io/Mucosal-Immunology-Lab-KUL/
Is it possible that I can host the data there? Thanks
Hi, @saeedfc. It could work, although Github has limits on filesize. How large are your files? Github has a strict 100MB limit on file sizes: https://help.github.com/en/github/managing-large-files/conditions-for-large-files.
Hi @matthewspeir My file is 101MB and soon I will need to host nearly 500MB at least. So I better start a website. But how how to port again the output of a build? Can I click on index.html to strt eh browser? It doesn't work for me. As I said above, I encounter an error. Can you kindly help with that? Thanks and kind regards, Saeed
Hi Saeed, if your lab has no webserver and your University doesn't offer any web hosting, then you could either buy some commercially (around 20eur per month, we can recommend some companies) or we could host the dataset for you, password-protected.
But as far as I know, IT at KU Leuven offers web hosting. They offer box cloud storage, as you say (which doesn't work for hosting a cell browser, the details are complicated, but box.com support has repeatedly confirmed to me that while it sometimes may work, they have no definite plan to allow byte range requests). In addition to box, KU Leuven offers many other services, feel free to contact your University's IT department at https://set.kuleuven.be/set-it/home, you can look at their full list of services there first for "web hosting" or "website" and otherwise send them an email.
We could try hosting your dataset and add some features to make it very easy for you to update your data... what type of changes would you want to make? meta data changes?
On Sat, Feb 29, 2020 at 8:20 AM Saeed notifications@github.com wrote:
Hi @matthewspeir https://github.com/matthewspeir My file is 101MB and soon I will need to host nearly 500MB at least. So I better start a website. But how how to port again the output of a build? Can I click on index.html to strt eh browser? It doesn't work for me. As I said above, I encounter an error. Can you kindly help with that? Thanks and kind regards, Saeed
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/maximilianh/cellBrowser/issues/159?email_source=notifications&email_token=AACL4TOWUF6JL6JT7SA6C2DRFC3KHA5CNFSM4K46ZZC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENLSH6Q#issuecomment-592913402, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACL4TPBSWSEZKMZY44ESN3RFC3KHANCNFSM4K46ZZCQ .
HI Maximilian,
I shall get back once I get it touch with he IT. Indeed I see that webhosting service is available. Thanks a lot for your time! Saeed
I'm using github.. seems to be working pretty well. I think it can go up to 1GB? But another option is using a google cloud bucket! Unlimited size! (you pay for it).
I've closed this issue, but please feel free to open if you feel like there's still more that needs to be done.
Dear,
Thanks for the tool. I was wondering, if it would be possible for me to deposit the output of cbBuild on a shared folder (on university server storage which is accessible by everyone at the lab) in the lab and everyone can access it by clicking on index.html file or some other URL? Maybe a silly question, But I don't know how this works. Also, At the moment,now, when I build with -p 8888 option, I am able to open the browser. But after I close the terminal, if I click on index.html, I get the error saying "Could not find a dataset at dataset.json?701c98b551. If you are sure that the link is correct, please contact the administrator of this server, or cells@ucsc.edu if this is running at UCSC. "