gateway doesn't lead to correct address

[campsj]

First of all, great tool you created! Very useful.

I'm hosting the gateway on our hpc and it is working but I have the issue that is not pointing to the right IP address (although when you type it in in chrome it actually loads under this IP address). This is quite annoying because every time I want to open a dataset from the file crawler it opens it under localhost and I have to change it to the correct IP address. Any idea on your side how we can fix this?

(cxg_VIP) [glanl@by08tq ~]$ cellxgene-gateway
2020-09-23 17:46:13,765:cellxgene_gateway:INFO:Got required env: {'CELLXGENE_LOCATION': '/gpfs01/home/glanl/miniconda3/envs/cxg_VIP/bin/cellxgene', 'CELLXGENE_DATA': '/gpfs01/bhcbio/flatfile/single_cell_datasets'}
2020-09-23 17:46:13,765:cellxgene_gateway:INFO:Got optional env: {'EXTERNAL_HOST': '10.190.91.50:5005', 'EXTERNAL_PROTOCOL': 'http', 'GATEWAY_IP': '10.190.91.50', 'GATEWAY_PORT': 5005, 'GATEWAY_EXTRA_SCRIPTS': None, 'GATEWAY_TTL': None, 'GATEWAY_ENABLE_UPLOAD': False, 'GATEWAY_ENABLE_ANNOTATIONS': True, 'GATEWAY_ENABLE_BACKED_MODE': False, 'CELLXGENE_ARGS': None}
 * Serving Flask app "cellxgene_gateway.gateway" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
2020-09-23 17:46:13,769:werkzeug:INFO: * Running on http://0.0.0.0:5005/ (Press CTRL+C to quit)

[alokito]

Hi @campsj ! Thanks for trying out Cellxgene Gateway. I am not sure why the links in the file crawler are opening up to localhost for you, since they do not contain any hostname information; if you view source on the page they should look like the following:

<a href='/view/pbmc3k.h5ad'>pbmc3k.h5ad</a>

and thus should open on the same domain as the file crawler is hosted on; are you seeing something similar?

[campsj]

If I go into filecrawler and click on a dataset I get this: http://localhost:5005/view/BC_IMM_IND_GSE114725.h5ad/ If I right click and copy the link address I get this: http://10.190.91.50:5005/view/BC_IMM_IND_GSE114725.h5ad

It seems my browser automatically switches it to localhost but then if I change localhost again to 10.190.91.50 it works (seems to add a slash to the address at the end). Do you think this issue is linked to the gateway or it is somehow linked to my browser?

[katiedaco]

@campsj , the same thing is happening with me! I recently upgraded cellxgene-gateway, so I wonder if that is related? I can right click and 'open in new tab' and it is fine, but if I left click to open the dataset it goes to localhost. I don't even know how to troubleshoot this. I am using a chrome-based browser, how about you? And agreed - cellxgene-gateway is really neat, and so glad it's available :)

[campsj]

Ok so it doesn't seem to be an issue that I only have troubles with. I'm using chrome but I also tried it with firefox and it gave me the same issues.

[campsj]

Could this be a mac/windows issue? Colleagues of mine with a windows can perfectly open it in firefox while I can't on my mac

[alokito]

Hi there, my apologies for the slow response but it got busy at work for a while. I have been unable to replicate this issue. I have tried the following with AWS: Launch a new Amazon Linux 2 instance, pasting in the following as the user-data script:

#!/bin/bash
yum update -y
yum install -y python3 python3-pip  gcc python3-devel wget
cd /home/ec2-user
python3 -m venv my_env && source ./my_env/bin/activate
pip install cellxgene-gateway cellxgene
mkdir cellxgene_data && wget wget https://raw.githubusercontent.com/chanzuckerberg/cellxgene/master/example-dataset/pbmc3k.h5ad -O cellxgene_data/pbmc3k.h5ad

export CELLXGENE_LOCATION=/home/ec2-user/my_env/bin/cellxgene
export CELLXGENE_DATA=/home/ec2-user/cellxgene_data
export GATEWAY_IP=$(curl http://169.254.169.254/latest/meta-data/public-ipv4)
export GATEWAY_PORT=5005
export EXTERNAL_HOST=${GATEWAY_IP}:${GATEWAY_PORT}
export EXTERNAL_PROTOCOL=http
cellxgene-gateway

If I now browse to the public IP address on port 5005, it seems to work fine in Chrome on both Mac and PC. Can you describe in more detail how to replicate?

[alokito]

BTW which version of Cellxgene Gateway is reported by

import cellxgene_gateway
print(cellxgene_gateway.__version__)

should be 0.2.2.

[campsj]

My apologies now for answering late. The version is 0.2.2.

I guess the problem is unrelated to the cellxgene-gateway and will be very difficult to reproduce. I had datasets where it works and where it doesn't work, so very difficult to put a finger on it. For me we can close this issue and if I have a decent example that is reproducible I'll reopen it again.

[dfeinzeig]

https://github.com/Novartis/cellxgene-gateway/blob/d5246d4b4b2a3f7d734f99a7c7ff8e9e3a8aec82/cellxgene_gateway/cache_entry.py#L123

seems to handle caching with a trailing slash, but

https://github.com/Novartis/cellxgene-gateway/blob/d5246d4b4b2a3f7d734f99a7c7ff8e9e3a8aec82/cellxgene_gateway/filecrawl.py#L94

does not include a trailing slash, resulting in a redirect. is this intentional @alokito ?

[klkeys]

From our end, following @dfeinzeig's remark, adding a trailing backslash / in filecrawl.py:94 fixed the redirect problem that @campsj and @katiedaco observed.

[alokito]

@dfeinzeig The inconsistency is not intentional and should be resolved. @klkeys thanks for evidently finding a fix.

I think I may have an explanation for what was happening in the original issue reported by @campsj : the redirect is a 301, which means that if in a previous test run EXTERNAL_HOST had been set to localhost, the browser may have cached the association http://10.190.91.50:5005/view/BC_IMM_IND_GSE114725.h5ad -> http://localhost:5005/view/BC_IMM_IND_GSE114725.h5ad/.

I should have some time to properly look into this on Saturday and make a new release. In the meanwhile I am happy to accept the PR.

[klkeys]

all credit to @dfeinzeig for finding the solution, it wasn't my idea. thank you for the explanation @alokito!

[alokito]

My apologies, it looks like I totally forgot about make a release from this. However, it is now released as part of 0.2.3.