[bug] Image thumbnails are not loading on homepage

[hydrosquall]

I installed latentscope this week. The initial run seemed to work (I have a directory full of JSON and parquet). Unfortunately, the plot thumbnails are not loading when I return to the landing page.

I would appreciate any pointers on how to debug this.

Steps

I launched latentscope 0.3.0

ls-serve latent-scope-data --port 5001

Then when I visit: http://127.0.0.1:5001/

Displayed	The picture exists on filesystem

I'm not sure if this picture should be copied elsewhere in the file directory, or if the image URL is pointing to the wrong place.

[enjalot]

can you check the Settings in the top right of the app? It should show you where your data directory is on your system. Also there is an Export link for the dataset that you can follow to see all the files.

Ideally the paths match what you expect. it could be you did something like ls-serve ~/latent-scope/data the first time and without the ~/ the second time?

[hydrosquall]

Thanks @enjalot - that is helpful! It suggested this might be a relative vs absolute paths misunderstanding on my part.

It looks like most of the app works if DATA_DIR is a relative path (like ./my-local-directory ), but the thumbnails (and for that matter, serving any of the static files via send_file in app.py) don't work. From adding some app logs, the app was checking the directory where latentscope was installed, instead of in the directory where I was running ls-serve .

The thumbnails worked as long as DATA_DIR (whether set in .env or CLI) is a full path ( like ~/my-scopes-directory ).

Going forward, what do you think about the options of

1. Rejecting data dirs that are provided as relative instead of full paths, vs
2. Updating the app to make DATA_DIR be generated relative to where ls-serve was called instead vs
3. Some other option?

[enjalot]

yeah I'm not happy with the env file stuff. another option might be to default to ~/latent-scope-data and just allow override of that. then the env config could also live there.

maybe there is a ~/.latent-scope config that has all your env stuff and the data directory path?

On Tue, Jun 25, 2024, 6:20 PM Cameron Yick @.***> wrote:

Thanks @enjalot https://github.com/enjalot - that is helpful! It suggested this might be a relative vs absolute paths misunderstanding on my part.

It looks like most of the app works if DATA_DIR is a relative path (like ./my-local-directory ), but the thumbnails (and for that matter, serving any of the static files via send_file in app.py) don't work. From adding some app logs, the app was checking the directory where latentscope was installed, instead of in the directory where I was running ls-serve .

The thumbnails worked as long as DATA_DIR (whether set in .env or CLI) is a full path ( like ~/my-scopes-directory ).

Going forward, what do you think about the options of

1. Rejecting data dirs that are provided as relative instead of full paths, vs
2. Updating the app to make DATA_DIR be generated relative to where ls-serve was called instead vs
3. Some other option?

— Reply to this email directly, view it on GitHub https://github.com/enjalot/latent-scope/issues/45#issuecomment-2190084336, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAXPPPW3HYCGQRULLYJTT3ZJHULVAVCNFSM6AAAAABJ4KZYDCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJQGA4DIMZTGY . You are receiving this because you were mentioned.Message ID: @.***>

[hydrosquall]

Personally I like having latent-scope-data not sitting directly under ~, so that I can version my outputs in different Git repos independently. I also keep different env files for projects that are in different contexts (e.g. staging vs production). Not sure if I am an outlier user though.

The root level latent-scope config could be an interesting quiet fallback for people who didn't want to store multiple copies of their config though!

In the immediate term, my use-case is unblocked because I can just do the long filepaths - the discussion about the path forward here is more to smooth the experience for people who didn't know relative file paths for the DATA_DIR may cause some features (thumbnails and export buttons) to be unavailable.

[enjalot]

ah good points. for your usecase could we expand the relative path to the current execution path + relative path for the user?

On Tue, Jun 25, 2024 at 7:38 PM Cameron Yick @.***> wrote:

Personally I like having latent-scope-data not sitting directly under ~, so that I can version my outputs in different Git repos independently. I also keep different env files for projects that are in different contexts (e.g. staging vs production). Not sure if I am an outlier user though.

The root level latent-scope config could be an interesting quiet fallback for people who didn't want to store multiple copies of their config though!

In the immediate term, my use-case is unblocked because I can just do the long filepaths - the discussion about the path forward here is more to smooth the experience for people who didn't know relative file paths for the DATA_DIR may cause some features (thumbnails and export buttons) to be unavailable.

— Reply to this email directly, view it on GitHub https://github.com/enjalot/latent-scope/issues/45#issuecomment-2190206720, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAXPPK6RP62IEIDMZGPYMLZJH5NPAVCNFSM6AAAAABJ4KZYDCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJQGIYDMNZSGA . You are receiving this because you were mentioned.Message ID: @.***>

-- Ian Johnson - 周彦 http://enjalot.github.io/ http://enja.org

[hydrosquall]

Ah good point I noticed ~ gets expanded . Expanding relative path ( ./ ) seems like it should serve that purpose too.

On Tue, Jun 25, 2024 at 11:00 PM Ian Johnson @.***> wrote:

ah good points. for your usecase could we expand the relative path to the current execution path + relative path for the user?

On Tue, Jun 25, 2024 at 7:38 PM Cameron Yick @.***> wrote:

Personally I like having latent-scope-data not sitting directly under ~, so that I can version my outputs in different Git repos independently. I also keep different env files for projects that are in different contexts (e.g. staging vs production). Not sure if I am an outlier user though.

The root level latent-scope config could be an interesting quiet fallback for people who didn't want to store multiple copies of their config though!

In the immediate term, my use-case is unblocked because I can just do the long filepaths - the discussion about the path forward here is more to smooth the experience for people who didn't know relative file paths for the DATA_DIR may cause some features (thumbnails and export buttons) to be unavailable.

— Reply to this email directly, view it on GitHub < https://github.com/enjalot/latent-scope/issues/45#issuecomment-2190206720>,

or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAAXPPK6RP62IEIDMZGPYMLZJH5NPAVCNFSM6AAAAABJ4KZYDCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJQGIYDMNZSGA>

. You are receiving this because you were mentioned.Message ID: @.***>

-- Ian Johnson - 周彦 http://enjalot.github.io/ http://enja.org

— Reply to this email directly, view it on GitHub https://github.com/enjalot/latent-scope/issues/45#issuecomment-2190451855, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACE2MM4TUSIH5EIEFRXTJZDZJIVERAVCNFSM6AAAAABJ4KZYDCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJQGQ2TCOBVGU . You are receiving this because you authored the thread.Message ID: @.***>

[enjalot]

@hydrosquall i just pushed what i hope is a fix to main, expanding relative paths like:

 if directory.startswith("./") or directory.startswith("../") or not directory.startswith("/"):
        directory = os.path.abspath(directory)

might be naive but also might work?

[hydrosquall]

Thanks for looking into it. I tried to see if that might fix it by pip installing the current head of main, but ran into a different issue ( https://github.com/enjalot/latent-scope/issues/ ).

Regarding the actual directory expansion: I think this will work out ok for me on Mac, but python Pathlib might handle this for other platforms if you can assume the python version is at least 3.11. A thing to look out for users is that this updated path will be written back to the .env file, so that if you run latent-scope once using a relative path, then relocate the folder to a new location, the env file will the expanded path to the old location.

directoryObj = pathlib.path(directory)
directory = str(directoryObj.absolute())

[hydrosquall]

Confirmed that https://github.com/enjalot/latent-scope/commit/2e786da85c4b13fc0b3d09ddcd8f1e0da0d4701d fixed it when installing the latest main and running the local web build instructions (cd web/ && npm install && npm run dev

I ran ls-serve my-custom-folder-name , where my-custom-folder-name is a folder in the directory where I ran this command. In .env, that folder name was expanded into a full absolute path. I think we can close this issue, thanks for the patch @enjalot !