Scalene fails to start if the installation path contains spaces.

[domdfcoding]

Describe the bug If scalene is installed to a path which contains spaces (e.g. /tmp/path with spaces/venv/lib/python3.8/site-packages/scalene) it fails to start with the following error:

ERROR: ld.so: object '/tmp/path' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object 'with' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object 'space/venv/lib/python3.8/site-packages/scalene/libscalene.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
Scalene failed to initialize.
Traceback (most recent call last):
  File "/tmp/path with space/venv/lib/python3.8/site-packages/scalene/scalene_profiler.py", line 1778, in run_profiler
    pywhere.register_files_to_profile(
Exception: Unable to find p_whereInPython

The issue is that scalene puts the installation directory into the LD_PRELOAD environment variable here:

https://github.com/plasma-umass/scalene/blob/6b81f83d2d0291ef2b14708e701d930179731347/scalene/scalene_preload.py#L28-L31

but LD_PRELOAD is whitespace separated, so treats it as three paths:

• /tmp/path
• with
• space/venv/lib/python3.8/site-packages/scalene/libscalene.so

The suggestion in this thread is to create a symbolic link to libscalene.so in a directory without spaces in the path, and set LD_PRELOAD to that directory.

I tried that approach (although I am unfamiliar with the scalene codebase) but couldn't make it work. It just created a load of symlinks. Does this bit of code run in parallel?

I also tried setting LD_LIBRARY_PATH to the installation path and LD_PRELOAD to just libscalene.so, and scalene ran but the profiling results were empty.

To Reproduce Steps to reproduce the behavior:

1. mkdir "/tmp/path with space"
2. virtualenv venv
3. source venv/bin/activate
4. echo "pass" > t.py
5. python3 -m scalene t.py

Expected behavior Scalene runs successfully when the installation path contains spaces.

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: Ubuntu 20.04
• Browser N/A
• Version [e.g. 22]

Additional context Add any other context about the problem here.

[emeryberger]

This should do the trick - please try to install from the repository (python3 -m pip install git+https://github.com/plasma-umass/scalene) and let me know!

[emeryberger]

Closing because I believe this is now resolved. If not, please re-open - thanks!

[dheerajck]

@emeryberger I have this issue, I have tried it with pip install scalene and pip install git+https://github.com/plasma-umass/scalene

python --version
Python 3.11.6

scalene --version
Scalene version 1.5.38 (2024.03.24)

pwd
/home/dheeraj/profiler test

scalene test1.py
ERROR: ld.so: object '/home/dheeraj/profiler\' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object 'test/.venv/lib/python3.11/site-packages/scalene/libscalene.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
Scalene failed to initialize.
Traceback (most recent call last):
  File "/home/dheeraj/profiler test/.venv/lib/python3.11/site-packages/scalene/scalene_profiler.py", line 2029, in run_profiler
    Scalene.register_files_to_profile()
  File "/home/dheeraj/profiler test/.venv/lib/python3.11/site-packages/scalene/scalene_profiler.py", line 1895, in register_files_to_profile
    pywhere.register_files_to_profile(
Exception: Unable to find p_whereInPython

[bhack]

Same issue but not related to spaces:

  File "/opt/conda/lib/python3.10/site-packages/scalene/scalene_profiler.py", line 412, in profile
    Scalene.register_files_to_profile()
  File "/opt/conda/lib/python3.10/site-packages/scalene/scalene_profiler.py", line 1895, in register_files_to_profile
    pywhere.register_files_to_profile(
Exception: Unable to find p_whereInPython

[sternj]

@bhack and @dheerajck , would you both please give me more information on your environment? In particular, operating system, version, and Conda version if relevant.

[whinee]

Hello! I'm also here to report the same problem, but when running scalene for the first time.

scalene school/qsnt_gen_cfg.py

ERROR: ld.so: object '/home/lyra/systems/K11\' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object 'Company/40-49\' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object 'Codebases/40\' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object 'Codebases\' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object 'Management/40.11\' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object 'School\' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object 'A\' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object 'Template/.venv/lib64/python3.10/site-packages/scalene/libscalene.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
Scalene failed to initialize.
Traceback (most recent call last):
  File "/home/lyra/systems/K11 Company/40-49 Codebases/40 Codebases Management/40.11 School A Template/.venv/lib64/python3.10/site-packages/scalene/scalene_profiler.py", line 2128, in run_profiler
    Scalene.register_files_to_profile()
  File "/home/lyra/systems/K11 Company/40-49 Codebases/40 Codebases Management/40.11 School A Template/.venv/lib64/python3.10/site-packages/scalene/scalene_profiler.py", line 1992, in register_files_to_profile
    pywhere.register_files_to_profile(
Exception: Unable to find p_whereInPython

Environment: OS: Fedora 40 Linux Python version: 3.10 (venv) How python and package is installed: Poetry

Suspect

I believe this to be a problem in processing arguments. An answer in stackoverflow suggests that the program might be doing a naive split on spaces.

How to replicate

1. Make a directory with spaces, and go inside that directory
2. Install scalene in that directory using poetry with the following files

poetry.toml:

[virtualenvs]
in-project = true

pyproject.toml:

[tool.poetry]
name = "src"
version = "0.0.0"
description = ""
authors = ["whatever"]

packages = [
    { include = "src" }
]

[tool.poetry.dependencies]
python = ">=3.10,<3.11"

[tool.poetry.group.dev]
optional = false

[tool.poetry.group.dev.dependencies]
scalene = "^1.5.44.1"

src/test.py:

# put whatever to test in here

3. Run poetry install --no-root --with dev
4. Run scalene src/test.py, then see the error

Solution (at least for me)

Install scalene globally thru pipx, via the command pipx install scalene. It works!

However, the problem of libscalene.so is still not solved...

edit: It doesn't work. scalene apparently needs to have access to the dependencies... oh man...

[sternj]

Thank you for the verbose reproduction instructions @whinee ! I'm on it

[sternj]

I think you're right about the split on spaces, it's the only thing that makes sense. I've replicated with your instructions, I should have a proposed patch shortly

[whinee]

nice, thank you!

[sternj]

Hi all, please check #864 (branch issue400) to see if it fixes your issues! It fixes things in my testing. Thanks again for the pointer to the solution @whinee !