HELICS is unable to load installed libraries/modules

[LBorden24]

Describe the bug Attempting to run the fundamental example using the instructions linked here: https://docs.helics.org/en/latest/user-guide/examples/fundamental_examples/fundamental_default.html#co-simulation-execution Both the Charger and Battery processes fail due to being unable to import necessary modules matplotlib, numpy and helics. All three modules have been verified to exist in my virtual environment using pip.

Environment (please complete the following information):

• Operating System: Windows
• Language Extension: Python
• fails using PyCharm IDE's cli, and fails when using cmd.exe
• HELICS version: 3.5.2

  $ helics_app --version: 3.5.2

[trevorhardy]

Hmm, this is mysterious. I just set up a clean environment to run this in and it ran fine for me. Here are the commands I ran:

% conda create -n helics_test python=3.12
% conda activate helics_test
% pip install matplotlib helics numpy
% helics run --path=fundamental_default_runner.json

Does that give you any hints? I'm on macOS and pyhelics installed v3.5.2 (same as yours). I was just doing this all on the macOS native command line; maybe PyCharm wasn't using the correct environment? Are the libraries installed in the same environment as the interpreter PyCharm was using?

[LBorden24]

Unfortunately not. running these commands in the command line produces the same error, regardless of whether it's run in a fresh venv or in the one I attempted to use previously

[trevorhardy]

If you use the Python REPL in an environment where you have matplotlib installed (% python), are you able to do things like import matplotlib successfully?

[LBorden24]

I am, yes

[trevorhardy]

And after exiting the REPL and tryinghelics run --path=fundamental_default_runner.json you get an error message about not being able to import matplotlib?

[LBorden24]

yes, exactly :(

[trevorhardy]

What happens if you try making a test python script that just uses matplotlib (no HELICS stuff); does that import correctly?

[LBorden24]

yes, running a file with import numpy and import matplotlib as the only lines runs just fine, both in the IDE and from cmd.exe

[trevorhardy]

@josephmckinsey and @nightlark, can you offer any insight into the environment that helics runner is using? I haven't seen this problem before. Could this be a Windows thing; can either of you replicate this problem?

[LBorden24]

Also, if it's helpful, here's a screenshot of my command line with the error I'm getting. This is my initial configuration, i.e. not the clean environment I made earlier in this thread.

[trevorhardy]

Can you post the broker, Battery, and Charger log files from any run where this breaks? Maybe there's a hint in there?

[nightlark]

What happens if you import helics from a Python REPL?

On Thursday, June 27, 2024, LBorden24 @.***> wrote:

Also, if it's helpful, here's a screenshot of my command line with the error I'm getting. This is my initial configuration, i.e. not the clean environment I made earlier in this thread. library.helics.png (view on web) https://github.com/GMLC-TDC/HELICS-Examples/assets/153466401/a264bb21-2f16-498d-af60-5737be5c62df

— Reply to this email directly, view it on GitHub https://github.com/GMLC-TDC/HELICS-Examples/issues/107#issuecomment-2195555670, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA6JBZYCQQIIRSCZHFRDESDZJRT7DAVCNFSM6AAAAABKAK5SECVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJVGU2TKNRXGA . You are receiving this because you were mentioned.Message ID: @.***>

[LBorden24]

Battery.log broker.log Charger.log @trevorhardy here are the files from the clean environment's last run

@nightlark importing helics from the REPL does not produce any errors

[trevorhardy]

Battery.log broker.log Charger.log @trevorhardy here are the files from the clean environment's last run

@nightlark importing helics from the REPL does not produce any errors

Well, as I'm sure you knew, those logs hold no secrets.

[nightlark]

Can you modify some of the Python scripts run as part of the example (Charger.py, etc) to include a line at the top before all imports that prints out the environment variables that are set, and location of the python interpreter being used and site-packages folder?

[LBorden24]

screenshot of the resulting run:

copy pasted from the console in case that's easier to read: interpreter location C:\Users\BORDLR\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\python.exe Environment variablesenviron({'ALLUSERSPROFILE':** 'C:\ProgramData', 'APPDATA': 'C:\Users\BORDLR\AppData\Roaming', 'BOOTNUM': '28', 'COMMONPROGRAMFILES': 'C:\Program Files\Common Files', 'COMMONPROGRAMFILES(X86)': 'C:\Program Files (x86)\Common Files', 'COMMONPROGRAMW6432': 'C:\Program Files\Common Files', 'COMPUTERNAME': 'INL614857', 'COMSPEC': 'C:\WINDOWS\system32\cmd.exe', 'DRIVERDATA': 'C:\Windows\System32\Drivers\DriverData', 'FIRSTBOOT': '5/16/2024 2:07:46 PM', 'FPS_BROWSER_APP_PROFILE_STRING': 'Internet Explorer', 'FPS_BROWSER_USER_PROFILE_STRING': 'Default', 'HOMEDRIVE': 'C:', 'HOMEPATH': '\Users\BORDLR', 'LDMS_LOCAL_DIR': 'C:\Program Files (x86)\LANDesk\LDClient\Data', 'LOCALAPPDATA': 'C:\Users\BORDLR\AppData\Local', 'LOGONSERVER': '\\ADPRD01A', 'MACHINETYPE': '1', 'NUMBER_OF_PROCESSORS': '8', 'ONEDRIVE': 'C:\Users\BORDLR\OneDrive', 'OS': 'Windows_NT', 'PATH': 'C:\Users\BORDLR\helics_test\Scripts;C:\Program Files (x86)\VMware\VMware Workstation\bin\;C:\Program Files (x86)\Common Files\Oracle\Java\java8path;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files (x86)\HID Global\ActivClient\;C:\Program Files\HID Global\ActivClient\;C:\Program Files\dotnet\;C:\Users\BORDLR\AppData\Local\Microsoft\WindowsApps;;C:\Program Files\JetBrains\PyCharm 2024.1.2\bin;', 'PATHEXT': '.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC', 'PROCESSOR_ARCHITECTURE': 'AMD64', 'PROCESSOR_IDENTIFIER': 'Intel64 Family 6 Model 142 Stepping 12, GenuineIntel', 'PROCESSOR_LEVEL': '6', 'PROCESSOR_REVISION': '8e0c', 'PROGRAMDATA': 'C:\ProgramData', 'PROGRAMFILES': 'C:\Program Files', 'PROGRAMFILES(X86)': 'C:\Program Files (x86)', 'PROGRAMW6432': 'C:\Program Files', 'PROMPT': '(helics_test) $P$G', 'PSMODULEPATH': 'C:\Program Files\WindowsPowerShell\Modules;C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules', 'PUBLIC': 'C:\Users\Public', 'PYCHARM': 'C:\Program Files\JetBrains\PyCharm 2024.1.2\bin;', 'PYTHONUSERBASE': 'C:\Users\BORDLR\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages', 'SESSIONNAME': 'Console', 'STTASKFOLDERPATH': 'Ivanti\PatchSDK\Deployments', 'SYSTEMDRIVE': 'C:', 'SYSTEMROOT': 'C:\WINDOWS', 'TEMP': 'C:\Users\BORDLR\AppData\Local\Temp\1', 'TMP': 'C:\Users\BORDLR\AppData\Local\Temp\1', 'UATDATA': 'C:\WINDOWS\CCM\UATData\D9F8C395-CAB8-491d-B8AC-179A1FE1BE77', 'USERDNSDOMAIN': 'INEL.GOV', 'USERDOMAIN': 'INEL-NT', 'USERDOMAIN_ROAMINGPROFILE': 'INEL-NT', 'USERNAME': 'BORDLR', 'USERPROFILE': 'C:\Users\BORDLR', 'VIRTUAL_ENV': 'C:\Users\BORDLR\helics_test', 'VIRTUAL_ENV_PROMPT': '(helics_test) ', 'WINDIR': 'C:\WINDOWS', 'ZES_ENABLE_SYSMAN': '1', '_OLD_VIRTUAL_PATH': 'C:\Program Files (x86)\VMware\VMware Workstation\bin\;C:\Program Files (x86)\Common Files\Oracle\Java\java8path;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files (x86)\HID Global\ActivClient\;C:\Program Files\HID Global\ActivClient\;C:\Program Files\dotnet\;C:\Users\BORDLR\AppData\Local\Microsoft\WindowsApps;;C:\Program Files\JetBrains\PyCharm 2024.1.2\bin;', '_OLD_VIRTUAL_PROMPT': '$P$G'}) Traceback (most recent call last): File "C:\Users\BORDLR\helics_test\Scripts\HELICS-Examples-main\user_guide_examples\fundamental\fundamental_default\Battery.py", line 20, in import matplotlib.pyplot as plt ModuleNotFoundError: No module named 'matplotlib'

[trevorhardy]

If you use the Python REPL, is it using the same interpreter?

[phlptp]

At one point a while ago there was some mysterious import conflict with helics and matplotlib. Can you check what happens if you switch the order of the imports?

[LBorden24]

@trevorhardy the interpreter in that failed run is the default interpreter in the base python REPL, when I run the REPL in the helics_test venv I attempted to run the fundamental programs in, it gives a different value, specifically C:\Users\BORDLR\helics_test\Scripts\python.exe

@phlptp When changing the order of the imports it still crashes due to failing to import whichever module is listed first between helics, numpy, and matplotlib. interestingly importing logging seems to succeed.

[trevorhardy]

For what it's worth, I had a colleague on Windows here try this with a clean environment and it worked fine for him. Which doesn't help us but seems to point to this being a more specific problem to this environment.

[trevorhardy]

... 'PATH': 'C:\Users\BORDLR\helics_test\Scripts;C:\Program Files (x86)\VMware\VMware Workstation\bin\;C:\Program Files (x86)\Common Files\Oracle\Java\java8path;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files (x86)\HID Global\ActivClient\;C:\Program Files\HID Global\ActivClient\;C:\Program Files\dotnet\;C:\Users\BORDLR\AppData\Local\Microsoft\WindowsApps;;C:\Program Files\JetBrains\PyCharm 2024.1.2\bin;' ...

Does the PATH above contain where you expect to find, for example, matplotlib? Maybe add a print(sys.path) at the top to see where Python is looking as it might be different than PATH?

[LBorden24]

Good morning Trevor,

The above path does contain the C:\Users\BORDLR\helics_test\Scripts folder which contains within it the Libraries in question: Matplotlib etc. I also tried your suggestion of running sys.path and got different results when run in the REPL and within the program battery.py itself, but although the paths are not the exact same, they both still contain folders with the relevant libraries in them.

[LBorden24]

issue resolved by manually adding the folder with the libraries to the path using sys.path.append