Cannot open napari (bundled app version) on windows 11

ustcfdm commented 1 year ago

🐛 Bug

Cannot open napari (bundled app version) on windows 11.

To Reproduce

Steps to reproduce the behavior:

Download and install napari bundled app version 0.4.17
Doulbe click icon napari (0.4.17) to open.
A black powershell window appeared just one second then disappeared.

Environment

CPU: 12th Gen Intel(R) Core(TM) i9-12900KF 3.19 GHz RAM 32.0 GB System: 64-bit X64

Windows 11 professional version 22H2 OS version 22621.819

ustcfdm commented 1 year ago

By the way, I installed it on my Surface Pro 4 (windows 10). It can be opened without problem.

Czaki commented 1 year ago

Could you try to run it from cmd to see what error will show?

ustcfdm commented 1 year ago

Could you try to run it from cmd to see what error will show?

Should I run napari.exe in folder napari-0.4.17\envs\napari-0.4.17\Scripts from cmd? Here is what it shows:

Fatal Python error: init_sys_streams: can't initialize sys standard streams
Python runtime state: core initialized
Traceback (most recent call last):
  File "C:\Users\username\scoop\apps\python\current\lib\io.py", line 54, in <module>
ImportError: cannot import name 'text_encoding' from 'io' (unknown location)

It seems to be python version conflict? I have python 3.10 installed using scoop, and system variable PYTHONHOME and PYTHONPATH.

ustcfdm commented 1 year ago

I deleted PYTHONOME and PYTHONPATH, and it works now. But what if I want to keep them?

Czaki commented 1 year ago

I deleted PYTHONOME and PYTHONPATH, and it works now. But what if I want to keep them?

At this moment I do not know, but thanks to your check, we have a good starting point for investigating.

I'm linux man so could not simply validate ideas.

My first idea is to overwrite environment variables in the napari shortcut (set them to empty string) https://netlicensing.io/blog/2012/06/15/set-environment-variables-in-windows-shortcut/

For comparison. On your windows 10 (Surface Pro 4) you have settled these environment variables or not? It is important to know if the problem is specific to windows 11.

ustcfdm commented 1 year ago

For comparison. On your windows 10 (Surface Pro 4) you have settled these environment variables or not? It is important to know if the problem is specific to windows 11.

No, those environment variables are not settled on my windows 10.

Czaki commented 1 year ago

@jaimergp @goanpeca It looks like a question for you. It is possible to add a proper step to bundle to separate it from existing environments on the machine?

jaimergp commented 1 year ago

I thought conda activate (which we run as part of the shortcut) handled that environment sanitization, but apparently it doesn't... We'll have to include some logic in the shortcut creation tool. I am working on updating it as we speak, so I can include a fix for the next release.

I'll come up with a hotfix for you in a bit, we can include an activation script.

jaimergp commented 1 year ago

Try this from any CMD:

% "C:\...\napari-0.4.17\Scripts\conda.exe" env config vars set -p "C:\...\napari-0.4.17\envs\napari-0.4.17" PYTHONPATH="" PYTHONHOME=""

"C:\...\napari-0.4.17 should be absolute path to the root of napari installation.

ustcfdm commented 1 year ago

Try this from any CMD:
% "C:\...\napari-0.4.17\Scripts\conda.exe" env config vars set -p "C:\...\napari-0.4.17\envs\napari-0.4.17" PYTHONPATH="" PYTHONHOME=""
"C:\...\napari-0.4.17 should be absolute path to the root of napari installation.

This does not work on my computer. If I run it from Powershell, it shows the following message:

ForEach-Object : 无法绑定参数“Process”。无法将“System.String”类型的“C:\Users\fengm\AppData\Local\napari-0.4.17\env
s\napari-0.4.17”值转换为“System.Management.Automation.ScriptBlock”类型。
所在位置 行:1 字符: 89
+ ... vars set -p "C:\Users\username\AppData\Local\napari-0.4.17\envs\napari-0 ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [ForEach-Object]，ParameterBindingException
    + FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.PowerShell.Commands.ForEachObjectCommand

If I run it from Command Prompt, it shows the following message:

Fatal Python error: init_sys_streams: can't initialize sys standard streams
Python runtime state: core initialized
Traceback (most recent call last):
  File "C:\Users\username\scoop\apps\python\current\lib\io.py", line 54, in <module>
ImportError: cannot import name 'text_encoding' from 'io' (unknown location)

jaimergp commented 1 year ago

Oh, you need to unset the offending PYTHON* vars before running the command, then. From CMD:

% set "PYTHONHOME="
% set "PYTHONPATH="
% "C:\...\napari-0.4.17\Scripts\conda.exe" env config vars set -p "C:\...\napari-0.4.17\envs\napari-0.4.17" PYTHONPATH="" PYTHONHOME=""

jaimergp commented 1 year ago

@ustcfdm - preview installers are available at https://github.com/napari/packaging/actions/runs/3522319169

If you want to give this a try! It's 'experimental' so some stuff might not work as you'd expect. But hopefully it should have fixed your issue? Let me know, your feedback will be appreciated!

ustcfdm commented 1 year ago

@ustcfdm - preview installers are available at https://github.com/napari/packaging/actions/runs/3522319169

If you want to give this a try! It's 'experimental' so some stuff might not work as you'd expect. But hopefully it should have fixed your issue? Let me know, your feedback will be appreciated!

@jaimergp It seems the issue is not fixed in the preview version. Still the same error.

jaimergp commented 1 year ago

Hm, the shortcuts are supposedly unsetting the variables for you now... would you be available for a 1:1 debugging session, @ustcfdm ? You can book a slot in my calendar here: https://calendly.com/jaimergp-quansight/30-min-call?month=2022-12.

I am going to transfer this issue over to napari/packaging where I'll be doing the work.

jaimergp commented 1 year ago

Windows' conda-standalone was choosing an older menuinst for whatever reason 🤷

Latest available (picked by Linux and macOS): menuinst 2.0a-38_ge820413_h24e6785 napari/label/bundle_tools_2
Windows: menuinst 2.0a-20_g59bd104_h6e9b396 napari/label/bundle_tools_2

I have checked that build 38 does work outside an installer, so I am rebuilding the conda-standalone binaries with stricter requirements. If there are no conflicts, it should be fixed in the next nightly.

jaimergp commented 1 year ago

Well, turns out this is becoming a rabbit hole.

Windows builds for our menuinst have been failing since build 21. In other words, last good one was build 20. That explains why the solver chooses build 20: it's the only one available!
Why is this happening? The CI was all green but it's a false positive! The runner just gave up and said "ye, this is good". I'll need to add some extra checks for artifact existence.
The errors are logging-related. Thousands of blocks like this:

  --- Logging error ---
  Traceback (most recent call last):
    File "C:\Miniconda3\conda-bld\menuinst_1670499904820\_h_env\lib\logging\__init__.py", line 1103, in emit
      stream.write(msg + self.terminator)
  ValueError: underlying buffer has been detached
  Call stack:
    File "C:\Miniconda3\conda-bld\menuinst_1670499904820\_h_env\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 351, in <module>
      main()
    File "C:\Miniconda3\conda-bld\menuinst_1670499904820\_h_env\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 333, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "C:\Miniconda3\conda-bld\menuinst_1670499904820\_h_env\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 249, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "C:\Miniconda3\conda-bld\menuinst_1670499904820\_h_env\lib\site-packages\setuptools\build_meta.py", line 412, in build_wheel
      return self._build_with_temp_dir(['bdist_wheel'], '.whl',
    File "C:\Miniconda3\conda-bld\menuinst_1670499904820\_h_env\lib\site-packages\setuptools\build_meta.py", line 397, in _build_with_temp_dir
      self.run_setup()
    File "C:\Miniconda3\conda-bld\menuinst_1670499904820\_h_env\lib\site-packages\setuptools\build_meta.py", line 335, in run_setup
      exec(code, locals())
    File "<string>", line 33, in <module>
    File "C:\Miniconda3\conda-bld\menuinst_1670499904820\_h_env\lib\site-packages\setuptools\__init__.py", line 87, in setup
      return distutils.core.setup(**attrs)
    File "C:\Miniconda3\conda-bld\menuinst_1670499904820\_h_env\lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
      return run_commands(dist)
    File "C:\Miniconda3\conda-bld\menuinst_1670499904820\_h_env\lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
      dist.run_commands()
    File "C:\Miniconda3\conda-bld\menuinst_1670499904820\_h_env\lib\site-packages\setuptools\_distutils\dist.py", line 968, in run_commands
      self.run_command(cmd)
    File "C:\Miniconda3\conda-bld\menuinst_1670499904820\_h_env\lib\site-packages\setuptools\dist.py", line 1217, in run_command
      super().run_command(command)
    File "C:\Miniconda3\conda-bld\menuinst_1670499904820\_h_env\lib\site-packages\setuptools\_distutils\dist.py", line 987, in run_command
      cmd_obj.run()
    File "C:\Miniconda3\conda-bld\menuinst_1670499904820\_h_env\lib\site-packages\wheel\bdist_wheel.py", line 399, in run
      log.info(f"removing {self.bdist_dir}")
  Message: 'removing build\\bdist.win-amd64\\wheel'
  Arguments: ()

I am now debugging locally to see what's causing this. setuptools? wheel?

jaimergp commented 1 year ago

Hm, in a new attempt this is not observed. But the process dies after running the build script (maybe in the linkage analysis?):

INFO:conda_build.build:Packaging menuinst
Packaging menuinst
INFO:conda_build.build:Packaging menuinst-2.0a-38_ge820413_h6e9b396
Packaging menuinst-2.0a-38_ge820413_h6e9b396
compiling .pyc files...
number of files: 47
Error: Process completed with exit code -1.

jaimergp commented 1 year ago

I think I finished the work needed to fix this. We'll be able to tell in the nightlies of today.

jaimergp commented 1 year ago

Btw, for the curious visitors:

It had to do with a rare bug in conda-build, which choked on the OSX launchers at the linkage analysis phase on Windows. This made it crash with exit code -1. Reported here.
The CI scripts checked for exit code >= 1, which ignored -1 and thus returned a green step. Reported here.

jaimergp commented 1 year ago

With the build issues solved and the latest menuinst being deployed on our installers, the shortcuts include the env var mitigations and are no longer affected by PYTHONHOME or PYTHONPATH setups. Yesterday's nightlies (~11h ago) contain the fixes.

Thanks for the report @ustcfdm, this led me to find some issues in the pipelines that needed fixes! Closing this now.

napari / packaging