Closed anotherkhan closed 1 year ago
What do you get if you do: $ pip freeze | grep hsds
?
Thanks for quick reply.
> pip freeze | findstr hsds
hsds==0.7.0b11
Thanks for replacing 'grep' with 'findstr'. :)
That's strange. Are you using Anaconda for your python environment? What happens if you just run "hsds" in the Anaconda command window? You should see a hsds message: "Usage: starts hsds.."
Yes I am using Anaconda(miniconda3)
>hsds
usage: Starts hsds a REST-based service for HDF5 data.
hsds: error: one of the arguments --root_dir --bucket_name is required
If I do a runall.sh from the git bash prompt on Windows 10, it finds hsds, but I get an error: ValueError: Unsupported signal: 2`. So even with the script finding hsds (likely dependent on you've enabled Anaconda for the Git Mash Mingw environment), there are still issues...
The no-docker option uses Python sockets which apparently aren't totally compatible between Windows and Max/Linux. From: https://docs.python.org/3/library/socket.html: "Note Some behavior may be platform dependent, since calls are made to the operating system socket APIs."
So it seems like supporting no-docker on Windows would be a bit of work.
For now, I'd recommend running HSDS on Windows with Docker. I'll try this out now myself with Docker Desktop 4.6.0 and see if any changes are needed to the Docker install guide (https://github.com/HDFGroup/hsds/blob/master/docs/docker_install_posix.md)
Thank you for you help. As recommended I tried running HSDS on Windows with Docker. Docker Desktop version 4.5.1.
After building docker custom image I am unable to start server.
$ ./runall.sh
no AWS or AZURE env set, using POSIX storage
creating directory C:\Users\user\hsds\hsds_data/C:\Users\sami.a.khan\hsds\hsds_data\hsdstest
mkdir: cannot create directory ‘C:\\Users\\user\\hsds\\hsds_data/C:\\Users\\user\\hsds\\hsds_data\\hsdstest’: No such file or directory
Running docker-compose -f admin/docker/docker-compose.posix.yml up
Creating network "hsds_default" with the default driver
Creating hsds_head_1 ...
Creating hsds_head_1 ... done
Creating hsds_sn_1 ...
Creating hsds_rangeget_1 ...
Creating hsds_sn_1 ... done
Creating hsds_rangeget_1 ... done
Creating hsds_dn_1 ...
Creating hsds_dn_2 ...
Creating hsds_dn_3 ...
Creating hsds_dn_4 ...
Creating hsds_dn_1 ... done
Creating hsds_dn_3 ... done
Creating hsds_dn_2 ... done
Creating hsds_dn_4 ... done
1: waiting for server startup (status: 000)
2: waiting for server startup (status: 000)
3: waiting for server startup (status: 000)
4: waiting for server startup (status: 000)
Thanks for trying with Docker Desktop. I wasn't successful myself -- my Windows box is a bit old and the cpu doesn't have the hyper-v support for Docker Desktop.
With the runall.sh, it looks like the script is not doing the right thing with the windows file paths. In the Git Bash shell, try setting the ROOT_DIR using the equivalent Linux path. E.g.: $ export ROOT_DIR=/c/Users/sami.a.khan/hsds/hsds_data/hsdstest
and then do a runall.sh.
I guess it would be best if there was a windows bat script that you could run from the windows cmd prompt to start the server. Would be more than happy if you'd like to contribute it!
BTW, I'm curious if anyone has tried running HSDS on windows using WSL2 (windows subsystem for Linux). By all accounts (see https://medium.com/swlh/data-science-using-wsl2-seamless-integration-of-ubuntu-and-windows-10-dbe94cbac2b6) its very convenient for running Linux apps on Windows. I expect the --no-docker option would "just work" on WSL2, but don't have a way to test it myself.
Thanks, I set ROOT_DIR path mentioned above and then executed runall.sh in GIT bash. No issues with as earlier with creating a directory but stuck on waiting for server to startup.
$ ./runall.sh
no AWS or AZURE env set, using POSIX storage
creating directory /c/Users/sami.a.khan/hsds/hsds_data/hsdstest/hsdstest
Running docker-compose -f admin/docker/docker-compose.posix.yml up
Recreating hsds_head_1 ...
Recreating hsds_head_1 ... done
Recreating hsds_rangeget_1 ...
Recreating hsds_sn_1 ...
Recreating hsds_rangeget_1 ... done
Recreating hsds_dn_1 ...
Recreating hsds_dn_2 ...
Recreating hsds_dn_3 ...
Recreating hsds_dn_4 ...
Recreating hsds_sn_1 ... done
Recreating hsds_dn_3 ... done
Recreating hsds_dn_1 ... done
Recreating hsds_dn_2 ... done
Recreating hsds_dn_4 ... done
1: waiting for server startup (status: 000)
2: waiting for server startup (status: 000)
3: waiting for server startup (status: 000)
4: waiting for server startup (status: 000)
5: waiting for server startup (status: 000)
6: waiting for server startup (status: 000)
7: waiting for server startup (status: 000)
Are the containers starting? Run $docker ps
to get status.
If so, run $docker logs hsds_sn_1
and check to see if any errors are showing up.
Please see the logs from above docker command execution
>docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
43820fde4d8a hdfgroup/hsds "/bin/bash -c 'sourc…" About a minute ago Restarting (126) 2 seconds ago hsds_dn_2
9db32e14909e hdfgroup/hsds "/bin/bash -c 'sourc…" About a minute ago Restarting (126) 13 seconds ago hsds_dn_4
ba9a666a9226 hdfgroup/hsds "/bin/bash -c 'sourc…" About a minute ago Restarting (126) 10 seconds ago hsds_dn_1
7157e91555dc hdfgroup/hsds "/bin/bash -c 'sourc…" About a minute ago Restarting (126) 10 seconds ago hsds_dn_3
4ede901fc00d hdfgroup/hsds "/bin/bash -c 'sourc…" About a minute ago Restarting (126) 8 seconds ago hsds_rangeget_1
426cfb88f763 hdfgroup/hsds "/bin/bash -c 'sourc…" About a minute ago Restarting (126) 12 seconds ago hsds_sn_1
79a41bd29910 hdfgroup/hsds "/bin/bash -c 'sourc…" About a minute ago Restarting (126) 21 seconds ago hsds_head_1
>docker logs hsds_sn_1
/bin/bash: /entrypoint.sh: /bin/bash^M: bad interpreter: No such file or directory
/bin/bash: /entrypoint.sh: /bin/bash^M: bad interpreter: No such file or directory
/bin/bash: /entrypoint.sh: /bin/bash^M: bad interpreter: No such file or directory
/bin/bash: /entrypoint.sh: /bin/bash^M: bad interpreter: No such file or directory
/bin/bash: /entrypoint.sh: /bin/bash^M: bad interpreter: No such file or directory
/bin/bash: /entrypoint.sh: /bin/bash^M: bad interpreter: No such file or directory
/bin/bash: /entrypoint.sh: /bin/bash^M: bad interpreter: No such file or directory
/bin/bash: /entrypoint.sh: /bin/bash^M: bad interpreter: No such file or directory
/bin/bash: /entrypoint.sh: /bin/bash^M: bad interpreter: No such file or directory
/bin/bash: /entrypoint.sh: /bin/bash^M: bad interpreter: No such file or directory
Strange!
What do you get with: $ docker images hdfgroup/hsds
?
Please see below:
$ docker images hdfgroup/hsds
REPOSITORY TAG IMAGE ID CREATED SIZE
hdfgroup/hsds latest 12a310c694d8 32 hours ago 486MB
Looks like you built the image locally - likely building on Windows introduced some unexpected weirdness.
Try fetching the image from DockerHub. Do this:
$ ./stopall.sh
$ docker rmi 12a310c694d8
$ ./runall.sh
This should fetch the image from the hsds repo: https://hub.docker.com/repository/docker/hdfgroup/hsds/general.
I've made some updates in the nodocker branch that should allow HSDS to run without Docker Desktop. Here's how to try it out:
git pull
git checkout nodocker
python uninstall hsds
python setup.py install
runall.bat
From here you can follow the regular post setup instructions. To kill the server, do a ctrl-C and say "Y" and answer "Y" to "Terminate batch job?"
Will be most curious if anyone would like to report how this works for them. Since at least the version of windows I have doesn't support Unix Sockets (but see: https://bugs.python.org/issue33408), the runall.bat script runs HSDS without docker but using TCP/IP connections. This should work on MacOS/Linux too for anyone who wants to support cross-machine communication, but doesn't want to run Docker.
BTW, I'm curious if anyone has tried running HSDS on windows using WSL2 (windows subsystem for Linux). By all accounts (see https://medium.com/swlh/data-science-using-wsl2-seamless-integration-of-ubuntu-and-windows-10-dbe94cbac2b6) its very convenient for running Linux apps on Windows. I expect the --no-docker option would "just work" on WSL2, but don't have a way to test it myself.
I have! It works fine. Have not done much with it, but I do remember I got it to work and was able to use hsload to interact with it.
Thank you @jreadey for making the update. I followed your instruction above. The server starts but shuts down immediately with following error.
(base) C:\Users\sami.a.khan\hsds>hsds --root_dir data --host localhost --port 5101 --password_file admin/config/passwd.txt --logfile hs.log --loglevel DEBUG --config_dir=admin/config --count=4
set logging to: 10
logfile: C:\Users\sami.a.khan\hsds\hs.log
DEBUG:root:looking for hsds-servicenode in PATH env var folders
INFO:root:using cmd_dir: C:\Users\sami.a.khan\Miniconda3\Scripts
endpoint: http://localhost:5101
using cmd_path: C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\__init__.py", line 23, in <module>
from . import multiarray
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\multiarray.py", line 10, in <module>
from . import overrides
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\overrides.py", line 6, in <module>
from numpy.core._multiarray_umath import (
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
return ep.load()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
return self.resolve()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
from . import servicenode
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 19, in <module>
from .util.lruCache import LruCache
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\util\lruCache.py", line 12, in <module>
import numpy
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\__init__.py", line 144, in <module>
from . import core
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\__init__.py", line 49, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.7 from "C:\Users\sami.a.khan\Miniconda3\python.exe"
* The NumPy version is: "1.22.3"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: No module named 'numpy.core._multiarray_umath'
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\__init__.py", line 23, in <module>
from . import multiarray
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\multiarray.py", line 10, in <module>
from . import overrides
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\overrides.py", line 6, in <module>
from numpy.core._multiarray_umath import (
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
return ep.load()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
return self.resolve()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
from . import servicenode
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 19, in <module>
from .util.lruCache import LruCache
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\util\lruCache.py", line 12, in <module>
import numpy
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\__init__.py", line 144, in <module>
from . import core
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\__init__.py", line 49, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.7 from "C:\Users\sami.a.khan\Miniconda3\python.exe"
* The NumPy version is: "1.22.3"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: No module named 'numpy.core._multiarray_umath'
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\__init__.py", line 23, in <module>
from . import multiarray
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\multiarray.py", line 10, in <module>
from . import overrides
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\overrides.py", line 6, in <module>
from numpy.core._multiarray_umath import (
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
return ep.load()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
return self.resolve()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
from . import servicenode
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 19, in <module>
from .util.lruCache import LruCache
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\util\lruCache.py", line 12, in <module>
import numpy
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\__init__.py", line 144, in <module>
from . import core
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\__init__.py", line 49, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.7 from "C:\Users\sami.a.khan\Miniconda3\python.exe"
* The NumPy version is: "1.22.3"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: No module named 'numpy.core._multiarray_umath'
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\__init__.py", line 23, in <module>
from . import multiarray
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\multiarray.py", line 10, in <module>
from . import overrides
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\overrides.py", line 6, in <module>
from numpy.core._multiarray_umath import (
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
return ep.load()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
return self.resolve()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
from . import servicenode
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 19, in <module>
from .util.lruCache import LruCache
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\util\lruCache.py", line 12, in <module>
import numpy
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\__init__.py", line 144, in <module>
from . import core
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\__init__.py", line 49, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.7 from "C:\Users\sami.a.khan\Miniconda3\python.exe"
* The NumPy version is: "1.22.3"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: No module named 'numpy.core._multiarray_umath'
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\__init__.py", line 23, in <module>
from . import multiarray
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\multiarray.py", line 10, in <module>
from . import overrides
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\overrides.py", line 6, in <module>
from numpy.core._multiarray_umath import (
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
return ep.load()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
return self.resolve()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
from . import servicenode
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 19, in <module>
from .util.lruCache import LruCache
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\util\lruCache.py", line 12, in <module>
import numpy
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\__init__.py", line 144, in <module>
from . import core
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\__init__.py", line 49, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.7 from "C:\Users\sami.a.khan\Miniconda3\python.exe"
* The NumPy version is: "1.22.3"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: No module named 'numpy.core._multiarray_umath'
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\__init__.py", line 23, in <module>
from . import multiarray
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\multiarray.py", line 10, in <module>
from . import overrides
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\overrides.py", line 6, in <module>
from numpy.core._multiarray_umath import (
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
return ep.load()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
return self.resolve()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
from . import servicenode
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 19, in <module>
from .util.lruCache import LruCache
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\util\lruCache.py", line 12, in <module>
import numpy
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\__init__.py", line 144, in <module>
from . import core
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numpy\core\__init__.py", line 49, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.7 from "C:\Users\sami.a.khan\Miniconda3\python.exe"
* The NumPy version is: "1.22.3"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: No module named 'numpy.core._multiarray_umath'
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
INFO:root:all processes ready!
INFO:root:Ready after: 5.09 s
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
Exception in thread Thread-3:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
Exception in thread Thread-4:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
Exception in thread Thread-2:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
Exception in thread Thread-5:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
Exception in thread Thread-6:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
READY! use endpoint: http://localhost:5101
got exception: I/O operation on closed file.
shutting down server
INFO:root:hsds app stop at 1648241168.5733004
DEBUG:root:all subprocesses exited
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-script.py", line 33, in <module>
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds')())
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\app.py", line 303, in main
app.stop()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 364, in stop
logging.debug("all subprocesses exited")
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 2002, in debug
root.debug(msg, *args, **kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1366, in debug
self._log(DEBUG, msg, args, **kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1514, in _log
self.handle(record)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1524, in handle
self.callHandlers(record)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1586, in callHandlers
hdlr.handle(record)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 894, in handle
self.emit(record)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1028, in emit
stream.write(msg + self.terminator)
KeyboardInterrupt
INFO:root:hsds app stop at 1648241168.6051414
DEBUG:root:all subprocesses exited
Sounds like something is out of whack with your numpy setup. Try this:
$ pip uninstall numpy
$ conda install numpy
@jreadey, Thank you for quick reply. I update numpy as mentioned above. no more numpy related errors but server starts and shuts down.
>runall.bat
(base) C:\Users\sami.a.khan\hsds>hsds --root_dir data --host localhost --port 5101 --password_file admin/config/passwd.txt --logfile hs.log --loglevel DEBUG --config_dir=admin/config --count=4
set logging to: 10
logfile: C:\Users\sami.a.khan\hsds\hs.log
DEBUG:root:looking for hsds-servicenode in PATH env var folders
INFO:root:using cmd_dir: C:\Users\sami.a.khan\Miniconda3\Scripts
endpoint: http://localhost:5101
using cmd_path: C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
INFO:root:all processes ready!
INFO:root:Ready after: 5.05 s
READY! use endpoint: http://localhost:5101
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
Traceback (most recent call last):
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
return ep.load() File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
return get_distribution(dist).load_entry_point(group, name)
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
return ep.load()
return get_distribution(dist).load_entry_point(group, name)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
return get_distribution(dist).load_entry_point(group, name) return get_distribution(dist).load_entry_point(group, name)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
return self.resolve()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
return ep.load()
return ep.load()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
return get_distribution(dist).load_entry_point(group, name)
return self.resolve() File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
module = __import__(self.module_name, fromlist=['__name__'], level=0) File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
return self.resolve() return self.resolve()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
return ep.load() File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
return ep.load() File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
from . import servicenode
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
return self.resolve()module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 24, in <module>
return self.resolve()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
from .basenode import healthCheck, baseInit
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
from . import servicenode File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\basenode.py", line 21, in <module>
from . import servicenode
from . import servicenode
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 24, in <module>
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 24, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 24, in <module>
module = __import__(self.module_name, fromlist=['__name__'], level=0) File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
import psutil
from .basenode import healthCheck, baseInitfrom .basenode import healthCheck, baseInit
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
from .basenode import healthCheck, baseInit
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\basenode.py", line 21, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\basenode.py", line 21, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\psutil\__init__.py", line 109, in <module>
import psutil
from . import servicenode
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 24, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\psutil\__init__.py", line 109, in <module>
from . import servicenode File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\basenode.py", line 21, in <module>
import psutil from . import _pswindows as _psplatform
from .basenode import healthCheck, baseInit
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 24, in <module>
from . import _pswindows as _psplatform File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\psutil\__init__.py", line 109, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\psutil\_pswindows.py", line 35, in <module>
import psutil
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\basenode.py", line 21, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\psutil\_pswindows.py", line 35, in <module>
from .basenode import healthCheck, baseInitfrom . import _pswindows as _psplatform File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\psutil\__init__.py", line 109, in <module>
from ._psutil_windows import ABOVE_NORMAL_PRIORITY_CLASS
import psutil
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\basenode.py", line 21, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\psutil\_pswindows.py", line 35, in <module>
from ._psutil_windows import ABOVE_NORMAL_PRIORITY_CLASS
ModuleNotFoundErrorfrom . import _pswindows as _psplatform
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\psutil\__init__.py", line 109, in <module>
:
import psutilModuleNotFoundErrorfrom ._psutil_windows import ABOVE_NORMAL_PRIORITY_CLASSNo module named 'psutil._psutil_windows' File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\psutil\_pswindows.py", line 35, in <module>
from . import _pswindows as _psplatform
:
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\psutil\__init__.py", line 109, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\psutil\_pswindows.py", line 35, in <module>
No module named 'psutil._psutil_windows'from ._psutil_windows import ABOVE_NORMAL_PRIORITY_CLASS
ModuleNotFoundError ModuleNotFoundError: from . import _pswindows as _psplatformfrom ._psutil_windows import ABOVE_NORMAL_PRIORITY_CLASS: No module named 'psutil._psutil_windows'
No module named 'psutil._psutil_windows'
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\psutil\_pswindows.py", line 35, in <module>
ModuleNotFoundError
: from ._psutil_windows import ABOVE_NORMAL_PRIORITY_CLASSNo module named 'psutil._psutil_windows'
ModuleNotFoundError: No module named 'psutil._psutil_windows'
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
Exception in thread Thread-2:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
Exception in thread Thread-3:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
Exception in thread Thread-4:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
Exception in thread Thread-5:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
Exception in thread Thread-6:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
got exception: I/O operation on closed file.
shutting down server
INFO:root:hsds app stop at 1648242296.7249045
DEBUG:root:all subprocesses exited
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-script.py", line 33, in <module>
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds')())
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\app.py", line 303, in main
app.stop()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 364, in stop
logging.debug("all subprocesses exited")
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 2002, in debug
root.debug(msg, *args, **kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1366, in debug
self._log(DEBUG, msg, args, **kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1514, in _log
self.handle(record)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1524, in handle
self.callHandlers(record)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1586, in callHandlers
hdlr.handle(record)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 894, in handle
self.emit(record)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1028, in emit
stream.write(msg + self.terminator)
KeyboardInterrupt
INFO:root:hsds app stop at 1648242296.761941
DEBUG:root:all subprocesses exited
Some progress!
Try the same trick with psutil:
$ pip uninstall psutil
$ conda install psutil
Thanks :-) @jreadey. Do you recommend to reinstall miniconda?
runall.bat
(base) C:\Users\sami.a.khan\hsds>hsds --root_dir data --host localhost --port 5101 --password_file admin/config/passwd.txt --logfile hs.log --loglevel DEBUG --config_dir=admin/config --count=4
set logging to: 10
logfile: C:\Users\sami.a.khan\hsds\hs.log
DEBUG:root:looking for hsds-servicenode in PATH env var folders
INFO:root:using cmd_dir: C:\Users\sami.a.khan\Miniconda3\Scripts
endpoint: http://localhost:5101
using cmd_path: C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')()) sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')()) File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
return get_distribution(dist).load_entry_point(group, name) File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)return get_distribution(dist).load_entry_point(group, name)
return get_distribution(dist).load_entry_point(group, name)return get_distribution(dist).load_entry_point(group, name)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
return get_distribution(dist).load_entry_point(group, name) File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
return ep.load() File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
return ep.load()return ep.load()return ep.load()return ep.load()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
return ep.load()
return self.resolve()return self.resolve()return self.resolve() File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
return self.resolve() File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
return self.resolve()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
return self.resolve()module = __import__(self.module_name, fromlist=['__name__'], level=0) File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
module = __import__(self.module_name, fromlist=['__name__'], level=0) File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
module = __import__(self.module_name, fromlist=['__name__'], level=0)
from . import servicenodemodule = __import__(self.module_name, fromlist=['__name__'], level=0)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
from . import servicenode
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
from . import servicenode
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 27, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 27, in <module>
from . import servicenode File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 27, in <module>
from . import servicenode
from .domain_sn import GET_Domain, PUT_Domain, DELETE_Domain, GET_Domainsfrom . import servicenodefrom .domain_sn import GET_Domain, PUT_Domain, DELETE_Domain, GET_Domains
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 27, in <module>
from .domain_sn import GET_Domain, PUT_Domain, DELETE_Domain, GET_Domains
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 27, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 27, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\domain_sn.py", line 39, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\domain_sn.py", line 39, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\domain_sn.py", line 39, in <module>
from .domain_sn import GET_Domain, PUT_Domain, DELETE_Domain, GET_Domains from .domain_sn import GET_Domain, PUT_Domain, DELETE_Domain, GET_Domainsfrom .domain_sn import GET_Domain, PUT_Domain, DELETE_Domain, GET_Domains
from .util.storUtil import getStorKeys, getCompressorsfrom .util.storUtil import getStorKeys, getCompressors
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\domain_sn.py", line 39, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\domain_sn.py", line 39, in <module>
from .util.storUtil import getStorKeys, getCompressors File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\domain_sn.py", line 39, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\util\storUtil.py", line 20, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\util\storUtil.py", line 20, in <module>
from .util.storUtil import getStorKeys, getCompressors File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\util\storUtil.py", line 20, in <module>
from .util.storUtil import getStorKeys, getCompressorsfrom .util.storUtil import getStorKeys, getCompressorsimport numcodecs as codecs
import numcodecs as codecs
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\util\storUtil.py", line 20, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\util\storUtil.py", line 20, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\util\storUtil.py", line 20, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\__init__.py", line 100, in <module>
import numcodecs as codecs File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\__init__.py", line 100, in <module>
import numcodecs as codecsimport numcodecs as codecs File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\__init__.py", line 100, in <module>
import numcodecs as codecsfrom numcodecs.shuffle import Shufflefrom numcodecs.shuffle import Shuffle
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\__init__.py", line 100, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\__init__.py", line 100, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\__init__.py", line 100, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\shuffle.py", line 4, in <module>
from numcodecs.shuffle import Shuffle File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\shuffle.py", line 4, in <module>
from numcodecs.shuffle import Shuffle
from numcodecs.shuffle import Shuffle
from numcodecs.shuffle import Shuffle File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\shuffle.py", line 4, in <module>
from ._shuffle import _doShuffle, _doUnshuffle File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\shuffle.py", line 4, in <module>
from ._shuffle import _doShuffle, _doUnshuffle File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\shuffle.py", line 4, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\shuffle.py", line 4, in <module>
from ._shuffle import _doShuffle, _doUnshuffleModuleNotFoundError from ._shuffle import _doShuffle, _doUnshuffleModuleNotFoundError
: from ._shuffle import _doShuffle, _doUnshuffle
: from ._shuffle import _doShuffle, _doUnshuffle
No module named 'numcodecs._shuffle'
ModuleNotFoundErrorNo module named 'numcodecs._shuffle'ModuleNotFoundErrorModuleNotFoundError
ModuleNotFoundError:
: : : No module named 'numcodecs._shuffle'No module named 'numcodecs._shuffle'No module named 'numcodecs._shuffle'No module named 'numcodecs._shuffle'
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
INFO:root:all processes ready!
INFO:root:Ready after: 5.09 s
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
Exception in thread Thread-2:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
Exception in thread Thread-3:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
Exception in thread Thread-4:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
Exception in thread Thread-5:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
Exception in thread Thread-6:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
READY! use endpoint: http://localhost:5101
got exception: I/O operation on closed file.
shutting down server
INFO:root:hsds app stop at 1648243049.8040502
DEBUG:root:all subprocesses exited
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-script.py", line 33, in <module>
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds')())
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\app.py", line 303, in main
app.stop()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 364, in stop
logging.debug("all subprocesses exited")
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 2002, in debug
root.debug(msg, *args, **kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1366, in debug
self._log(DEBUG, msg, args, **kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1514, in _log
self.handle(record)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1524, in handle
self.callHandlers(record)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1586, in callHandlers
hdlr.handle(record)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 894, in handle
self.emit(record)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1028, in emit
stream.write(msg + self.terminator)
KeyboardInterrupt
INFO:root:hsds app stop at 1648243049.8315725
(base) C:\Users\sami.a.khan\hsds>hsds --root_dir data --host localhost --port 5101 --password_file admin/config/passwd.txt --logfile hs.log --loglevel DEBUG --config_dir=admin/config --count=4
set logging to: 10
logfile: C:\Users\sami.a.khan\hsds\hs.log
DEBUG:root:looking for hsds-servicenode in PATH env var folders
INFO:root:using cmd_dir: C:\Users\sami.a.khan\Miniconda3\Scripts
endpoint: http://localhost:5101
using cmd_path: C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-node-script.py", line 33, in <module>
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')()) sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')())
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds-node')()) File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
return get_distribution(dist).load_entry_point(group, name) File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 474, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)return get_distribution(dist).load_entry_point(group, name)
return get_distribution(dist).load_entry_point(group, name)return get_distribution(dist).load_entry_point(group, name)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
return get_distribution(dist).load_entry_point(group, name) File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
return ep.load() File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2846, in load_entry_point
return ep.load()return ep.load()return ep.load()return ep.load()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
return ep.load()
return self.resolve()return self.resolve()return self.resolve() File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2450, in load
return self.resolve() File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
return self.resolve()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
return self.resolve()module = __import__(self.module_name, fromlist=['__name__'], level=0) File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\pkg_resources\__init__.py", line 2456, in resolve
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
module = __import__(self.module_name, fromlist=['__name__'], level=0) File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
module = __import__(self.module_name, fromlist=['__name__'], level=0)
from . import servicenodemodule = __import__(self.module_name, fromlist=['__name__'], level=0)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
from . import servicenode
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
from . import servicenode
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 27, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\node_runner.py", line 5, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 27, in <module>
from . import servicenode File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 27, in <module>
from . import servicenode
from .domain_sn import GET_Domain, PUT_Domain, DELETE_Domain, GET_Domainsfrom . import servicenodefrom .domain_sn import GET_Domain, PUT_Domain, DELETE_Domain, GET_Domains
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 27, in <module>
from .domain_sn import GET_Domain, PUT_Domain, DELETE_Domain, GET_Domains
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 27, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\servicenode.py", line 27, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\domain_sn.py", line 39, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\domain_sn.py", line 39, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\domain_sn.py", line 39, in <module>
from .domain_sn import GET_Domain, PUT_Domain, DELETE_Domain, GET_Domains from .domain_sn import GET_Domain, PUT_Domain, DELETE_Domain, GET_Domainsfrom .domain_sn import GET_Domain, PUT_Domain, DELETE_Domain, GET_Domains
from .util.storUtil import getStorKeys, getCompressorsfrom .util.storUtil import getStorKeys, getCompressors
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\domain_sn.py", line 39, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\domain_sn.py", line 39, in <module>
from .util.storUtil import getStorKeys, getCompressors File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\domain_sn.py", line 39, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\util\storUtil.py", line 20, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\util\storUtil.py", line 20, in <module>
from .util.storUtil import getStorKeys, getCompressors File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\util\storUtil.py", line 20, in <module>
from .util.storUtil import getStorKeys, getCompressorsfrom .util.storUtil import getStorKeys, getCompressorsimport numcodecs as codecs
import numcodecs as codecs
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\util\storUtil.py", line 20, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\util\storUtil.py", line 20, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\util\storUtil.py", line 20, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\__init__.py", line 100, in <module>
import numcodecs as codecs File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\__init__.py", line 100, in <module>
import numcodecs as codecsimport numcodecs as codecs File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\__init__.py", line 100, in <module>
import numcodecs as codecsfrom numcodecs.shuffle import Shufflefrom numcodecs.shuffle import Shuffle
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\__init__.py", line 100, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\__init__.py", line 100, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\__init__.py", line 100, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\shuffle.py", line 4, in <module>
from numcodecs.shuffle import Shuffle File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\shuffle.py", line 4, in <module>
from numcodecs.shuffle import Shuffle
from numcodecs.shuffle import Shuffle
from numcodecs.shuffle import Shuffle File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\shuffle.py", line 4, in <module>
from ._shuffle import _doShuffle, _doUnshuffle File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\shuffle.py", line 4, in <module>
from ._shuffle import _doShuffle, _doUnshuffle File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\shuffle.py", line 4, in <module>
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\numcodecs\shuffle.py", line 4, in <module>
from ._shuffle import _doShuffle, _doUnshuffleModuleNotFoundError from ._shuffle import _doShuffle, _doUnshuffleModuleNotFoundError
: from ._shuffle import _doShuffle, _doUnshuffle
: from ._shuffle import _doShuffle, _doUnshuffle
No module named 'numcodecs._shuffle'
ModuleNotFoundErrorNo module named 'numcodecs._shuffle'ModuleNotFoundErrorModuleNotFoundError
ModuleNotFoundError:
: : : No module named 'numcodecs._shuffle'No module named 'numcodecs._shuffle'No module named 'numcodecs._shuffle'No module named 'numcodecs._shuffle'
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
DEBUG:root:0/6 ready
DEBUG:root:sleeping for 1
INFO:root:all processes ready!
INFO:root:Ready after: 5.09 s
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
Exception in thread Thread-2:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
Exception in thread Thread-3:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
Exception in thread Thread-4:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
WARNING:root:process C:\Users\sami.a.khan\Miniconda3\python.exe ended, result: ('', None)
Exception in thread Thread-5:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
Exception in thread Thread-6:
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\sami.a.khan\Miniconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 15, in _enqueue_output
for line in iter(out.readline, b''):
ValueError: I/O operation on closed file
READY! use endpoint: http://localhost:5101
got exception: I/O operation on closed file.
shutting down server
INFO:root:hsds app stop at 1648243049.8040502
DEBUG:root:all subprocesses exited
Traceback (most recent call last):
File "C:\Users\sami.a.khan\Miniconda3\Scripts\hsds-script.py", line 33, in <module>
sys.exit(load_entry_point('hsds==0.7.0b11', 'console_scripts', 'hsds')())
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\app.py", line 303, in main
app.stop()
File "C:\Users\sami.a.khan\Miniconda3\lib\site-packages\hsds-0.7.0b11-py3.7.egg\hsds\hsds_app.py", line 364, in stop
logging.debug("all subprocesses exited")
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 2002, in debug
root.debug(msg, *args, **kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1366, in debug
self._log(DEBUG, msg, args, **kwargs)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1514, in _log
self.handle(record)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1524, in handle
self.callHandlers(record)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1586, in callHandlers
hdlr.handle(record)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 894, in handle
self.emit(record)
File "C:\Users\sami.a.khan\Miniconda3\lib\logging\__init__.py", line 1028, in emit
stream.write(msg + self.terminator)
KeyboardInterrupt
INFO:root:hsds app stop at 1648243049.8315725
Actually, I get the same error you are seeing when I try running setup.py in a clean Anaconda environment. Looks like newer version of some packages has introduced incompatibilities.
Let me take a look at this and see if I can fix up the install script.
It seems the newer version of aiobotocore is the culprit. I've updated the setup.py script to use aiobotcore==2.1.0
. This is working for me with both MacOS and Windows. Please give it a try and let me know how it goes.
Reminder: to run on Windows, use the runall.bat script.
Closing this issue even though I'm not 100% on HSDS running in Windows (and have no means to verify).
In general, I'd recommend using WSL2 on Windows as offering the best approach (as @bilalshaikh42 verified).
Just put in some fixes for HSDS on Windows - it should be running correctly now. Docker on Windows is not supported, but you "runall.bat" (i.e. running the hsds app) works. The github python-package workflow runs on Linux Ubuntu, Mac, and Windows-latest, so should catch any future regressions.
I am following quick start steps to install HSDS server on my local machine (Windows 10) without docker.
When I execute ./runall.sh --no-docker from Git Bash, I receive following error: