alex-petrenko / faster-fifo

Faster alternative to Python's multiprocessing.Queue (IPC FIFO queue)
MIT License
179 stars 29 forks source link

pip install build fails #30

Closed awarebayes closed 2 years ago

awarebayes commented 2 years ago

Hello! I was trying to install your package but encountered a build error. It happens on python 3.8 and 3.9 on windows (we have a simulator for windows, using it with SampleFactory) because of this line:

https://github.com/alex-petrenko/faster-fifo/blob/8f59ead3206a2a8e8a377e31a09be4a3ac8f993f/setup.py#L36

I will make a PR shortly. Here is the log:

 Traceback (most recent call last):
    File "D:\Code\Anaconda\envs\RL\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 349, in <module>
      main()
    File "D:\Code\Anaconda\envs\RL\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 331, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "D:\Code\Anaconda\envs\RL\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 117, in get_requires_for_build_wheel
      return hook(config_settings)
    File "C:\Users\haker\AppData\Local\Temp\pip-build-env-gf28ize3\overlay\Lib\site-packages\setuptools\build_meta.py", line 162, in get_requires_for_build_wheel
      return self._get_build_requires(
    File "C:\Users\haker\AppData\Local\Temp\pip-build-env-gf28ize3\overlay\Lib\site-packages\setuptools\build_meta.py", line 143, in _get_build_requires
      self.run_setup()
    File "C:\Users\haker\AppData\Local\Temp\pip-build-env-gf28ize3\overlay\Lib\site-packages\setuptools\build_meta.py", line 158, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 36, in <module>
      packages=setuptools.find_packages(where='./', include='faster_fifo*'),
    File "C:\Users\haker\AppData\Local\Temp\pip-build-env-gf28ize3\overlay\Lib\site-packages\setuptools\__init__.py", line 65, in find
      convert_path(where),
    File "C:\Users\haker\AppData\Local\Temp\pip-build-env-gf28ize3\overlay\Lib\site-packages\setuptools\_distutils\util.py", line 130, in convert_path
      raise ValueError("path '%s' cannot end with '/'" % pathname)
awarebayes commented 2 years ago

Oh, i think porting this on windows will be impossible, so closing it...

alex-petrenko commented 2 years ago

Yes, this is rather Posix-specific I'm afraid.

I never tried running SF on Windows, and I can imagine there will be multiple issues. When I was actively developing SF there were massive problems with shared memory tensor support in Pytorch on Windows. If now this problem is resolved, it might be possible to make it work on Windows!

Instead of trying to port faster-fifo, I suggest looking at this: https://github.com/alex-petrenko/sample-factory/blob/master/sample_factory/utils/faster_fifo_stub.py Using regular mp queue should be fine in most cases, unless you're dealing with extreme throughput, 100000 FPS or more.

alex-petrenko commented 2 years ago

I know that there were other people interested in a Windows version. Please let me know if you manage to get it working, or if not what are the hardest issues. I'd really appreciate it!