realitix / vulkan

The ultimate Python binding for Vulkan API
Apache License 2.0
505 stars 46 forks source link

Upgrade to Vulkan 1.3 #77

Closed julianfl0w closed 10 months ago

julianfl0w commented 2 years ago

Testing in progress

realitix commented 2 years ago

Hello @julianfl0w , do you have others commits to do or is your pull request ready ? Thanks for the good work !

julianfl0w commented 2 years ago

@realitix i have nothing more to add

okuma10 commented 2 years ago

will this be merged with pyvulkan?

okuma10 commented 2 years ago

@realitix?

realitix commented 2 years ago

I'm testing right now to check the test works

realitix commented 2 years ago

Sorry but I have errors with your pr:

How to reproduce the error:

  1. clone your pr: git clone https://github.com/julianfl0w/vulkan.git
  2. create vent: python3 -m venv venv
  3. load venv: . venv/bin/activate
  4. install: python setup.py install
Error:
/usr/lib/python3.10/distutils/dist.py:274: UserWarning: Unknown distribution option: 'long_descripiton'
  warnings.warn(msg)
Traceback (most recent call last):
  File "/home/realitix/git/vulkan/.eggs/cffi-1.15.0-py3.10-linux-x86_64.egg/cffi/cparser.py", line 336, in _parse
    ast = _get_parser().parse(fullcsource)
  File "/home/realitix/git/vulkan/.eggs/pycparser-2.21-py3.10.egg/pycparser/c_parser.py", line 147, in parse
    return self.cparser.parse(
  File "/home/realitix/git/vulkan/.eggs/pycparser-2.21-py3.10.egg/pycparser/ply/yacc.py", line 331, in parse
    return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)
  File "/home/realitix/git/vulkan/.eggs/pycparser-2.21-py3.10.egg/pycparser/ply/yacc.py", line 1199, in parseopt_notrack
    tok = call_errorfunc(self.errorfunc, errtoken, self)
  File "/home/realitix/git/vulkan/.eggs/pycparser-2.21-py3.10.egg/pycparser/ply/yacc.py", line 193, in call_errorfunc
    r = errorfunc(token)
  File "/home/realitix/git/vulkan/.eggs/pycparser-2.21-py3.10.egg/pycparser/c_parser.py", line 1931, in p_error
    self._parse_error(
  File "/home/realitix/git/vulkan/.eggs/pycparser-2.21-py3.10.egg/pycparser/plyparser.py", line 67, in _parse_error
    raise ParseError("%s: %s" % (coord, msg))
pycparser.plyparser.ParseError: <cdef source string>:11240:5: before: HMONITOR

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/realitix/git/vulkan/setup.py", line 8, in <module>
    setup(
  File "/home/realitix/git/vulkan/venv/lib/python3.10/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.10/distutils/core.py", line 108, in setup
    _setup_distribution = dist = klass(attrs)
  File "/home/realitix/git/vulkan/venv/lib/python3.10/site-packages/setuptools/dist.py", line 453, in __init__
    _Distribution.__init__(
  File "/usr/lib/python3.10/distutils/dist.py", line 292, in __init__
    self.finalize_options()
  File "/home/realitix/git/vulkan/venv/lib/python3.10/site-packages/setuptools/dist.py", line 831, in finalize_options
    ep(self)
  File "/home/realitix/git/vulkan/venv/lib/python3.10/site-packages/setuptools/dist.py", line 852, in _finalize_setup_keywords
    ep.load()(self, ep.name, value)
  File "/home/realitix/git/vulkan/.eggs/cffi-1.15.0-py3.10-linux-x86_64.egg/cffi/setuptools_ext.py", line 219, in cffi_modules
    add_cffi_module(dist, cffi_module)
  File "/home/realitix/git/vulkan/.eggs/cffi-1.15.0-py3.10-linux-x86_64.egg/cffi/setuptools_ext.py", line 49, in add_cffi_module
    execfile(build_file_name, mod_vars)
  File "/home/realitix/git/vulkan/.eggs/cffi-1.15.0-py3.10-linux-x86_64.egg/cffi/setuptools_ext.py", line 25, in execfile
    exec(code, glob, glob)
  File "vulkan/vulkan_build.py", line 13, in <module>
    ffi.cdef(cdef)
  File "/home/realitix/git/vulkan/.eggs/cffi-1.15.0-py3.10-linux-x86_64.egg/cffi/api.py", line 112, in cdef
    self._cdef(csource, override=override, packed=packed, pack=pack)
  File "/home/realitix/git/vulkan/.eggs/cffi-1.15.0-py3.10-linux-x86_64.egg/cffi/api.py", line 126, in _cdef
    self._parser.parse(csource, override=override, **options)
  File "/home/realitix/git/vulkan/.eggs/cffi-1.15.0-py3.10-linux-x86_64.egg/cffi/cparser.py", line 389, in parse
    self._internal_parse(csource)
  File "/home/realitix/git/vulkan/.eggs/cffi-1.15.0-py3.10-linux-x86_64.egg/cffi/cparser.py", line 394, in _internal_parse
    ast, macros, csource = self._parse(csource)
  File "/home/realitix/git/vulkan/.eggs/cffi-1.15.0-py3.10-linux-x86_64.egg/cffi/cparser.py", line 338, in _parse
    self.convert_pycparser_error(e, csource)
  File "/home/realitix/git/vulkan/.eggs/cffi-1.15.0-py3.10-linux-x86_64.egg/cffi/cparser.py", line 367, in convert_pycparser_error
    raise CDefError(msg)
cffi.CDefError: cannot parse "HMONITOR hmonitor;"
<cdef source string>:11240:5: before: HMONITOR

Can you fix this ? Thanks

julianfl0w commented 2 years ago

@realitix I've naively dumped my entire package set to a Conda environment.yml . Importantly, it enforces Python3.9

This works on Windows, NOT TESTED ON LINUX

Usage: conda env create -f environment.yml pip install .

realitix commented 2 years ago

Sorry but it has to work on Linux to be merged, it shouldn't be too much work.

julianfl0w commented 2 years ago

win32 header has been manually removed from Linux headers
vulkan wrapper raster pipeline is working on Ubuntu, with my library "vulkanese"
image
https://github.com/julianfl0w/vulkanese

Raytrace pipeline is failing at the final step, with error

NameError: name 'vkCreateRayTracingPipelinesKHR' is not defined

This behavior is consistent in Windows and Linux. Perhaps an issue with Extension functions in the new XML

julianfl0w commented 2 years ago

using VENV setup on Linux, Conda environment.yml on Windows

julianfl0w commented 2 years ago

RayTrace bug is killing me. Could you kindly review my build procedure diagram? I believe the issue lies therein https://github.com/julianfl0w/vulkan/blob/master/README.md#architecture

julianfl0w commented 2 years ago

solved it. I simply needed to use vkGetInstanceProcAddr function. please try now

okuma10 commented 2 years ago

@realitix can this be merged ?

julianfl0w commented 2 years ago

@okuma10 have you been able to successfully test my fork on your machine? I am currently dealing with an obscure bug involving vkGetBufferDeviceAddress, a requirement of raytracing

okuma10 commented 2 years ago

@julianfl0w no, sadly I'm a bit of a noob at this and I don't know how to install your fork on my machine. That's why I'm hoping this gets in to pyVulkan's master.

julianfl0w commented 2 years ago

try pip install git+https://github.com/julianfl0w/vulkan

realitix commented 2 years ago

Hello, I tried again and I got this error now:

python example_sdl2.py 
Traceback (most recent call last):
  File "/home/realitix/git/vulkan/example/example_sdl2.py", line 11, in <module>
    from vulkan import *
  File "/home/realitix/git/vulkan/venv/lib/python3.10/site-packages/vulkan-1.1.99.1-py3.10.egg/vulkan/__init__.py", line 1, in <module>
    from vulkan._vulkan import * # noqa
  File "/home/realitix/git/vulkan/venv/lib/python3.10/site-packages/vulkan-1.1.99.1-py3.10.egg/vulkan/_vulkan.py", line 99, in <module>
    if "nt" in os.name.lower() or "win" in os.platform.lower():
AttributeError: module 'os' has no attribute 'platform'

You should directly use the module platform and not os.platform. Can you update your patch ?

julianfl0w commented 2 years ago

i introduced that bug recently. please try now. i've just tested on linux ive started my own DLL/SO + Header wrapper, because vkGetBufferDeviceAddress is segfaulting and it is difficult to debug when accessed as a wrapped function

realitix commented 2 years ago

Hello @julianfl0w, thanks for the update. I tried with your last revision and now I have this error:

Traceback (most recent call last):
  File "/home/realitix/git/vulkan/example/example_sdl2.py", line 11, in <module>
    from vulkan import *
  File "/home/realitix/git/vulkan/.venv/lib/python3.10/site-packages/vulkan-1.1.99.1-py3.10.egg/vulkan/__init__.py", line 1, in <module>
    from vulkan._vulkan import * # noqa
  File "/home/realitix/git/vulkan/.venv/lib/python3.10/site-packages/vulkan-1.1.99.1-py3.10.egg/vulkan/_vulkan.py", line 104, in <module>
    lib = ffi.dlopen(name)
OSError: cannot load library '/home/realitix/git/vulkan/.venv/lib/python3.10/site-packages/vulkan-1.1.99.1-py3.10.egg/vulkan/libvulkan.so': /home/realitix/git/vulkan/.venv/lib/python3.10/site-packages/vulkan-1.1.99.1-py3.10.egg/vulkan/libvulkan.so: cannot open shared object file: No such file or directory
julianfl0w commented 2 years ago

just some git trouble. try now

realitix commented 2 years ago

It works now.

realitix commented 2 years ago

The code is hard to review because you lanched the generator, can you create the patch without launching the generator, in order to get only the difference in the patch. And in another commit, we can launch the generator to split it in 2 commits.

Moreover, can you change theses elements:

Thanks a lot for your work.

julianfl0w commented 2 years ago

code has been simplified and updated to realitix conventions. retested on Windows and Linux. GetBufferDeviceAddress command is still segfaulting, but the raster engine is working well.

I have begun a project to turn (dll/so + c header) into a python library, (IE. without vk.xml) using Python's own ctypes. This schema is at once more maintainable and more generic. However, this project has presented difficulty, and been shelved for now as SODLL https://github.com/julianfl0w/sodll

I am now investigating use of CuPy for my (simple) raytracing ambitions

julianfl0w commented 2 years ago

can you clarify your concerns with the generator? ive not made architectural changes there

okuma10 commented 2 years ago

any progress on this? @realitix

mathrack commented 2 years ago

I think it would be simpler to split the PR :

I have looked at the Vulkan 3 part, and I have obtained something slightly different for the generator.

okuma10 commented 2 years ago

is this issue dead?

okuma10 commented 2 years ago

I'm poking this thread again

realitix commented 10 months ago

It's updated now.