Compatibility with newer build tools

IanTrudel commented 2 years ago

[ ] Investigating build tools. (autoconf, make/cmake, SCons, pdm, etc.)
[x] Updating Python. (esoteric branch)
[x] Adding -fcommon to CFLAGS. (esoteric branch)
[ ] Updating setup.py to pyproject.toml. (partially done for Cython files, esoteric branch)
[ ] Preparing for Python 3.13. (no GIL, real threads)

Supporting a newer version of GCC (10+) requires declaring external global variables using extern. A temporary fix is to use the flag -fcommon. See more at https://gcc.gnu.org/gcc-10/porting_to.html.

The Makefile uses Python 3.8, whereas the latest version of Python is 3.10.5. It should probably just be python3 instead.

Links

pdm
pyproject
setup.py deprecated
PEP703 (Making the Global Interpreter Lock Optional in CPython)
SCons

redhog commented 2 years ago

Hm, this is the fault of my machine I think, it was defaulting to py3.6 even though I had 3.8 or something like that... Should fix.

IanTrudel commented 11 months ago

This is still an ongoing issue for both -fcommon and Python. To be noted that setuptools is deprecated, see https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html#summary.

pysmlib is failing to build. Pip recommends using --use-pep517, but it doesn't work either.

Successfully installed glass-lib-0.1
. build/env/bin/activate; cd pysmlib; pip install --use-pep517 -e .
Obtaining file:///home/ian/Workspace/InfiniteGlass/pysmlib
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... error
  error: subprocess-exited-with-error

  × Getting requirements to build editable did not run successfully.
  │ exit code: 1
  ╰─> [63 lines of output]
      <string>:4: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
      !!

              ********************************************************************************
              Requirements should be satisfied by a PEP 517 installer.
              If you are using pip, you can try `pip install --use-pep517`.
              ********************************************************************************
      !!
      /home/ian/Workspace/InfiniteGlass/build/env/bin/python: No module named pip
      Traceback (most recent call last):
        File "/tmp/pip-build-env-1gd2cvr4/overlay/lib/python3.11/site-packages/setuptools/installer.py", line 96, in _fetch_build_egg_no_warn
          subprocess.check_call(cmd)
        File "/usr/lib/python3.11/subprocess.py", line 413, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['/home/ian/Workspace/InfiniteGlass/build/env/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpksyz4qy7', '--quiet', 'Cython>=0.15.1']' returned non-zero exit status 1.

      The above exception was the direct cause of the following exception:

      Traceback (most recent call last):
        File "/home/ian/Workspace/InfiniteGlass/build/env/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/ian/Workspace/InfiniteGlass/build/env/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/ian/Workspace/InfiniteGlass/build/env/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 132, in get_requires_for_build_editable
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1gd2cvr4/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 450, in get_requires_for_build_editable
          return self.get_requires_for_build_wheel(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1gd2cvr4/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1gd2cvr4/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-1gd2cvr4/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 488, in run_setup
          self).run_setup(setup_script=setup_script)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1gd2cvr4/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 338, in run_setup
          exec(code, locals())
        File "<string>", line 4, in <module>
        File "/tmp/pip-build-env-1gd2cvr4/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 907, in fetch_build_eggs
          return _fetch_build_eggs(self, requires)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1gd2cvr4/overlay/lib/python3.11/site-packages/setuptools/installer.py", line 38, in _fetch_build_eggs
          resolved_dists = pkg_resources.working_set.resolve(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1gd2cvr4/overlay/lib/python3.11/site-packages/pkg_resources/__init__.py", line 829, in resolve
          dist = self._resolve_dist(
                 ^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1gd2cvr4/overlay/lib/python3.11/site-packages/pkg_resources/__init__.py", line 865, in _resolve_dist
          dist = best[req.key] = env.best_match(
                                 ^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1gd2cvr4/overlay/lib/python3.11/site-packages/pkg_resources/__init__.py", line 1135, in best_match
          return self.obtain(req, installer)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1gd2cvr4/overlay/lib/python3.11/site-packages/pkg_resources/__init__.py", line 1147, in obtain
          return installer(requirement)
                 ^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1gd2cvr4/overlay/lib/python3.11/site-packages/setuptools/installer.py", line 98, in _fetch_build_egg_no_warn
          raise DistutilsError(str(e)) from e
      distutils.errors.DistutilsError: Command '['/home/ian/Workspace/InfiniteGlass/build/env/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpksyz4qy7', '--quiet', 'Cython>=0.15.1']' returned non-zero exit status 1.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build editable did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
make: *** [Makefile:48: build/env/bin/pysmlib] Error 1

IanTrudel commented 9 months ago

Some clarifications are necessary in order to provide a seamless migration to the newest approach in Python.

First and foremost, setuptools is still in use but focuses on project.toml instead of the legacy setup.py.

The newer approach is easier to package, integrate and generally comfortably configurable. It can build packages using python -m build or just let you install the current directory as would the setup.py approach.

Concretely, this won't change much in our world besides rewriting setup.py into pyproject.toml The current build system in InfiniteGlass should probably work as in.

https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html

However, I'm growing interested in pdm, which might give InfiniteGlass a greater control over package management. This would require small changes in the main Makefile.

https://github.com/pdm-project/pdm

Second, Python 3.13 with PEP 703 is coming soon, a huge boost for a window manager like InfiniteGlass. Say, Au revoir GIL!

https://peps.python.org/pep-0703/

@redhog What's your take on these?

Once again, I would like to remind you that the greatest impediment to run InfiniteGlass on multiple distributions are the aging tools. I managed to make it work on a few distros and even macOS over the years but still can't get it to work as my daily driver.

IanTrudel commented 9 months ago

@redhog Please, take a look at the branch esoteric, where I attempted to fix some of the ongoing issues in regards to using newer build tools. It is noteworthy that Cython now enjoys a whole lot of typecast. Unfortunately, my efforts only led me to open Xephyr with a lot of memory corruption.

Installing collected packages: pysmlib
Successfully installed pysmlib-0.1
GLASS_DEBUGGER="" BUILD="build" XSERVERPATH="/usr/bin/Xephyr" XSERVEROPTS=":100 -ac -screen 1280x768x24 -host-cursor -extension MIT-SHM -nolisten tcp" scripts/xstartup.sh

xinit: XFree86_VT property unexpectedly has 0 items instead of 1
Copying 1
inet6/maerk:43225,inet/maerk:41503,local/maerk:@/tmp/.ICE-unix/1542392,unix/maerk:/tmp/.ICE-unix/1542392
_IceTransSocketINETConnect() no usable address for maerk:43225
NEW CONN <glass_ghosts.session.Server object at 0x7f0dcb3b19d0>
NEW CONN DONE <glass_ghosts.session.Server object at 0x7f0dcb3b19d0> 139696861133824 {139696861133824: <glass_ghosts.session.Server.Connection object at 0x7f0db5c8bc00>} 94465926165760 139696861133824
malloc(): memory corruption (fast)
Found XInput version 2.0
corrupted size vs. prev_size
malloc(): memory corruption (fast)
malloc(): memory corruption (fast)
malloc(): memory corruption (fast)
malloc(): memory corruption (fast)
malloc(): memory corruption (fast)
malloc(): memory corruption (fast)
malloc(): memory corruption (fast)
_IceTransSocketINETConnect() no usable address for maerk:43225
NEW CONN <glass_ghosts.session.Server object at 0x7f0dcb3b19d0>
NEW CONN DONE <glass_ghosts.session.Server object at 0x7f0dcb3b19d0> 139696861287984 {139696861133824: <glass_ghosts.session.Server.Connection object at 0x7f0db5c8bc00>, 139696861287984: <glass_ghosts.session.Server.Connection object at 0x7f0db5cb1630>} 94465926332432 139696861287984
malloc(): memory corruption (fast)
malloc(): memory corruption (fast)
malloc(): memory corruption (fast)
malloc(): memory corruption (fast)
malloc(): memory corruption (fast)
malloc(): memory corruption (fast)
malloc(): memory corruption (fast)
^Cxinit: connection to X server lost

waiting for X server to shut down .xterm: fatal IO error 11 (Ressource temporairement non disponible) or KillClient on X server ":100"
xterm: fatal IO error 11 (Ressource temporairement non disponible) or KillClient on X server ":100"

IanTrudel commented 9 months ago

Additional notes:

You told me in the past that such situations might be caused by not having some components running.

I tried to run the glass-theme within the venv after I started Xephyr and then the glass-renderer to no avail. I set no_splash mode but the file format had changed in the meantime, so I also did it from the code to make sure it would know.

None of these solve the issues. Blank as my eyes staring into the emptiness.

That might bring us back to pysmlib. The casts may be necessary but there may still require some addressing or inference. I'm not familiar enough with Cython for this. That would be something worthy of memory corruption, wouldn't it? :)

IanTrudel commented 9 months ago

I managed to run the test suite for the better or worse.

nose2 -s tests -v
test_list_shaders (tests.test_renderer.RendererTest.test_list_shaders) ... ERROR
test_multiterminal_view_animation (tests.test_renderer.RendererTest.test_multiterminal_view_animation) ... ok
test_multiwindow_view_animation (tests.test_renderer.RendererTest.test_multiwindow_view_animation) ... ok
test_subwindow (tests.test_renderer.RendererTest.test_subwindow) ... ERROR
test_terminal1 (tests.test_renderer.RendererTest.test_terminal1) ... ok
test_terminal_coords_animation (tests.test_renderer.RendererTest.test_terminal_coords_animation) ... X protocol error:
<class 'Xlib.error.BadWindow'>: code = 3, resource_id = <Resource 0x0120000c>, sequence_number = 671, major_opcode = 18, minor_opcode = 0
ok
test_terminal_view_animation (tests.test_renderer.RendererTest.test_terminal_view_animation) ... ok

======================================================================
ERROR: test_list_shaders (tests.test_renderer.RendererTest.test_list_shaders)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ian/Workspace/InfiniteGlass/tests/test_renderer.py", line 70, in tearDown
    self.display.mainloop.do(False)
  File "/home/ian/Workspace/InfiniteGlass/glass-lib/InfiniteGlass/mainloop.py", line 64, in do
    self.timeouts.pop(timeout)(timestamp)
  File "/home/ian/Workspace/InfiniteGlass/tests/test_renderer.py", line 214, in done
    assert "shaders" in output
           ^^^^^^^^^^^^^^^^^^^
TypeError: argument of type 'NoneType' is not iterable

======================================================================
ERROR: test_subwindow (tests.test_renderer.RendererTest.test_subwindow)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ian/Workspace/InfiniteGlass/tests/test_renderer.py", line 70, in tearDown
    self.display.mainloop.do(False)
  File "/home/ian/Workspace/InfiniteGlass/glass-lib/InfiniteGlass/mainloop.py", line 64, in do
    self.timeouts.pop(timeout)(timestamp)
  File "/home/ian/Workspace/InfiniteGlass/tests/test_renderer.py", line 201, in done
    self.assertEqual(output["rendering-views"]["IG_VIEW_DESKTOP"]["Main"]["coords"],
                     ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
KeyError: 'IG_VIEW_DESKTOP'

----------------------------------------------------------------------
Ran 7 tests in 143.536s

FAILED (errors=2)

IanTrudel commented 9 months ago

You must be using an ancient and unupdated distro, @redhog. I installed the latest Ubuntu on a VM, dependencies.txt, and tried the main branch without success. The esoteric branch gives me similar results as on Arch Linux. This project is due for an upgrade to survive.

IanTrudel commented 9 months ago

The esoteric branch has some success in combination with make run-in-docker.

IanTrudel commented 9 months ago

Further tests are getting interesting...

Installed a fresh Arch Linux in a VM, esoteric branch is running (startx scripts/xinitrc).
Found out that lsp-plugins is required, it has to be added to dependencies.arch.txt. Probably due to liblsp-r3d-glx-lib.
Built glass-renderer and glass-annotator with clang; InfiniteGlass works and it runs considerably faster within Xephyr.
glass-renderer/item.h is missing the following declaration (at least to work with clang): extern void item_menu_update_space_pos_from_window(Item *item, int x, int y, int width, int height);
Still getting memory corruption (see below).

Starting renderer without debugger
NEW CONN <glass_ghosts.session.Server object at 0x7f95a5adb650>
NEW CONN DONE <glass_ghosts.session.Server object at 0x7f95a5adb650> 140280706376528 {140280706376528: <glass_ghosts.session.Server.Connection object at 0x7f95a5ac2350>} 94525958923520 140280706376528
malloc(): memory corruption (fast)
Found XInput version 2.0
Starting renderer without debugger
corrupted size vs. prev_size
Starting renderer without debugger
corrupted size vs. prev_size
Starting renderer without debugger
corrupted size vs. prev_size
Starting renderer without debugger
corrupted size vs. prev_size
Starting renderer without debugger

(process:3595777): librsvg-WARNING **: 14:46:51.334: cannot render on a cairo_t with a failure status (status=InvalidMatrix)

IanTrudel commented 9 months ago

You mentioned SCons in issue #87.

I'm warming up to the idea of using SCons. It seems that VirtualEnv, Cython and pdm are integrating well with it. No problem compiling C code either. Incremental builds should be possible. A proof-of-concept would be interesting.

Autoconf should still be used to mitigate dependencies hell as I find it easier to figure out missing dependencies when running ./configure.

redhog / InfiniteGlass

Compatibility with newer build tools #88