No wheels for Python 3.11

[mrswats]

Bug Report

There are no wheels built for python 3.11

Build Problem

pip install opentimelineio

Build log: opentimelineio_build.log

Incorrect Functionality and General Questions

When installing opentimelineio on python 3.11 it has to build the wheels.

To Reproduce

1. Operating System Fedora release 38 (Thirty Eight)

2. Python version Python 3.11.5 (main, Aug 28 2023, 00:00:00) [GCC 13.2.1 20230728 (Red Hat 13.2.1-1)]

3. Example snippet that demonstrates the issue - if it's a build issue (See above)

4. OpenTimelineIO release version or commit hash Latest available. 0.15.0

5. Compiler information:

   $ gcc -v
   Using built-in specs.
   COLLECT_GCC=gcc
   COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/13/lto-wrapper
   OFFLOAD_TARGET_NAMES=nvptx-none
   OFFLOAD_TARGET_DEFAULT=1
   Target: x86_64-redhat-linux
   Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,objc,obj-c++,ada,go,d,m2,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-libstdcxx-backtrace --with-libstdcxx-zoneinfo=/usr/share/zoneinfo --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl=/builddir/build/BUILD/gcc-13.2.1-20230728/obj-x86_64-redhat-linux/isl-install --enable-offload-targets=nvptx-none --without-cuda-driver --enable-offload-defaulted --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux --with-build-config=bootstrap-lto --enable-link-serialization=1
   Thread model: posix
   Supported LTO compression algorithms: zlib zstd
   gcc version 13.2.1 20230728 (Red Hat 13.2.1-1) (GCC)

Expected Behavior

Not to have to build the wheels at install time.

[mrswats]

I'd be happy to submit a PR if it's needed but checking the GitHub workflows it seems that they should be there already.

Checking on https://pypi.org/project/OpenTimelineIO/#files you can see there are wheels for python3.10 and below, but not for 3.11.

[JeanChristopheMorinPerso]

Hi @mrswats, the next release will have 3.11 wheels.

[mrswats]

Hi @JeanChristopheMorinPerso, thank you very much for the swift reply! Do you have an estimate on when the next release will be?

[JeanChristopheMorinPerso]

It'll probably happen in the next couple of weeks. We already started the process. @reinecke might have a better estimate.

[filmtailor-mainframe]

Is there any update on this?

[werwack]

Hi @JeanChristopheMorinPerso , @reinecke

I'm facing an important issue at the moment: I have been working on a big Blender add-on for several years now, it is used by a great bunch of artists and studios and it relies on openTimelineIO for a part of what it does. The next Blender version, 4.1, is about to be out (matter of days), and it is based on Python 3.11. But at the moment there are still no wheel for OTIO on Python 3.11 😥 This means that all these users that would like to upgrade to Blender 4.1 will not be able to use some important functionalities of the add-on... Which may be an issue for them in production 😟

It would be greatly greatly appreciated if the packages of OTIO on pypi could be updated with the wheels for 3.11. Can you help on this, please?

Blender is following the VFX Reference Platform recommendations when providing Python in their releases: Blender 4.1 Release Notes. Since OpenTimelineIO is also (and mainly?) targeting the VFX industry, it would be more that helpful to provide the wheels supporting those recommendations, don't you think so?

Thanks a loooooooooot for your understanding and for your help here!

BTW: The add-on is named StoryLiner and the documentation page is here: https://werwackfx.com/storyliner/doc/

[aclark4life]

@werwack I can try to help if no else gets to it first! I'll take a look at the OTIO build sometime today (EDT).

[werwack]

@aclark4life, thanks a lot for proposing! That would be appreciated 😊 The thing is that the wheels really have to get into Pypi, with all platforms supported. Indeed the add-on is deployed at quite a large scale and it download the latest version of OpenTimelineIO when it is registered. I cannot patch it. Is this possible?

[aclark4life]

@werwack If you look at this branch I just created you can see that there were no 3.11 wheels for 0.15.0: https://github.com/aclark4life/OpenTimelineIO/blob/v0.15/.github/workflows/python-package.yml and that is most likely due to: https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/4011#discussion_r1355889323

Further I think it's unlikely that the OTIO folks would want to go back and try to build those wheels now, given that they left them out on purpose back then! So you may want to try to coordinate your release with the OTIO folks, otherwise start looking into workarounds.

I was going to suggest you have folks install the pre-release, but I couldn't test that here because it looks like I just built my own wheels for both 0.15.0 and 0.17.0.dev1

(tmpgc26o3xr) [alexclark@alexclarks-Air tmpgc26o3xr ]$ pip install opentimelineio
Collecting opentimelineio
  Using cached OpenTimelineIO-0.15.0.tar.gz (5.1 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting pyaaf2<1.7,>=1.4 (from opentimelineio)
  Using cached pyaaf2-1.6.0-py2.py3-none-any.whl.metadata (2.9 kB)
Using cached pyaaf2-1.6.0-py2.py3-none-any.whl (103 kB)
Building wheels for collected packages: opentimelineio
  Building wheel for opentimelineio (pyproject.toml) ... done
  Created wheel for opentimelineio: filename=OpenTimelineIO-0.15.0-cp311-cp311-macosx_14_0_arm64.whl size=1216254 sha256=a3f09d47aff86ab954e9b31f3ab28334be7a02dd965438616036209948ca7c36
  Stored in directory: /Users/alexclark/Library/Caches/pip/wheels/a2/21/c1/877e3bcf8bc81eaa6d9ac113c026bc8b66dd00e90f6982c855
Successfully built opentimelineio
Installing collected packages: pyaaf2, opentimelineio
Successfully installed opentimelineio-0.15.0 pyaaf2-1.6.0
(tmpgc26o3xr) [alexclark@alexclarks-Air tmpgc26o3xr ]$ pip install opentimelineio==0.17.0.dev1 
Collecting opentimelineio==0.17.0.dev1
  Using cached OpenTimelineIO-0.17.0.dev1.tar.gz (2.8 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: opentimelineio
  Building wheel for opentimelineio (pyproject.toml) ... done
  Created wheel for opentimelineio: filename=OpenTimelineIO-0.17.0.dev1-cp311-cp311-macosx_14_0_arm64.whl size=1100517 sha256=05d78ee051b2c65029fbe97786fdcaa4ab3fea42e393babd32a4f54ba4e9f7c2
  Stored in directory: /Users/alexclark/Library/Caches/pip/wheels/4c/c7/a0/e6d1a57271f1883ba270db438c191451cb269c41b2da96c4b1
Successfully built opentimelineio
Installing collected packages: opentimelineio
  Attempting uninstall: opentimelineio
    Found existing installation: OpenTimelineIO 0.15.0
    Uninstalling OpenTimelineIO-0.15.0:
      Successfully uninstalled OpenTimelineIO-0.15.0
Successfully installed opentimelineio-0.17.0.dev1

Seems like I should have gotten the macos wheel for 10.9+ but maybe it's because I'm on M2 here. 🤷

[werwack]

@aclark4life Thank you for your help. I'll see if someone from OTIO can help on that. I more and more believe that OpenTimelineIO should not be used in production anymore cause its maintenance do not appear to be reliable. That's very sad, for an app from the Academy Software Foundation 😥

[aclark4life]

@werwack I'm new to the community but from what I can tell it's the exact opposite. The challenge of defining and maintaining the VFX Reference Platform is enormous, but exciting that folks came together in 2018 to do just that. For my part, as someone with years of Python experience, but no VFX experience, I'm interested in helping out in a variety of ways including publishing wheels to PyPI. Also, I've been on the other side, and I think it's unfair to make any assumptions about the quality of the OTIO software based entirely on this issue. That said, I understand your frustration.

[werwack]

Unfair??? The thing is this exact scenario already happened 2 years ago: Support for Python 3.10?

When an application - or a library that has to be integrated in the in-house tools - is considered by a studio to be used for the next productions - so for the next 3 to 5 coming years, one of the major points we look at is if it will be supported enough during that time in order to avoid a break in our production flow. As much as possible we try to avoid to change the version of tools during production to ensure the stability of the of the whole pipeline. But sometimes we have to do so to take advantage of a new feature or to get a fix. And we also would like to reuse your production chain for the next production. We then usually prefer going to tools from big manufacturers or libraries that are followed by an organization that is recognized as a reference for our industry, such as the Academy Software Foundation. Hence our choice of adopting OpenTimelineIO in the company I am working for.

We thought that such a big organization would be synchronized with some others that provides guidelines and recommendations for our industry, such as the VFX Reference Platform so that things are clear for everybody, software development companies as well as studios. When a "desynchronization" happens, and that can happens of course, there are consequences on the user side. So, from professional to professionals, and view to the fact that those recommendations are online from months now, I would have expected some more anticipation here, or at least more transparency on the faced issues.

But don't get me wrong. I'm not angry, I'm just blocked. I really appreciate all the work done by all the contributors, all the involvement and the quality of the work. There are probably many things happening on your side we don't see. But my users, my clients and me are in production...

May I also add that the last post from the support side is from 9th of October 2023? Some visibility on the next steps for all the users would have been very welcome, so that we can organize ourselves.

(@aclark4life thank you for your understanding, btw 😉)

[aclark4life]

Unfair??? The thing is this exact scenario already happened 2 years ago: Support for Python 3.10?

Sorry! Perhaps unfair is the wrong word. Because I'm new, I'm just excited to help out and finally have had some time recently to focus on this community.

Outside this community, I'm interested in seeing open source maintainers get paid for their work. I'm not sure if it's something that will happen in my lifetime, but it's something I'm interested in and try to advocate for as much as I'm able to. In fact, my project is in it's fifth year of funding and it's one of the most satisfying things I've worked on in my entire life.

Back to this community, AFAICT the ASWF faces a very common challenge in which they are relying on volunteers from the industry to support the VFX reference platform. All the folks I've encountered here are experts in their industry, but not necessarily experts in open source. The "big organization" is the LF and they provide a lot of $$$ for the infrastructure, but no money to "pay the maintainers". That's unfortunate, but not surprising. This is a systemic issue that's not easy to fix.

Anyway, I'm here for two reasons:

• I'm hooked on the overlap of open source and VFX and want to consume as much knowledge about it as a possibly can as quickly as I can.
• I'm hoping my contributions lead me to something resembling a job in the industry within some sooner-rather-than-later period of time.

Back to your blocker, I hope I can help future releases of Python, Blender, Storyliner, VFX reference platform, more as you describe because I agree that's how it should be. Unlike solving some of the bigger systemic issues, I think this issue can be resolved to everyone's satisfaction.

[reinecke]

Hi @werwack -

Thank you for your passion and patience - I can understand your frustration. As @aclark4life mentioned, none of our project maintainers are full-time on the project, we push the project forward largely using time our employers donate or from our personal lives. This means that the time we have available can vary, and sometimes the specific things we work on are aligned with the things our employers need as sponsors of the project. It is, however, my goal to reserve some more dedicated time to focus on the important "housekeeping" projects like cutting releases so the issues you're dealing with become less prevalent.

Speaking more directly to the issue at hand, we recently gave the green light to the current 0.16.0 feature set, so we're hoping to work on cutting that release this week.

At the moment, making a release requires quite a few manual steps and takes quite a lot of time - for this release we have a meta goal in gathering information to drive toward automation of this process. 0.17.0 is meant to be a "fast follow" release and our hope is to use that to drive our automation forward. We hope this work will improve our release cadence this year - hopefully some of the issues you're feeling will start to be relieved.

Thank you for sharing your perspective, context, and participating in the community.

[werwack]

Hi @reinecke, @aclark4life,

@aclark4life, thank you for your answer.

And thank you both for your involvement! I've been contributing to open-source projects for 5 years now. Now, with StoryLiner and some others that should come later in the year, I'm releasing my own ones. I really like this approach of the development, I think it can be of great benefits for everybody.

I think I get quite well what you describe, in particular that thin edge between company support and personal involvement 😉

If I may, I would like to insist on one point, from my experience with users: I believe that a lot of troubles could be avoided when providing support if there is a clear feedback to them. It's not so much an issue for users if the project is not ready, as long as they know when it will be, and what is the reason of the delay. They can understand, and they can organize themselves. (as long as the announced release is reliable, of course). What happened here is that we, as users, have been in the darkness since October, with no sight of change up to now. That is... disturbing. Annoying when in production.

So thanks a lot @reinecke for your feedback, for the upcoming release, and above all for your comprehension of all this !

[reinecke]

@werwack Totally understandable, thank you for your feedback and suggestion!

[werwack]

Hi @reinecke,

I come to see if it is possible to have some visibility on the release of OpenTimelineIO for Python 3.11 on Pypi. You mentioned last week that it would be on the way.

Thank you

[apetrynet]

Hi @werwack

There's a PR in the works for the next release.

[reinecke]

0.16.0 is now in PyPI with Python 3.11 wheels.

[werwack]

@reinecke Thanks a lot!!! Works like a charm! The installer of StoryLiner was able to find it on Pypi and automatically installed it without trouble. I quickly tested this new version and so far it seems all good. Thanks for all the efforts of the contributors, really appreciated 😉