search

enthought / comtypes

A pure Python, lightweight COM client and server framework, based on the ctypes Python FFI package.
Other
291 stars 96 forks source link

Future of this package #216

Open lassoan opened 4 years ago

lassoan commented 4 years ago

This package is very useful, thanks a lot for providing it! I would like to use it in a large open-source project (3D Slicer) but I'm not sure if I can rely on it in the long term, due to the followings:

  1. It seems that there have been no new tags (and releases on PyPI) for almost two years
  2. It is still not a native Python3, but patched using 2to3
  3. Large backlog of open issues and 10+ open pull requests

Could somebody provide information about current status and future plans for this package?

I see that there have been some recent commits, so I still have some hope. Thanks in advance!

cfarrow commented 4 years ago

Hi Andras. I know of no long term plans for this package, except perhaps for users keeping it running for their use cases. The package can live on for a while in this self-service mode, but nice-to-haves like native python 3 support are not likely be prioritized.

I do virtually no work on this package, as my job and personal life no longer involve coding. @vasily-v-ryabov does the bulk of reviewing and committing. I'd welcome a discussion about building a critical mass of contributors and admins for this project.

lassoan commented 4 years ago

@cfarrow thanks a lot for the quick answer. @vasily-v-ryabov do you have any comments?

vasily-v-ryabov commented 4 years ago

Hi guys, I have some plans on diving deeper into the comtypes source code with potential improvements. My interest is mostly comtypes' usage in pywinauto as a dependency (we're already using it for few years). I'll try to explain my immediate interests and nice-to-have things which I can support as a reviewer.

No more global things for now.

lassoan commented 4 years ago

A new release would be important, because if there are no new releases then fixes and improvements will not get to users. It would be also important to merge pull request and review issues. Setting up AppVeyor would be nice because then pull requests could be more confidently merged.

If there is a consensus that Python2 support can be dropped then updating the syntax for Python3 should not be hard, especially if there are automated tests. I can help with this, as we try to avoid depending libraries that are not updated to Python3.

vasily-v-ryabov commented 4 years ago

pywinauto is not planning to drop Py2.7 support. So I vote against it in comtypes as well. Also I'd prefer to keep comtypes a pure Python library (without any dependencies). We may think about six usage only, but I'm not sure it's so necessary.

lassoan commented 4 years ago

pywinauto is not planning to drop Py2.7 support

My understanding is that projects are actively encouraged to drop Python2 support before the end of 2020 to reduce frictions in the Python ecosystem (https://python3statement.org/). Anyway, if you are ready to pull the plug on Python2, then I'm ready to help with the update.

vasily-v-ryabov commented 4 years ago

I think it would be easy to make better Py2/Py3 support with already working AppVeyor tests. So making better CI is a fundamental task for further progress. Currently it is implemented as python setup.py test command which looks a bit old fashioned. And this command is not running on AppVeyor due to some failures in the full test set. I added some installation tests to AppVeyor a while ago. I'd appreciate if this work was continued.

wkschwartz commented 4 years ago

I just wanted to contribute one user's voice to this discussion.

  1. I vote for the quote below. My use case is packaging apps using PyOxidizer. The apps depend on comtypes. I haven't gotten far enough down the rabbit hole of getting it to work to know whether comtypes needs to change for me to use PyOxidizer. I just wanted to let you know that's a use case I care about.

    Also I'm interested in more convenient packaging into a single executable (low priority for now, may be raised in the future) so I may think about on-the-fly code generation into a memory without saving to cache file or something like that.

  2. I care about CI/automated testing because it would give me confidence in comtypes' reliability, which matters a lot for my apps.
  3. On Python 2/3 in response to the quote below: Since Python 2 has died, perhaps it would be better to use future than six. Putting future; python_version <= "2.7" in setup.py's install_requires would burden the few remaining Python 2 users rather than the mass of Python 3 users.

    We may think about six usage only, but I'm not sure it's so necessary.

nanonyme commented 4 years ago

Even if there's no larger changes to the project done, it would be really great if https://github.com/enthought/comtypes/pull/172 (or something equivalent) could be integrated and a new release could be done. Currently this project is randomly failing on Windows with new Python 3 versions which is causing issues with pywinauto. (which depends on this project for UIAutomation support) Afaik many other UI automation tools also use Windows UIAutomation through comtypes so they have the same problem.

snoopyjc commented 4 years ago

Please let Python 2 die!

nanonyme commented 4 years ago

IMHO debating killing Python 2 is futile if the person who is more or less maintaining this project intents on keeping the support for now. This should not and must not mean halting improving the quality of this project on Python 3 though.

vasily-v-ryabov commented 3 years ago

comtypes==1.1.8 is out.

junkmd commented 2 years ago

A long time has passed since this issue was posted.

And now,

As I planned in #327, I am trying to tie the comtypes type definition system into Python's type hinting system.

I think it is important to once again discuss plans in the future, and this issue is a good place to discuss it.

What are participants thoughts on this now?

vasily-v-ryabov commented 2 years ago

Removing Py3.4 is OK to me. Even if someone has legacy Python 3.4 environment, there is a very low risk something is broken in comparison with 3.5 or even 3.6. But keeping Py2.7 covers more legacy users in my opinion. I would remove Py2.7 after 1 more year of support which shouldn't bother all us so much. For example, we can release comtypes 1.1.13 and 1.1.14 with Py2.7 support and then release comtypes 1.2.0 with Py3.6+ support only. So type hints are natively supported from some moment without any additional efforts.

junkmd commented 2 years ago

@vasily-v-ryabov

I agree with your suggestion to support only Py 3.6+ when dropping Py 2.7.

With Py3.6+, we can use the built-in typing. And enum module is also supported, which may be able to provide new features related to enumerations.

I understood that it will be about a year later we release the version of Py2.7 dropped.

I will develop this package according to that plan.

junkmd commented 1 year ago

Hello members, collaborators and contributors(including @cfarrow, @jaraco, @vasily-v-ryabov and @cmin764)!

I would like to put more efforts into comtypes, including the introduction of type annotations and supporting enumerations in the future.

I would like to be a collaborator on this project.

Any opinions would be appreciated.

vasily-v-ryabov commented 1 year ago

@cfarrow @jaraco I think merge/maintainer permissions make sense for @junkmd. If there are no objections, I will add him as collaborator in next 2 days.

cfarrow commented 1 year ago

I support this.

On Mon, Nov 21, 2022, 10:17 AM Vasily Ryabov @.***> wrote:

@cfarrow https://github.com/cfarrow @jaraco https://github.com/jaraco I think merge/maintainer permissions make sense for @junkmd https://github.com/junkmd. If there are no objections, I will add him as collaborator in next 2 days.

— Reply to this email directly, view it on GitHub https://github.com/enthought/comtypes/issues/216#issuecomment-1322469060, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGNU7TK7DDC3NB3CNAMRPLWJO4EJANCNFSM4PF6YPBQ . You are receiving this because you were mentioned.Message ID: @.***>

vasily-v-ryabov commented 1 year ago

@junkmd invitation is sent. Welcome to the club! :)

junkmd commented 1 year ago

@vasily-v-ryabov @cfarrow

Thank you very much!:smile:

junkmd commented 1 year ago

I added the good first issue tag as my first work as a collaborator.

I hope to see more newcomers in this community.

junkmd commented 1 year ago

I triaged some issues in my free time this weekend. I closed issues that are currently resolved and issues that have not seen movement for several years.

But it is very voluminous, so It is taking time to do "issue inventory".

I plan to make some progress on this little by little.

junkmd commented 1 year ago

Today I had free time because I was on vacation for the COVID-19 vaccination.

I sorted out the remaining issues, currently under 100.

I think it is important for the community to make sure that newcomers are not put off by the number of issues.

junkmd commented 1 year ago

@cfarrow @vasily-v-ryabov

We have included issues on 32-64bit and issues on VARIANT with typecode VT_ARRAY | BT_BSTR in the scope of milestones for the next release. https://github.com/enthought/comtypes/milestone/3

However, these are no activities for resolve them recently.

It seems to me that it is better to release a new version that includes the changes of npsupport(it made no longer backward compatible), adding type-annotations, and some refactoring, and excludes #80, #347, and #193.

I think it is necessary to release it once to also notify users of the drop_py2 plan.

If you can give me the admin rights to PyPI, I will release this after the new year vacations. https://pypi.org/user/junkmd/

About #80, #347, and #193, I think they depend on the contributors who want to resolve these.

Of course, I intend to do that there will be no conflicts when we merge drop_py2 to master in the future.

vasily-v-ryabov commented 1 year ago

OK, I moved all 3 issues and 1 more to 1.3.0 milestone for now. Maybe they will move to 1.3.1 or even later milestones depending on the available resources. I think it's time to check the change log and maybe other related things. I can release 1.2.0 to PyPI in January, no problem. Maybe I'll even have time to release it in December, though it adds more risk to break something during holidays. :)

junkmd commented 1 year ago

Understood. Thank you for your effort and planning. I hope you enjoy the holidays without any risks.

junkmd commented 1 year ago

@vasily-v-ryabov

When do you plan to release 1.2.0?

vasily-v-ryabov commented 1 year ago

@junkmd sorry for disappearing for a long time. I can make a release at the first half of May.

junkmd commented 1 year ago

@vasily-v-ryabov Thank you for the reply. I see.

Is there anything I can do before or after this release?

vasily-v-ryabov commented 1 year ago

Sorry, I just get back to open source activities. If you have time to quickly update CHANGES.txt, that would be nice. If not, I will try to find time tomorrow.

junkmd commented 1 year ago

@vasily-v-ryabov

Welcome back!

I will update CHANGES.txt and PR it.

Please wait a moment as I will be summarizing the changes.

edited: Please see #481

vasily-v-ryabov commented 1 year ago

1.2.0 has been pushed to PyPI. The only problem is that Project description on PyPI looks not nice: https://pypi.org/project/comtypes/1.2.0/ in comparison with previous one: https://pypi.org/project/comtypes/1.1.14/ I guess PyPI still doesn't support Markdown very well. It's better to rely on RST format in the future.

vasily-v-ryabov commented 1 year ago

Also I renamed master to main.

junkmd commented 1 year ago

@vasily-v-ryabov

Thanks for all your work.

The only problem is that Project description on PyPI looks not nice:

Oops... I wondered about this phenomenon because I remembered that the black description was rendered correctly in PyPI even though the content of long_description is its README.md.

I read the PyPA documentation and realized we forgot the long_description_content_type="text/markdown"!

Set the long_description_content_type to an accepted Content-Type-style value for your README file’s markup, such as text/plain, text/x-rst (for reStructuredText), or text/markdown. https://packaging.python.org/en/latest/guides/making-a-pypi-friendly-readme/#including-your-readme-in-your-package-s-metadata

vasily-v-ryabov commented 1 year ago

@junkmd thanks for finding this! I think it could be fixed in 1.2.1.

junkmd commented 1 year ago

It has been a month since comtypes==1.2.0 was released. I am relieved that no reports of regression bugs have been reported.

I noticed that Python 3.7 is now EOL. https://devguide.python.org/versions/#unsupported-versions

Even so, I still think it would be better to keep the minimum Python version supported by 1.3.0 at 3.7, because the community might get confused if that is changed.

Since distutils will be no longer in Python 3.12(PEP632), it is necessary to change setup.py or other actions in order to use comtypes in the new Python.

It might be time to consider when comtypes==1.3.0 should be released.

Any opinions would be appreciated.

junkmd commented 8 months ago

Can we release 033d6e8 (and adding changelogs) as 1.2.1?

vasily-v-ryabov commented 8 months ago

1.2.1 has been released.

junkmd commented 8 months ago

I am planning to observe movements after the release of 1.2.1 over the next few weeks (for regression and bug reports).

1.3.0 will be released after that observation.

In light of this, I have reviewed the current milestones for 1.3.0.

The following items, which have been present since before the drop_py2 plan was initiated, have shown little recent activity and seems unlikely to be resolved before the release. Therefore, they has been removed from the 1.3.0 milestone.

The following items, originally included in the drop_py2 plan, were also removed from the 1.3.0 milestone. This decision was made due to the significant impact of the changes required for inclusion in 1.3.0 and the necessity to announce these changes to comtypes users prior to implementation.

manju1847 commented 8 months ago

We are currently using PyWinAuto with Python 3.8. Here is our installed package list

C:\Users\Administrator>pip freeze comtypes==1.1.11 Pillow==9.3.0 pywin32==303 pywinauto==0.6.8 six==1.16.0

Now we are planning to move to latest Python 3.12. Looking at this thread I think we cannot use PyWinAuto with Python 3.12. What is the maximum Python version that we can move to provided we still use PyWinAuto

junkmd commented 8 months ago

1.2.1 supports Python3.12.

https://github.com/enthought/comtypes/releases/tag/1.2.1

If you have a problem with the pywinauto installation itself, please submit an issue at pywinauto.

manju1847 commented 8 months ago

Even with comtypes 1.2.1, PyWinAuto installation fails with Python 3.12.

Submitted the issue in PyWinAuto https://github.com/pywinauto/pywinauto/issues/1368

junkmd commented 7 months ago

I released 1.3.0 this Monday, I am about to embrace first weekend.

I am relieved that there have been no regression reports immediately following the release.

A few weeks later, I am planning to release 1.3.1, which #490 was merged.

If there are pull requests from contributors addressing some of the remaining "good first issue", I would like to prioritize reviewing them.

junkmd commented 4 months ago

I created a 'typing' label for the classification of features related to static typing.

junkmd commented 3 months ago

While summarizing my thoughts on numpy for #551, I noticed a few things during my research.

As mentioned in #220, the version of numpy where numpy.ctypeslib._typecodes was removed is 1.16. numpy==1.16.6 has been EOL for over 5 years according to https://endoflife.date/numpy. However, according to oldest-supported-numpy, the oldest numpy that runs on Python3.7 (the oldest Python this package currently supports) is numpy==1.14.5. Therefore, we may need to continue maintenance considering that users using comtypes with Python3.7 may also be using numpy==1.14.x or numpy==1.15.x.

On the other hand, numpy supporting Python3.8 is 1.17 and above. This means that when this project drops support for Python3.7 (the specific timing is still undecided), there will be no need for production code to attempt to import numpy.ctypeslib._typecodes.

Furthermore, there is also room to consider dropping to support numpy<1.16 while continuing to support Python3.7.

jaraco commented 3 months ago

I noticed that Python 3.7 is now EOL. https://devguide.python.org/versions/#unsupported-versions

Even so, I still think it would be better to keep the minimum Python version supported by 1.3.0 at 3.7, because the community might get confused if that is changed.

I've found that dropping support for EOL Pythons is generally uncontroversial, especially if the proper metadata is present. I'd advise to do that as a matter of course with this (or any other) project unless there are specific functional reasons to keep it.

junkmd commented 3 months ago

I've found that dropping support for EOL Pythons is generally uncontroversial, especially if the proper metadata is present. I'd advise to do that as a matter of course with this (or any other) project unless there are specific functional reasons to keep it.

I certainly agree with what you’re saying. Providing a long grace period when we dropped support for Python 2.7 was a special case.

When Python 2 support was dropped, the version increment was made from 1.2.1 to 1.3.0. When Python 2.6 was dropped in the past, the increment was made from 1.1.10 to 1.1.11.

In this project, when releasing the version that dropping support for Python 3.7 (or any Python 3.Y.Z that reaches EOL), which digit of comtypes==x.y.z should be incremented?

jaraco commented 3 months ago

Even if this project honors the well-specified semver standard, the choice for what qualifies as a "bugfix" (patch bump) and not (minor bump) is far from deterministic. Here's the methodology I follow:

  1. By default, any change is a minor bump.
  2. If all changes are corrections to previously-released behaviors (bugfix) or are not affecting user behavior (docs changes, metadata updates, etc.), then a patch bump may be used (at the release manager's discretion). A minor bump may still be elected if the changes were substantial or there's reason to make room for bugfixes on the previous minor release.
  3. If any change to behavior is likely to break a user's expectations, then it's a breaking change and should be a major bump.

By that methodology, since the ecosystem prevents installation of incompatible versions, it's not a breaking change. Since it's only a metadata change not affecting behavior, it could be a patch bump. On the other hand, because it's not a correction and because one may want to release bugfixes for versions supporting older Pythons, I almost always use a minor bump when dropping support for a Python.

It's this last point that I think is the most crucial. Using the 1.1.11 release as an example, if a bug was discovered that affected both 1.1.10 and 1.1.11 and we wished to fix for Python 2.6 users, it would not be possible without a post-release tag (e.g. 1.1.10.post1) and I'm not even sure those are allowed on PyPI. That's why I would prefer to use a minor bump for such changes.

junkmd commented 3 months ago
  • By default, any change is a minor bump.
  • If all changes are corrections to previously-released behaviors (bugfix) or are not affecting user behavior (docs changes, metadata updates, etc.), then a patch bump may be used (at the release manager's discretion). A minor bump may still be elected if the changes were substantial or there's reason to make room for bugfixes on the previous minor release.
  • If any change to behavior is likely to break a user's expectations, then it's a breaking change and should be a major bump.

since the ecosystem prevents installation of incompatible versions, it's not a breaking change.

Thank you for sharing your methodology.

With reference to that, I have the following very rough release schedule in mind:

junkmd commented 3 months ago

Since it's only a metadata change not affecting behavior, it could be a patch bump.

Is "metadata" the classifiers in setup.py for this package?

When we dropped support for Python2, 'Programming Language :: Python :: 2.7' was removed from classifiers, and the distribution format was no longer a universal wheel, so I think this qualifies as a "change in metadata". However, the current classifiers include 'Programming Language :: Python :: 3' but not 'Programming Language :: Python :: 3.Y'. So, I realized that it might be not possible to determine the supported Python3.Y version from the PyPI metadata for this package. Is that correct?

P.S. I am also aware that it is not recommended to have a large amount of code written in setup.py as it is now. I confess that I do not know much how to properly replace (or abolish) things like class test(Command) and class post_install(install) when adopting more modern ways such as pyproject.toml or setup.cfg.