search

jquast / wcwidth

Python library that measures the width of unicode strings rendered to a terminal
Other
393 stars 58 forks source link

Dropping python 2 and 3.5 and earlier? #102

Open jquast opened 9 months ago

jquast commented 9 months ago

Thoughts,

image

I am a strong believer in backporting and so on, having worked in many restrictive environments, I can sympathize with the small number of the tens of thousands of wcwidth downloads performed every day by python 2.7. And, though they should be version-pinning when working with such legacy software versions, there are cases where they are not.

The reason I held onto python 2 support so long was because it wasn't too difficult, we had found simple 2/3 switches that work in the codebase and the tests have provided nearly 100% coverage.

However I think a majority of the bugs of wcswidth calculations have been resolved, and python 2.7 users can now benefit from this. And that I cannot expect any python 2.7 users will want to make use of any new API's that are discussed in open issues. And for that reason, I am a proponent of dropping support of older python releases.

vfazio commented 9 months ago

For informational purposes: https://devguide.python.org/versions/

GalaxySnail commented 9 months ago

The latest version of setuptools which supports python 2.7 is setuptools 44.1.1 [1], the latest version of pip which supports python 2.7 and 3.5 is pip 20.3.4 [2].

[1] https://setuptools.pypa.io/en/latest/history.html#v45-0-0 [2] https://pip.pypa.io/en/stable/news/#v21-0

If we decide to drop python 2, we can maintain a legacy branch 0.2.x for python 2, and backport bug fixes only (including unicode table updates). The question is: what can we get from dropping python 2 and 3.5 (or even 3.6, 3.7)? Typing on python 2 is difficult, but I don't think #71 is an unsolvable problem, maybe it just need some more investigation.

If we drop python 2 only (requires python>=3.5):

[3] https://mypy-lang.blogspot.com/2023/10/mypy-16-released.html [4] https://github.com/python/mypy/commit/14743a1cdd2a07ecc56ce01cc9d54130fb32931e

If we drop python<=3.5 (requires python>=3.6):

If we drop python<=3.6 (requires python>=3.7):

[5] https://tox.wiki/en/4.11.3/faq.html#testing-end-of-life-python-versions

IMO currently it seems not worth to maintain 2 different branches. We can reconsider this if #71 is proven unsolvable unless dropping python 2, or we encounter more problems in the future.

jquast commented 9 months ago

I agree with everything you say. I don't have much inclination to do the work to remove python 2 support myself, I would only be inclined to remove it when I am unable to reasonably test it or continue to support it.

On 'What do we get?',

there is little to gain, only to drop the maintenance burden for tasks like #71. It is minor burden for the changes I expect to make, even in the ~30 hours of work I recently dedicated, only ~1 hours work was required to ensure python 2 compatibility.

We have previously discussed the idea of Cython or C-compiled assistance for performance (I will open an issue shortly), and, continuing to support python 2 and 3.5, and, the operating systems and build tools that would accompany it, might be too burdensome to maintain at that time, but I would only worry about it then.

And for typing on the whole, #71, I am in no hurry to drop support for ~1 million downloads/month of "users", for the ~10 developer who might benefit from types, and even then I believe there is little to gain there as this library API is basic, obvious and the types are in the docstrings, it's hardly worth the attention!

On 'it seems not worth to maintain 2 different branches'

I propose to drop python 2/mark a release as "final python 2 release". With #91 and #97 now released for python 2.7, and, 'ucs-detect' tool and results mostly complete, and over time with open number of issues low, I can say with high confidence that wcwidth seems to be >99% accurate for all versions of unicode up to 15.0

And, as for Unicode updates, version 15 is able to accurately write a great majority of the world's languages. I expect only very minority and ancient languages, symbols/emojis, and rare punctuation additions in future releases.

And, that in the case of supporting legacy software that is stuck on python 2.7, I see little need to continue to support python 2.7 for Unicode versions 16+. They would not likely also be using the latest 2023 releases of terminal emulators, and also be in need of viewing characters exclusive to future releases like Unicode 16.

vfazio commented 9 months ago

The pypi stats don't give great insights on what python versions are pulling which versions AFAICT, so I ran a quick query on the public pypi dataset for the past 30 days

SELECT REGEXP_EXTRACT(details.python, r"[0-9]+\.[0-9]+") AS python_version, file.version, COUNT(*) AS num_downloads FROM `bigquery-public-data.pypi.file_downloads` WHERE file.project = 'wcwidth' AND 
  DATE(timestamp) BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY) AND CURRENT_DATE()
GROUP BY `python_version`, file.version
ORDER BY `python_version`, file.version
python_version version num_downloads
  0.0.1 24
  0.1.0 24
  0.1.1 22
  0.1.2 23
  0.1.3 21
  0.1.4 71
  0.1.5 80
  0.1.6 78
  0.1.7 34878
  0.1.8 18099
  0.1.9 12046
  0.2.0 153
  0.2.1 23
  0.2.10 563454
  0.2.11 75314
  0.2.12 80469
  0.2.2 99
  0.2.3 176
  0.2.4 2942
  0.2.5 608248
  0.2.6 2400764
  0.2.7 16711
  0.2.8 1433167
  0.2.9 1664501
2.7 0.1.4 6
2.7 0.1.5 212
2.7 0.1.6 202
2.7 0.1.7 46574
2.7 0.1.8 6904
2.7 0.1.9 19847
2.7 0.2.10 170671
2.7 0.2.11 29474
2.7 0.2.12 40155
2.7 0.2.2 53
2.7 0.2.3 71
2.7 0.2.4 426
2.7 0.2.5 77507
2.7 0.2.6 42646
2.7 0.2.7 133
2.7 0.2.8 199542
2.7 0.2.9 327602
3.1 0.1.1 2
3.1 0.1.3 36
3.1 0.1.4 1
3.1 0.1.5 4
3.1 0.1.6 2314
3.1 0.1.7 25206
3.1 0.1.8 10131
3.1 0.1.9 48818
3.1 0.2.0 138
3.1 0.2.10 2047626
3.1 0.2.11 252603
3.1 0.2.12 334320
3.1 0.2.2 132
3.1 0.2.3 203
3.1 0.2.4 823
3.1 0.2.5 330516
3.1 0.2.6 1083648
3.1 0.2.7 5479
3.1 0.2.8 2525359
3.1 0.2.9 4161309
3.11 0.1.4 5
3.11 0.1.5 1
3.11 0.1.6 3
3.11 0.1.7 20459
3.11 0.1.8 1376
3.11 0.1.9 25514
3.11 0.2.0 96
3.11 0.2.10 1112439
3.11 0.2.11 151078
3.11 0.2.12 178585
3.11 0.2.2 110
3.11 0.2.3 524
3.11 0.2.4 216
3.11 0.2.5 112013
3.11 0.2.6 418648
3.11 0.2.7 6204
3.11 0.2.8 1394119
3.11 0.2.9 2199217
3.12 0.0.1 1
3.12 0.1.0 1
3.12 0.1.1 1
3.12 0.1.2 1
3.12 0.1.3 1
3.12 0.1.4 6
3.12 0.1.5 4
3.12 0.1.6 3
3.12 0.1.7 291
3.12 0.1.8 106
3.12 0.1.9 1472
3.12 0.2.0 222
3.12 0.2.10 221023
3.12 0.2.11 32029
3.12 0.2.12 36664
3.12 0.2.2 3
3.12 0.2.3 4
3.12 0.2.4 9
3.12 0.2.5 6580
3.12 0.2.6 5668
3.12 0.2.7 46
3.12 0.2.8 204408
3.12 0.2.9 401904
3.13 0.1.8 2
3.13 0.1.9 7
3.13 0.2.10 199
3.13 0.2.11 18
3.13 0.2.12 44
3.13 0.2.5 24
3.13 0.2.6 18
3.13 0.2.8 385
3.13 0.2.9 728
3.3 0.1.7 2
3.3 0.2.10 7
3.3 0.2.11 4
3.3 0.2.12 4
3.3 0.2.5 2
3.3 0.2.6 2
3.3 0.2.8 3
3.3 0.2.9 9
3.4 0.1.6 1
3.4 0.1.7 281
3.4 0.2.10 388
3.4 0.2.11 48
3.4 0.2.12 60
3.4 0.2.5 3
3.4 0.2.6 3
3.4 0.2.8 481
3.4 0.2.9 862
3.5 0.1.5 12
3.5 0.1.6 1
3.5 0.1.7 3258
3.5 0.1.8 40
3.5 0.1.9 317
3.5 0.2.10 5887
3.5 0.2.11 828
3.5 0.2.12 1051
3.5 0.2.3 16
3.5 0.2.4 2698
3.5 0.2.5 5024
3.5 0.2.6 194
3.5 0.2.7 6
3.5 0.2.8 5840
3.5 0.2.9 9973
3.6 0.1.7 51450
3.6 0.1.8 6732
3.6 0.1.9 8665
3.6 0.2.0 84
3.6 0.2.10 684398
3.6 0.2.11 83264
3.6 0.2.12 99211
3.6 0.2.2 31
3.6 0.2.3 9
3.6 0.2.4 546
3.6 0.2.5 54253
3.6 0.2.6 37351
3.6 0.2.7 192
3.6 0.2.8 766426
3.6 0.2.9 1289951
3.7 0.1.4 1
3.7 0.1.5 54
3.7 0.1.6 116
3.7 0.1.7 181080
3.7 0.1.8 31894
3.7 0.1.9 108883
3.7 0.2.0 34
3.7 0.2.10 1587073
3.7 0.2.11 176225
3.7 0.2.12 222093
3.7 0.2.2 697
3.7 0.2.3 1601
3.7 0.2.4 14324
3.7 0.2.5 765784
3.7 0.2.6 201724
3.7 0.2.7 830
3.7 0.2.8 1877494
3.7 0.2.9 3064506
3.8 0.0.1 1
3.8 0.1.0 1
3.8 0.1.3 5
3.8 0.1.4 2
3.8 0.1.6 480
3.8 0.1.7 56617
3.8 0.1.8 120587
3.8 0.1.9 164427
3.8 0.2.0 277
3.8 0.2.10 2216282
3.8 0.2.11 259522
3.8 0.2.12 327542
3.8 0.2.2 289
3.8 0.2.3 547
3.8 0.2.4 4191
3.8 0.2.5 602905
3.8 0.2.6 429148
3.8 0.2.7 2981
3.8 0.2.8 2516766
3.8 0.2.9 4498352
3.9 0.0.1 2
3.9 0.1.4 3
3.9 0.1.5 2
3.9 0.1.6 6
3.9 0.1.7 36367
3.9 0.1.8 8682
3.9 0.1.9 49313
3.9 0.2.0 231
3.9 0.2.10 1369850
3.9 0.2.11 194189
3.9 0.2.12 226996
3.9 0.2.2 249
3.9 0.2.3 3838
3.9 0.2.4 3016
3.9 0.2.5 469177
3.9 0.2.6 1070897
3.9 0.2.7 5369
3.9 0.2.8 1658737
3.9 0.2.9 2633441
hugovk commented 7 months ago

Please see https://github.com/jquast/wcwidth/pull/117 to drop 2.7 and 3.5.

And for typing on the whole, #71, I am in no hurry to drop support for ~1 million downloads/month of "users", for the ~10 developer who might benefit from types, and even then I believe there is little to gain there as this library API is basic, obvious and the types are in the docstrings, it's hardly worth the attention!

You can add me as number 11 who would like types :) I also maintain PrettyTable which accounts for about 10 million downloads/month of wcwidth, around 20%.