Closed stonebig closed 1 year ago
audit trail:
b1 (2022-08-07)
interlude Python-3.11.0rc1 (2022-08-09)
b2 (2022-08-16)
b3 (2022-09-18)
b4 (2022-10-16)
b5 (2022-10-23)
<details>
and </details>
to simplify publicationrc (2022-10-31)
postponed:
- [ ] BLAS move (if we loose cgohlke site)
While the UC Irvine LFD site hasn't been taken down yet, the site has already been archived and it seems like Gohlke has stopped compiling new wheels. For example, MKL wheels are not available for Numpy 1.23, which was released 3 days before the last update of the LFD site, or for Numpy 1.23.1. I suppose even he didn't build every release of Numpy against MKL though. Intel's own builds are obviously even further behind (wheels for CPython 3.10 aren't even available yet).
Could you add binaries for Python 3.9.13?
Could you add binaries for Python 3.9.13?
There is no resources nor interest nor environmental benefit to maintain & update old cpython-3.x versions.
Has anyone tried to contact @cgohlke to know the status? It will be a big lost, as the MKL is one of the best features of WinPython.
He announced on that site a month or two ago that UC Irvine LFD lost funding, so the site would be shutting down before July.
I see. Too bad.
I hope he shares his building scripts so it might be replicated. Maybe even by a GitHub Actions or something?
Has anyone tried to contact @cgohlke to know the status? It will be a big lost, as the MKL is one of the best features of WinPython.
Hummm, looking at an Intel site, it seems that "pip install mkl" may offer a path forward: "pip install mkl numpy"
Counting MKL casualties ... at first look, it seems that:
So:
b1 (2022-08-06/0x)
Changes from WinPython 2022-02:
variation per version:
MD5 | SHA-1 | SHA-256 | Binary | Size | SHA3-256 |
---|---|---|---|---|---|
d1ef69786866695a92f5e0afee2678e8 | 02969dcac7325a751db66afacb5a965ef758da07 | 8f3442f7824c4e1dd96370f0b0965ee50de28da5b982cb26bab9c7336618c3c2 | Winpython64-3.10.6.1dotb1.exe | 27 422 957 Bytes | e1a1a870d8316e493baea608f6ad0278ce8090eb97783a0747c1de8ded6a299a |
e28245aeff00dac838d23abc1ae1c4b3 | aef7f5a9910985a8672f98dab954447fa5e34d61 | a03d95a7fe34c5d7811b4d39250da43994e8da54a4d09790cbb95224b14228af | Winpython32-3.10.6.1dotb1.exe | 26 194 508 Bytes | 61b65601e9b2b7a9707ed05c485c4c030fc1708608d1fe665eb37356dd8786ee |
89cafaaeae4d72ba04ba03b6be26941c | e47cc57875e4caf3d1d1716bf3340d380f994c9c | 5d029b45227195ef2a21c9ac1a5070e229cedc441030e92a54911d183ab7331f | Winpython64-3.10.6.1b1.exe | 818 844 345 Bytes | ed2513b70619c8855919a72950f3d289d1d1fbd2e8f0618dc2e4c067a3c411ae |
Areas of particular interest for testers:
Next build effort:
@stonebig , Does it mean you were able to overcome the MKL issues?
@stonebig , Does it mean you were able to overcome the MKL issues?
Winpython64-3.10.6.1b1.exe of today is made with intel MKL standard wheel. Not sure it's all perfect yet in every corner: testing and feedback is recommanded.
@stonebig , I will test it. What I meant is that you wrote that the Intel distribution of NumPy / SciPy is behind the release cycle. Did you find an alternative which is updated with each release of NumPy / SciPy and in a timely manner?
@stonebig , I will test it. What I meant is that you wrote that the Intel distribution of NumPy / SciPy is behind the release cycle. Did you find an alternative which is updated with each release of NumPy / SciPy and in a timely manner?
on "current" cpython, it shall remain timely. The problem is for PyPy or un-final-released cpython-3.11 Winpython64-3.10.6.1b1.exe is uploaded on sourceforge now
interlude (2022-08-09): Python-3.11.0rc1 (following python-3.11.0rc1 development cycle)
The Winpython-3.11.0.0 versions follows the cycle of Python-3.11.0 (alphas, betas, release_candidates). It is intend for enthousiasts and testers willing to find bugs and contribute issues or patchs upstreams. It shall help python-3.11.0 final hitting the ground running.
Apparently:
The 3.11.0.1 will actually be "for true release" and follow the normal scheme
Changes from Python-3.11.0b4 :
Areas of particular interest for testers:
Next build effort:
official wheels (if it can exist outside of Christoph Gohlke during rc phase of python-3.11.0... 20% speed-up is the honney pot)
MD5 | SHA-1 | SHA-256 | Binary | Size | SHA3-256 |
---|---|---|---|---|---|
29dbaebee6c9cf916b9d5c2ec443f9c9 | 9b12cc5de33e6fb689403e4381a0cb56def62631 | 45b4d1213918ff5a294d1d5d35fef39d6d960b6fafc0634afd062a825738562e | Winpython64-3.11.0.0rc1.exe | 594 622 799 Bytes | 62eee5c4e618f84b9320cfe2dad4f04b07226af4be5e7076121b5b1746724386 |
ef446e563f6f90bb54a253f2df83e994 | f2bd214dc542ade49bdc7d32db74dbe4d3d1245a | 689f4c106aaca893b6f84fefa4c8847cc76e5ff026c85de61f8f3c7802e98a07 | Winpython64-3.11.0.0dotrc1.exe | 24 390 514 Bytes | 2696710ae016579c539d98b475c131b14293fdf175b13eb344fe04f7c7210c0a |
download is at: https://sourceforge.net/projects/winpython/files/WinPython_3.11/3.11.0.0/betas/
The following packages are included in WinPython-64bit v3.11.0.0 rc1.
b2 (2022-08-15: infrastructure modernization)
Changes from WinPython 2022-03 b1:
Areas of particular interest for testers:
Next build effort:
Python-3.11 stack: numba
MD5 | SHA-1 | SHA-256 | Binary | Size | SHA3-256 |
---|---|---|---|---|---|
c011894729edca5aac940adc8089d102 | 21812e597bf910c2c23500dc6b9a9c72de38a85f | 75e73bbdd8dd43c815bd198f6a1f4dc00795d02ce2f2ec597111425c5f004429 | Winpython64-3.10.6.1dotb2.exe | 27 446 291 Bytes | 7d301b0d872ec50ddce7858ef91f3d173a0630df89d51da114fdb4ca1a87838e |
31267e493b68b9d91f2e753bcfccdcd2 | b0d7e44ce82a9d80e912f9a01c416ec29363a24a | 7ce0e86b8f428b8ba8d5e6fbb7c1d797ac24370f339f4c1816a10e1c1ffdf8cd | Winpython32-3.10.6.1dotb2.exe | 26 217 438 Bytes | c49f4ff02a26cb289e6fa5a7c1fd35a5cac071cfbb1cfdf6a23b84ee85b1f2ac |
236b8b9aa2f5e0db3d45762ae0c2ae68 | b296dab32b07e4cc0555f5397fde312d93ff5cec | 08c16d7e4dd3f166c69aa595a8645fb8db3f3713b80d9c2898034f149540887d | Winpython64-3.10.6.1b2.exe | 818 306 506 Bytes | c186013a33c1854b248d74045c04e0c8c4f30d8f6376ebcfc66fde835d61548a |
The following packages are included in WinPython-64bit v3.10.6.1 b2.
@stonebig , When you say MKL
is added, do you mean you add the wheel for MKL but also a version of NumPy
and SciPy
which are compiled with MKL
, right?
Could you share the links to those wheels?
@stonebig , When you say
MKL
is added, do you mean you add the wheel for MKL but also a version ofNumPy
andSciPy
which are compiled withMKL
, right?Could you share the links to those wheels?
No, the hope is that the standard wheel of numpy see and choose the mkl. The mkl itself is a shared dll
@stonebig , I think it won't. I think one must build NumPy / SciPy in a way that utilize MKL. Do you have any hint to think differently?
Humm, maybe you're right, https://www.intel.com/content/www/us/en/developer/articles/tool/oneapi-standalone-components.html#python
or
I don't know how to check mkl is used per numpy If you can find à heck test or command
@stonebig , When you say
MKL
is added, do you mean you add the wheel for MKL but also a version ofNumPy
andSciPy
which are compiled withMKL
, right?Could you share the links to those wheels?
so the possible link is this one, https://pypi.org/project/intel-numpy/#files and https://pypi.org/project/intel-scipy/1.7.3/#files ... but it stops at python-3.9 ... you trade python-3.10 freshness and speed-up to get mkl.... wired
If the mkl is unusable in current python without cgohlke , mkl will go away to make room for alternatives.
winpython 2022-03b1 python 3.10
winpython 2022-03b1 python 3.11 (so we notice cgohlke was recompiling intel_numpy)
This is Intel NumPy, They recompile it with MKL. It seems your path is just add the MKL DLL's not a recompiled NumPy?
What I'm saying is that I'm not sure just adding the DLL is enough without recompilation. What do you think?
This is Intel NumPy, They recompile it with MKL. It seems your path is just add the MKL DLL's not a recompiled NumPy?
What I'm saying is that I'm not sure just adding the DLL is enough without recompilation. What do you think?
interesting thread here https://github.com/scipy/scipy/issues/11812
and so mkl/numpy/scipy shall come from the same nest.... like cgohlke's lost one.
Yea, This is similar to LibTrampoline project at Julia. The question if this was implemented. It doesn't seem so.
Yea, This is similar to LibTrampoline project at Julia. The question if this was implemented. It doesn't seem so.
As I see numpy contributors in 'PyPy' and 'https://data-apis.org/array-api/latest/", I suppose:
These people are very busy making their Mac M1 & M2 working at maximum speed, so the 'MKL' thing can be percieved as 'passé'.
The constant move to 'latest' remains the strategy to:
'MKL' is becoming a numpy's legacy, otherwise it would be simpler to use it in official python-3.10/numpy
Where did you get the notion it becomes a legacy? There is no alternative, yet at least, to MKL on the x86
sphere where WinPython is targeting. OpenBLAS isn't close when you use LAPACK functions.
If WinPython switches to OpenBLAS it means the performance will deteriorate for numerical work much much more than the 30-40% achieved in Python 3.11 (Which won't help with numerical work).
What about replicating @cgohlke recipes to build NumPy and SciPy? I tried approaching him, yet I did not get any response. Maybe you can try? He'll probably answer you.
Where did you get the notion it becomes a legacy?
'pip intel-numpy' is not available for python-3.10, from the company which shall be the most motivated to feed it to you.
It is because their distribution isn't popular. Most of the people use Conda and they have NumPy with MKL (This is the default) on their repository.
So maybe using the package from Conda is the solution here?
Maybe in the long run, this might help: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2209877.
Will be done by july 2025, not exactly tomorrow.....
I wrote in the long run :-). If we get the recipes from @cgohlke I will be trying to replicate them for WinPython. But can we get in touch with him? Have you tried?
I wrote in the long run :-). If we get the recipes from @cgohlke I will be trying to replicate them for WinPython. But can we get in touch with him? Have you tried?
No There is no fun nor interest nor strategy that would motivate such a move/burden. It can only be done and sustained per a way bigger organization.
I don't need to recreate all the recipes he created, Only NumPy and SciPy.
There is a new contender lfortran that smells interesting, to replace intel fortran:
It is a compiler, It won't make OpenBLAS faster than MKL. In MKL the kernels are written in assembly. The magic isn't the Intel compiler, it is the kernels.
It is a compiler, It won't make OpenBLAS faster than MKL. In MKL the kernels are written in assembly. The magic isn't the Intel compiler, it is the kernels.
That is something only Christoph Gohlke, or Intel or Microsoft, can carry on with the required care, and forward-looking care in case of Christoph.
What about out of the box idea? Maybe build WinPython around MicroMamba?
It is a single file to start with, so all the scripts will be much easier around it.
it's indeed a possible path. Even if it's done per frenchies, I'm a bit unsettled per the server domain https://micro.mamba.pm
What do you mean by:
I'm a bit unsettled per the server domain https://micro.mamba.pm
pm = St Pierre & Miquelon there is also this https://henryiii.github.io/level-up-your-python/live/lab/index.html
the demise of https://www.lfd.uci.edu/~gohlke/pythonlibs/ and also indirectly the fall-of-the-tree of intel are problems.
b3 (2022-09-18: Python updates)
Changes from WinPython 2022-03 b1:
Areas of particular interest for testers:
Next build effort: rc
Python-3.11.0 final
MD5 | SHA-1 | SHA-256 | Binary | Size | SHA3-256 |
---|---|---|---|---|---|
b8824a2db66626ca186980ff97038909 | 8e1e9b416ef6cabd9bbd2f35ae699b1b6f770c90 | ca45cde9be759c2360854459f4fa1085c094106d0805d4b909da920578de403e | Winpython64-3.10.7.0dotb3.exe | 27 481 953 Bytes | 7468afbf86f5952af922a6a05d333ca085196332bb31f1d4429ca7e18eb5a3d5 |
134badae3901b5d7a425a1184a3dcc30 | 76241240a5adb4f81a64b2108746ced37d4ebf24 | fdd56009cc26843f3764b8660214e24410af2a696c6196ee8effc99e4ae838ef | Winpython32-3.10.7.0dotb3.exe | 26 262 652 Bytes | 1d04b470469a3a1dea47c24a699f452dd6bbf5c3f03dbbbb6a695b3e8066fb02 |
b916b7feb1d9525cafe34b5f2f7598b6 | cad7752c3b2d1d80afc38d1a813acfde7828d9f9 | 950b839137ab1a66fe33de7c91263285a872ac2a9020a8a918324fe8075a3fa2 | Winpython64-3.10.7.0b3.exe | 824 807 263 Bytes | 3d7a63d15098566d89ceb94de1e09704ef6738ae71577852d5d43f6dcbf400a9 |
e9d497e6b100718e9611d64391ba3bf4 | 1d36cc0c843e918eacfc188366106ffb1ca7a7a0 | 0846de243b8db99d30a8bb2da17453f2f0958cf5044db5112dfc0fe4d99cbadb | Winpython64-3.11.0.0rc2.exe | 594 106 503 Bytes | d6a9a11c9c0cf6b4670cdab67a18e066c1a6277fefec1f8b6857f95fbeba9eb1 |
The following packages are included in WinPython-64bit v3.10.7.0 b3.
The following packages are included in WinPython-64bit v3.11.0.0 rc2.
Do I understand correctly that WinPython-64bit v3.10.7.0 b3 uses pip to install numpy and scipy from the following links? https://pypi.org/project/numpy/ https://pypi.org/project/scipy/
If so, why does WinPython-64bit v3.10.7.0 b3 contain scipy v1.9.0? scipy v1.9.1 was released 2022-August-26.
Do I understand correctly that WinPython-64bit v3.10.7.0 b3 uses pip to install numpy and scipy from the following links? https://pypi.org/project/numpy/ https://pypi.org/project/scipy/
If so, why does WinPython-64bit v3.10.7.0 b3 contain scipy v1.9.0? scipy v1.9.1 was released 2022-August-26.
well, it's a miss
b4 (2022-10-16: Python updates)
Changes from WinPython 2022-03 b3
Areas of particular interest for testers:
Next build effort: rc
Python-3.11.0 final
MD5 | SHA-1 | SHA-256 | Binary | Size | SHA3-256 |
---|---|---|---|---|---|
1315e0cb3023b34c5178aa3bab20ddd3 | 2974535db7a1ea1a1edd35380aff5a955b09e125 | 06159df6a2b7ea7d4cc51b7fd8003f7b8797e75807cf82cdefd54cf7b6e06dfa | Winpython64-3.10.8.0dotb4.exe | 27 485 071 Bytes | 22519498a5d6b1e9023842d703df2c2236e5e523379888a0cc7569daee7b9cd1 |
c4112eb524196316ccf711df97820ceb | 7eccc437a5c1dd750514d7ce9753e149fffe835f | ac8eaa0e3f90fe79c45a62159a6bc194387b279bfc467a81a5d4d203b311bcc2 | Winpython32-3.10.8.0dotb4.exe | 26 269 280 Bytes | 378ac2fbd6a0076a25fc7e28595a6c33783fef12990ade3cf8d47188d4aa9250 |
85538a05af371eb884992412d721b3d6 | 7625e97437a4c868d3f5f50061478de896083126 | 8ce40dc5348b0a84363f6d137e58d545c1e5a5497b95842b05f73c4dfa1eeaae | Winpython64-3.10.8.0b4.exe | 828 566 769 Bytes | 514502af7c11005e3d979646250998d4611ea752ba7ce3dc88b978cc9fc1241e |
The following packages are included in WinPython-64bit v3.10.8.0 b4.
I don't see why you add MKL
.
Just adding it while using the regular NumPy
and SciPy
packages doesn't do a thing.
It just takes a volume but nothing is using it.
I don't see why you add
MKL
. Just adding it while using the regularNumPy
andSciPy
packages doesn't do a thing. It just takes a volume but nothing is using it.
I had a doubt. Did you experiment a bit with mamba ?
Yes. It worked like conda
just much much faster.
It seems you'll be able to create a portable distribution around it much easier.
b5 (2022-10-23: python-3.10.8 IDLE fix, python-3.11 preparation)
Changes from WinPython 2022-03 b4
Areas of particular interest for testers:
Next build effort: rc
Python-3.11.0 final (and hopefully some missing packages will be build)
MD5 | SHA-1 | SHA-256 | Binary | Size | SHA3-256 |
---|---|---|---|---|---|
3cc00ef9f7584b24b65be8759a9fdff4 | 10b3eb01e72e25d085568522c4a7c857b595d544 | 6746b9e41b2b982965f004a00a6ebedd72eb3228804caa7880598ac9a1da7c03 | Winpython64-3.10.8.0dotb5.exe | 27 486 393 Bytes | 6b3b0a8d1b7d59e4e2d261401c5e6412f0d38dcbbf48d92b9c71db12532bb071 |
d684fade00ddb59985a645c632536895 | 0c99d28345296b75ab732d00cb18c2579df49c6b | c1be752c8cb0b0be79c2c34aee30e5b3905f567d463eb664358e2c968516b13e | Winpython32-3.10.8.0dotb5.exe | 26 268 352 Bytes | 803dc69bdc81c55dbdff52ab2ca0fa4c5359e0475f123b68dfdf1a5015ebf03d |
faf861629e7ec92565ef4afbcaa438be | a5b7bc019801712dcc6009e85f9cdaa3ece52f08 | 93449163625279dc694bddcf1dccb4e303bfd19ea5eb73f78b50012625e2d84d | Winpython64-3.10.8.0b5.exe | 702 498 311 Bytes | dfc306bb1e55bf9d600caef73760aed45c7fc2aac93d80c73eed2753870eee0a |
65ea92619edd1a6713562ff84f9f64f1 | 125537f9965129f3de073de349a4cf2416f4c3a5 | 5bc558f3e34303b51e0243bd74d83afc203c300dc02820df7b2ec33bfd290691 | Winpython64-3.11.0.1b5.exe | 470 838 349 Bytes | b46d9c7f700a755856f651d898650aea6fed214f311110afa908ffb43ec89210 |
65ea92619edd1a6713562ff84f9f64f1 | 125537f9965129f3de073de349a4cf2416f4c3a5 | 5bc558f3e34303b51e0243bd74d83afc203c300dc02820df7b2ec33bfd290691 | Winpython64-3.11.0.1b5.exe | 470 838 349 Bytes | b46d9c7f700a755856f651d898650aea6fed214f311110afa908ffb43ec89210 |
The following packages are included in WinPython-64bit v3.10.8.0 b5.
The following packages are included in WinPython-64bit v3.11.0.1 b5.
Hello, it is possible to upgrade the python version of WinPython or do I have to reinstall it again? Sorry, if it not the right space, but I couldn't find any information about this topic.
release: october 2022
Wanted:
Focus:
Postpone:
Hopes/Wishes for 2022/2023: