SichangHe
commented
1 year ago Related #158.
Closed SichangHe closed 1 year ago
SichangHe
commented
1 year ago Related #158.
baijumeswani
commented
1 year ago We will be releasing the python package for mac in ort 1.16 release (which should happen very soon).
In the meantime, you could consider getting the nightly python package using
pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ onnxruntime-training-cpuThank you for the link and instruction. However, it seems that the website requires authentication:
$ pip3 install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ onnxruntime-training-cpu
Looking in indexes: https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/
Collecting onnxruntime-training-cpu
Downloading https://aiinfra.pkgs.visualstudio.com/2692857e-05ef-43b4-ba9c-ccf1c22c437c/_packaging/7982ae20-ed19-4a35-a362-a96ac99897b7/pypi/download/onnxruntime-training-cpu/1.16.dev20230828001/onnxruntime_training_cpu-1.16.0.dev20230828001-cp311-cp311-macosx_11_0_arm64.whl (8.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.0/8.0 MB 1.1 MB/s eta 0:00:00
User for aiinfra.pkgs.visualstudio.com:After I just pressed
WARNING: 401 Error, Credentials not correct for https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/cerberus/
INFO: pip is looking at multiple versions of onnxruntime-training-cpu to determine which version is compatible with other requirements. This could take a while.
Downloading https://aiinfra.pkgs.visualstudio.com/2692857e-05ef-43b4-ba9c-ccf1c22c437c/_packaging/7982ae20-ed19-4a35-a362-a96ac99897b7/pypi/download/onnxruntime-training-cpu/1.16.dev20230825001/onnxruntime_training_cpu-1.16.0.dev20230825001-cp311-cp311-macosx_11_0_arm64.whl (8.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.0/8.0 MB 1.8 MB/s eta 0:00:00
Downloading https://aiinfra.pkgs.visualstudio.com/2692857e-05ef-43b4-ba9c-ccf1c22c437c/_packaging/7982ae20-ed19-4a35-a362-a96ac99897b7/pypi/download/onnxruntime-training-cpu/1.16.dev20230824001/onnxruntime_training_cpu-1.16.0.dev20230824001-cp311-cp311-macosx_11_0_arm64.whl (8.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.0/8.0 MB 1.3 MB/s eta 0:00:00
Downloading https://aiinfra.pkgs.visualstudio.com/2692857e-05ef-43b4-ba9c-ccf1c22c437c/_packaging/7982ae20-ed19-4a35-a362-a96ac99897b7/pypi/download/onnxruntime-training-cpu/1.16.dev20230822001/onnxruntime_training_cpu-1.16.0.dev20230822001-cp311-cp311-macosx_11_0_arm64.whl (7.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.9/7.9 MB 1.8 MB/s eta 0:00:00
ERROR: Cannot install onnxruntime-training-cpu==1.16.0.dev20230822001, onnxruntime-training-cpu==1.16.0.dev20230824001, onnxruntime-training-cpu==1.16.0.dev20230825001 and onnxruntime-training-cpu==1.16.0.dev20230828001 because these package versions have conflicting dependencies.
The conflict is caused by:
onnxruntime-training-cpu 1.16.0.dev20230828001 depends on cerberus
onnxruntime-training-cpu 1.16.0.dev20230825001 depends on cerberus
onnxruntime-training-cpu 1.16.0.dev20230824001 depends on cerberus
onnxruntime-training-cpu 1.16.0.dev20230822001 depends on cerberus
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflictsThank you for your help.
SichangHe
commented
1 year ago I solved the above issue by manually installing from the .whl file.
SichangHe
commented
1 year ago baijumeswani
commented
1 year ago After installing onnxruntime-training-cpu, I cannot import onnxruntime any more.
This seems to be a problem. I will investigate.
After I reinstalled onnxruntime, I still have errors.
onnxruntime-training-cpu and onnxruntime cannot both be installed together. I would recommend uninstall onnxruntime-training-cpu before installing onnxruntime.
And, if I force reinstall onnxruntime-training-cpu, I am back to the first problem.
Will investigate. In the mean time, if you're looking for the training utilities in onnxruntime, you could try to (not sure if it will work).
import onnxruntime.training.api as orttraining
from onnxruntime.training import artifactsFor this issue:
$ pip3 install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ onnxruntime-training-cpu
Looking in indexes: https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/
Collecting onnxruntime-training-cpu
Downloading https://aiinfra.pkgs.visualstudio.com/2692857e-05ef-43b4-ba9c-ccf1c22c437c/_packaging/7982ae20-ed19-4a35-a362-a96ac99897b7/pypi/download/onnxruntime-training-cpu/1.16.dev20230828001/onnxruntime_training_cpu-1.16.0.dev20230828001-cp311-cp311-macosx_11_0_arm64.whl (8.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.0/8.0 MB 1.1 MB/s eta 0:00:00
User for aiinfra.pkgs.visualstudio.com:
Please first install all dependencies from pip. And then install onnxruntime-training-cpu from the link I shared.
pip install cerberus flatbuffers h5py numpy onnx packaging protobuf sympy setuptools
pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ onnxruntime-training-cpu@baijumeswani, thanks for the updated instructions. Unfortunately, I still cannot successfully run the generation script.
But, when I ran the generation script again, it still failed with the exact same problem as before.
$ python3 artifacts_gen.py
/opt/homebrew/lib/python3.11/site-packages/transformers/configuration_utils.py:380: UserWarning: Passing `gradient_checkpointing` to a config initialization is deprecated and will be removed in v5 Transformers. Using `model.gradient_checkpointing_enable()` instead, or if you are using the `Trainer` API, pass `gradient_checkpointing=True` in your `TrainingArguments`.
warnings.warn(
/opt/homebrew/lib/python3.11/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py:595: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
/opt/homebrew/lib/python3.11/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py:634: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
================ Diagnostic Run torch.onnx.export version 2.0.1 ================
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
Traceback (most recent call last):
File "/Users/sichanghe/AFileFolder/microsoft--onnxruntime-training-examples/on_device_training/mobile/ios/artifacts_gen.py", line 18, in <module>
import onnxruntime.training.onnxblock as onnxblock
File "/opt/homebrew/lib/python3.11/site-packages/onnxruntime/__init__.py", line 53, in <module>
from onnxruntime.capi import onnxruntime_validation
File "/opt/homebrew/lib/python3.11/site-packages/onnxruntime/capi/onnxruntime_validation.py", line 145, in <module>
has_ortmodule, package_name, version, cuda_version = validate_build_package_info()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/onnxruntime/capi/onnxruntime_validation.py", line 140, in validate_build_package_info
raise import_ortmodule_exception
File "/opt/homebrew/lib/python3.11/site-packages/onnxruntime/capi/onnxruntime_validation.py", line 70, in validate_build_package_info
from onnxruntime.training.ortmodule import ORTModule # noqa: F401
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/onnxruntime/training/ortmodule/__init__.py", line 21, in <module>
raise RuntimeError("ORTModule is not supported on this platform.")
RuntimeError: ORTModule is not supported on this platform.For your information, @baijumeswani.
baijumeswani
commented
1 year ago I updated the code in https://github.com/microsoft/onnxruntime/pull/17380.
Could you uninstall onnxruntime and onnxruntime-training-cpu and try to install the latest onnxruntime-training-cpu nightly using
pip install cerberus flatbuffers h5py numpy onnx packaging protobuf sympy setuptools pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ onnxruntime-training-cpu
SichangHe
commented
1 year ago It works. I successfully ran artifact_gen.py.
Thank you, @baijumeswani! I will keep you updated with how trying out the app goes for me.
I am trying to run
on_device_training/mobile/ioson an M1 Mac. I could not installonnxruntime-training.Inspecting the verbose output of Pip, it seems there exists no
.whlfor ARM. I've tried Python 3.11 and 3.10.Also,
onnxruntime-training==1.16.0specified inrequirements.txtis not released, yet. Is this an internal version?Alternatively, @vraspar, could you maybe compress the artifacts and upload them here so I can try out the iOS example?
Thanks!