search

huggingface / optimum-neuron

Easy, fast and very cheap training and inference on AWS Trainium and Inferentia chips.
Apache License 2.0
192 stars 59 forks source link

Inf1 inference test failures #383

Open musunita opened 9 months ago

musunita commented 9 months ago

INF1 Test Summary:

= 227 passed, 200 skipped, 248 deselected, 47195 warnings, 9 errors in 7509.34s (2:05:09

Test Steps :

Config: Inf1.6xlarge tests

Release version: neuron-cc 1.21.0.0+6cae3b7b8 neuronperf 1.8.15.0+5dccae385 optimum-neuron 0.0.16.dev0 torch-neuron 1.13.1.2.9.17.0

9 of Testcases have errors:

NeuronModelIntegrationTest::test_load_model_from_cache NeuronModelIntegrationTest::test_load_model_from_empty_cache NeuronModelIntegrationTest::test_load_model_from_hub NeuronModelIntegrationTest::test_load_model_from_hub_subfolder NeuronModelIntegrationTest::test_load_model_from_hub_subfolder NeuronModelIntegrationTest::test_load_model_from_local_path NeuronModelIntegrationTest::test_save_compiler_intermediary_files NeuronModelIntegrationTest::test_save_model NeuronModelIntegrationTest::test_trust_remote_code

Error logs:

RuntimeError: Please specify the token via the HF_TOKEN_OPTIMUM_NEURON_CI environment variable

Attached complete logs for the failures: inf1_on_test_results.txt

JingyaHuang commented 9 months ago

Hi @musunita , all these hub integration tests need a huggingface token. If you are running it locally, you can export a dummy HF token as env variable to run the test.

musunita commented 9 months ago

Can you clarify, how do you export dummy HF token as env variable ? With dummyHF token, it fails still

E           requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/api/whoami-v2

../aws_neuron_venv_pytorch/lib/python3.8/site-packages/requests/models.py:1021: HTTPError

The above exception was the direct cause of the following exception:

self = <huggingface_hub.hf_api.HfApi object at 0x7f724f02ee80>
token = 'faketoken'
jeffhataws commented 8 months ago

Setup account to get token here: https://huggingface.co/docs/huggingface_hub/quick-start#authentication

JingyaHuang commented 7 months ago

Hi folks, is the token issue solved? May I close this issue? @musunita @jeffhataws

HuggingFaceDocBuilderDev commented 5 months ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

HuggingFaceDocBuilderDev commented 4 months ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

HuggingFaceDocBuilderDev commented 3 months ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

HuggingFaceDocBuilderDev commented 3 months ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

jeffhataws commented 2 months ago

Let's close this for now. Will open new one if there are issues.

HuggingFaceDocBuilderDev commented 2 months ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

HuggingFaceDocBuilderDev commented 1 month ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

HuggingFaceDocBuilderDev commented 2 weeks ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!