[BUG] Incorrect Sort Parameter in fetch_models function

bhumika-scogo commented 3 weeks ago

Prerequisites

[X] I have read the documentation.
[X] I have checked other issues for similar problems.

Backend

Other cloud providers

Interface Used

CLI

CLI Command

$ conda create -n autotrain python=3.10 $ conda activate autotrain $ pip install autotrain-advanced $ conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia $ conda install -c "nvidia/label/cuda-12.1.0" cuda-nvcc $ conda install xformers -c xformers $ python -m nltk.downloader punkt $ pip install flash-attn --no-build-isolation # if you want to use flash-attn $ pip install deepspeed # if you want to use deepspeed

$ conda install chardet $ export HF_TOKEN=your_hugging_face_write_token $ autotrain app --host 0.0.0.0 --port 8000

UI Screenshots & Parameters

No response

Error Logs

 autotrain app --host 0.0.0.0 --port 8000

Your installed package `nvidia-ml-py` is corrupted. Skip patch functions `nvmlDeviceGet{Compute,Graphics,MPSCompute}RunningProcesses`. You may get incorrect or incomplete results. Please consider reinstall package `nvidia-ml-py` via `pip3 install --force-reinstall nvidia-ml-py nvitop`.
Your installed package `nvidia-ml-py` is corrupted. Skip patch functions `nvmlDeviceGetMemoryInfo`. You may get incorrect or incomplete results. Please consider reinstall package `nvidia-ml-py` via `pip3 install --force-reinstall nvidia-ml-py nvitop`.
INFO     | 2024-06-12 07:14:17 | autotrain.app.ui_routes:<module>:30 - Starting AutoTrain...
Traceback (most recent call last):
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status
    response.raise_for_status()
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://huggingface.co/api/models?sort=likes7d&direction=-1&limit=30&filter=transformers&filter=fill-mask

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/sdc/scogo2/miniconda3/envs/autotrain/bin/uvicorn", line 8, in <module>
    sys.exit(main())
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/uvicorn/main.py", line 409, in main
    run(
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/uvicorn/main.py", line 575, in run
    server.run()
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/uvicorn/server.py", line 65, in run
    return asyncio.run(self.serve(sockets=sockets))
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/uvicorn/server.py", line 69, in serve
    await self._serve(sockets)
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/uvicorn/server.py", line 76, in _serve
    config.load()
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/uvicorn/config.py", line 433, in load
    self.loaded_app = import_from_string(self.app)
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/uvicorn/importer.py", line 19, in import_from_string
    module = importlib.import_module(module_str)
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/autotrain/app/app.py", line 10, in <module>
    from autotrain.app.ui_routes import ui_router
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/autotrain/app/ui_routes.py", line 38, in <module>
    MODEL_CHOICE = fetch_models()
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/autotrain/app/models.py", line 316, in fetch_models
    _mc["text-classification"] = _fetch_text_classification_models()
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/autotrain/app/models.py", line 37, in _fetch_text_classification_models
    trending_models = list(
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/huggingface_hub/hf_api.py", line 1633, in list_models
    for item in items:
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/huggingface_hub/utils/_pagination.py", line 37, in paginate
    hf_raise_for_status(r)
  File "/sdc/scogo2/miniconda3/envs/autotrain/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py", line 358, in hf_raise_for_status
    raise BadRequestError(message, response=response) from e
huggingface_hub.utils._errors.BadRequestError:  (Request ID: Root=1-66694aca-669d86fa7029fcee52801f90;7aac79dd-8da4-4c8c-9e48-731c034eeebc)

Bad request:
Invalid sort parameter

Additional Information

I would appreciate it if you could investigate this issue and potentially:

Update the fetch_models function to use a valid sorting parameter for the Hugging Face Hub API.
Provide guidance on alternative ways to sort the fetched models within the AutoTrain application.

ksingh-scogo commented 3 weeks ago

@abhishekkrthakur it seems like this part has issues

requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://huggingface.co/api/models?sort=likes7d&direction=-1&limit=30&filter=transformers&filter=fill-mask

The URL that autotrain app is creating at run time seems incorrect https://huggingface.co/api/models?sort=likes7d&direction=-1&limit=30&filter=transformers&filter=fill-mask

If i remove 7d query string it works https://huggingface.co/api/models?sort=likes&direction=-1&limit=30&filter=transformers&filter=fill-mask

ksingh-scogo commented 3 weeks ago

@abhishekkrthakur all instances of likes7d needs to be changed to likes on this app/models.py

abhishekkrthakur commented 3 weeks ago

this was fixed by hf side, since there is no response to pr comment, ill consider this resolved :)

huggingface / autotrain-advanced