ImportError: cannot import name 'OpenAI' from 'openai' (/usr/local/lib/python3.10/dist-packages/openai/__init__.py)

[DaddyCodesAlot]

When attempting to use "from unsloth import FastLanguageModel", the following error pops up:

---

ImportError Traceback (most recent call last) Cell In[14], line 1 ----> 1 from unsloth import FastLanguageModel 2 import torch 3 import gc

File /usr/local/lib/python3.10/dist-packages/unsloth/init.py:154 144 warnings.warn( 145 "Unsloth: CUDA is not linked properly.\n"\ 146 "Try running python -m bitsandbytes then python -m xformers.info\n"\ (...) 150 "Unsloth will still run for now, but maybe it might crash - let's hope it works!" 151 ) 152 pass --> 154 from .models import 155 from .save import 156 from .chat_templates import *

File /usr/local/lib/python3.10/dist-packages/unsloth/models/init.py:15 1 # Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved. 2 # 3 # Licensed under the Apache License, Version 2.0 (the "License"); (...) 12 # See the License for the specific language governing permissions and 13 # limitations under the License. ---> 15 from .loader import FastLanguageModel 16 from .llama import FastLlamaModel 17 from .mistral import FastMistralModel

File /usr/local/lib/python3.10/dist-packages/unsloth/models/loader.py:16 1 # Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved. 2 # 3 # Licensed under the Apache License, Version 2.0 (the "License"); (...) 12 # See the License for the specific language governing permissions and 13 # limitations under the License. 15 from ._utils import is_bfloat16_supported, HAS_FLASH_ATTENTION, HAS_FLASH_ATTENTION_SOFTCAPPING ---> 16 from .llama import FastLlamaModel, logger 17 from .mistral import FastMistralModel 18 from .qwen2 import FastQwen2Model

File /usr/local/lib/python3.10/dist-packages/unsloth/models/llama.py:32 28 from transformers.modeling_attn_mask_utils import ( 29 _prepare_4d_causal_attention_mask_for_sdpa, 30 ) 31 from ..kernels import ---> 32 from ..tokenizer_utils import 33 if HAS_FLASH_ATTENTION: 34 from flash_attn import flash_attn_func

File /usr/local/lib/python3.10/dist-packages/unsloth/tokenizer_utils.py:1109 1105 PRE_CHECK = check_nvidia() 1108 from inspect import getsource -> 1109 import trl.trainer.sft_trainer 1110 from trl.trainer.sft_trainer import 1111 from transformers.trainer import

File /usr/local/lib/python3.10/dist-packages/trl/trainer/sft_trainer.py:43 41 from ..extras.dataset_formatting import get_formatting_func_from_dataset 42 from ..import_utils import is_liger_available, is_peft_available ---> 43 from .callbacks import RichProgressCallback 44 from .sft_config import SFTConfig 45 from .utils import ( 46 ConstantLengthDataset, 47 DataCollatorForCompletionOnlyLM, (...) 50 trl_sanitze_kwargs_for_tagging, 51 )

File /usr/local/lib/python3.10/dist-packages/trl/trainer/callbacks.py:37 34 from transformers.trainer_utils import has_length 36 from ..models.utils import unwrap_model_for_generation ---> 37 from .judges import BaseRankJudge 38 from .utils import truncate_right 41 if is_deepspeed_available():

File /usr/local/lib/python3.10/dist-packages/trl/trainer/judges.py:18 15 import llm_blender 17 if is_openai_available(): ---> 18 from openai import OpenAI 21 DEFAULT_PAIRWISE_SYSTEM_PROMPT = '''I require a leaderboard for various large language models. I'll provide you with prompts given to these models and their corresponding outputs. Your task is to assess these responses, and select the model that produces the best output from a human perspective. 22 23 ## Instruction (...) 46 Evaluate the models on the basis of the quality and relevance of their results, and select the model that generated the best result. Reply with the identifier of the best model. Our evaluation will only take into account the first character of your answer, so make sure it contains only one of the identifiers and nothing else (no quotation marks, no spaces, no new lines, ...). 47 ''' 50 class BaseJudge(ABC):

ImportError: cannot import name 'OpenAI' from 'openai' (/usr/local/lib/python3.10/dist-packages/openai/init.py)

[DaddyCodesAlot]

Seems to be an issue with a packaged called trl . My tempt fix is to install an earlier version of trl and then an earlier version of unsloth

!pip install "unsloth[cu118] @ git+https://github.com/unslothai/unsloth.git@600ffe2a175312e7f2fbcdffbe6b99e3df20c417"

and !pip install trl==0.9.6

[danielhanchen]

Oh my why is openai a necessary package? That's very weird - let me check TRL

[DaddyCodesAlot]

All good, it's from the latest version of TRL

[rohan-daniscox]

@DaddyCodesAlot are you using Databricks by any chance? I keep running into the same errors as you

[danielhanchen]

I added a fix to bypass this in the nightly branch if you guys wanna try it out @DaddyCodesAlot @rohan-daniscox

[DaddyCodesAlot]

@DaddyCodesAlot are you using Databricks by any chance? I keep running into the same errors as you

Databricks? As in the model DBRX? If so, I'd avoid using that model, to date it's implementation is buggy

[rohan-daniscox]

Databricks? As in the model DBRX? If so, I'd avoid using that model, to date it's implementation is buggy

No, I meant running unsloth on a Databricks runtime. I'll assume not.

I added a fix to bypass this in the nightly branch if you guys wanna try it out @DaddyCodesAlot @rohan-daniscox

Thanks Daniel, working with Databricks runtimes can be slightly painful as they have a host of other packages which can interact with huggingface packages in unexpected ways, and cannot be easily uninstalled. So my solution here was only to upgrade deepspeed, even though it is of no consequence to what I am doing. Very much a me problem

For example:

RuntimeError: Failed to import trl.trainer.dpo_trainer because of the following error (look up to see its traceback): cannot import name 'log' from 'torch.distributed.elastic.agent.server.api' (/local_disk0/.ephemeral_nfs/envs/pythonEnv-053435a9-8d99-412c-94e3-03f715fbfa74/lib/python3.10/site-packages/torch/distributed/elastic/agent/server/api.py) File /local_disk0/.ephemeral_nfs/envs/pythonEnv-053435a9-8d99-412c-94e3-03f715fbfa74/lib/python3.10/site-packages/trl/import_utils.py:142, in _LazyModule._get_module(self, module_name) 141 try: --> 142 return importlib.import_module("." + module_name, self.name) 143 except Exception as e: File /usr/lib/python3.10/importlib/init.py:126, in import_module(name, package) 125 level += 1 --> 126 return _bootstrap._gcd_import(name[level:], package, level) File :1050, in _gcd_import(name, package, level) File :1027, in _find_andload(name, import) File :1006, in _find_and_loadunlocked(name, import) File :688, in _load_unlocked(spec) File :883, in exec_module(self, module) File :241, in _call_with_frames_removed(f, *args, **kwds) File /local_disk0/.ephemeral_nfs/envs/pythonEnv-053435a9-8d99-412c-94e3-03f715fbfa74/lib/python3.10/site-packages/trl/trainer/dpo_trainer.py:47 46 from ..models import PreTrainedModelWrapper, create_reference_model ---> 47 from .callbacks import SyncRefModelCallback 48 from .dpo_config import DPOConfig, FDivergenceConstants, FDivergenceType File /local_disk0/.ephemeral_nfs/envs/pythonEnv-053435a9-8d99-412c-94e3-03f715fbfa74/lib/python3.10/site-packages/trl/trainer/callbacks.py:37 35 from transformers.trainer_utils import has_length ---> 37 from ..models.utils import unwrap_model_for_generation 38 from .judges import BasePairwiseJudge File /local_disk0/.ephemeral_nfs/envs/pythonEnv-053435a9-8d99-412c-94e3-03f715fbfa74/lib/python3.10/site-packages/trl/models/utils.py:32 31 if is_deepspeed_available(): ---> 32 import deepspeed 34 if TYPE_CHECKING: File /databricks/python/lib/python3.10/site-packages/deepspeed/init.py:22 21 from . import ops ---> 22 from . import module_inject 24 from .accelerator import get_accelerator File /databricks/python/lib/python3.10/site-packages/deepspeed/module_inject/init.py:6 1 # Copyright (c) Microsoft Corporation. 2 # SPDX-License-Identifier: Apache-2.0 3 4 # DeepSpeed Team ----> 6 from .replace_module import replace_transformer_layer, revert_transformer_layer, ReplaceWithTensorSlicing, GroupQuantizer, generic_injection 7 from .module_quantize import quantize_transformer_layer File /databricks/python/lib/python3.10/site-packages/deepspeed/module_inject/replace_module.py:587 584 return replaced_module --> 587 from ..pipe import PipelineModule 589 import re File /databricks/python/lib/python3.10/site-packages/deepspeed/pipe/init.py:6 1 # Copyright (c) Microsoft Corporation. 2 # SPDX-License-Identifier: Apache-2.0 3 4 # DeepSpeed Team ----> 6 from ..runtime.pipe import PipelineModule, LayerSpec, TiedLayerSpec File /databricks/python/lib/python3.10/site-packages/deepspeed/runtime/pipe/init.py:6 1 # Copyright (c) Microsoft Corporation. 2 # SPDX-License-Identifier: Apache-2.0 3 4 # DeepSpeed Team ----> 6 from .module import PipelineModule, LayerSpec, TiedLayerSpec 7 from .topology import ProcessTopology File /databricks/python/lib/python3.10/site-packages/deepspeed/runtime/pipe/module.py:19 18 from .. import utils as ds_utils ---> 19 from ..activation_checkpointing import checkpointing 20 from .topology import PipeDataParallelTopology, PipelineParallelGrid File /databricks/python/lib/python3.10/site-packages/deepspeed/runtime/activation_checkpointing/checkpointing.py:26 24 from torch import _C ---> 26 from deepspeed.runtime.config import DeepSpeedConfig 27 from deepspeed.utils import logger File /databricks/python/lib/python3.10/site-packages/deepspeed/runtime/config.py:41 39 from ..utils import logger ---> 41 from ..elasticity import ( 42 elasticity_enabled, 43 compute_elastic_config, 44 ensure_immutable_elastic_config, 45 ) 46 from ..elasticity.config import ElasticityConfigError File /databricks/python/lib/python3.10/site-packages/deepspeed/elasticity/init.py:10 9 if is_torch_elastic_compatible(): ---> 10 from .elastic_agent import DSElasticAgent File /databricks/python/lib/python3.10/site-packages/deepspeed/elasticity/elastic_agent.py:9 8 from datetime import datetime ----> 9 from torch.distributed.elastic.agent.server.api import log, _get_socket_with_port 10 from torch.distributed.elastic.metrics import put_metric ImportError: cannot import name 'log' from 'torch.distributed.elastic.agent.server.api' (/local_disk0/.ephemeral_nfs/envs/pythonEnv-053435a9-8d99-412c-94e3-03f715fbfa74/lib/python3.10/site-packages/torch/distributed/elastic/agent/server/api.py)

[danielhanchen]

@rohan-daniscox Oh lord what a cryptic error message yikes