microsoft / vscode-ai-toolkit

MIT License
896 stars 41 forks source link

Cannot create a conda environment with ':' in the prefix. #33

Closed PostArchitekt closed 8 months ago

PostArchitekt commented 9 months ago

Currently keep getting this error when fine-tuning:

Creating or resetting Conda environment from /mnt/c/Users/name/anaconda3/envs/DAFineTune2/setup/conda-environment.yml...
Error: generate-project[0]
   03:52:51.80 0 RunProcess error:
Error: generate-project[0]
   03:52:51.80 0 RunProcess error:CondaValueError: Cannot create a conda environment with ':' in the prefix. Aborting.
Error: generate-project[0]
elsaco commented 9 months ago

@PostArchitekt check your conda-environment.yml for errors, especially the prefix line.

PostArchitekt commented 9 months ago

Looks fine. Am I missing something?

name: phi-2-env
channels:
  - pytorch
  - nvidia
  - defaults
dependencies:
  - pip=23.2.1
  - python=3.9.18
  - cuda=11.8
  - pip:
    - -r requirements.txt
zcobol commented 9 months ago

So you're using phi-2 model. The output you posted is from the first-time-setup.sh in the setup directory. The script is run during project setup:

---cut---
# Todo make the name be dynamic with the project.
# Create or reset environment from conda.yaml located in the script's directory
echo "Creating or resetting Conda environment from $SCRIPT_DIR/conda-environment.yml..."
/opt/miniconda/bin/conda env create -f "$SCRIPT_DIR/conda-environment.yml"
---cut---

After step 3, when a new workspace is open, is the environment listed by conda? It should be at this starge:

conda env list

# conda environments:
#
base                     /opt/miniconda
phi-2-env             *  /opt/miniconda/envs/phi-2-env

Try running line 28 from a bash prompt to (re)create the environment:

/opt/miniconda/bin/conda env create -f "$SCRIPT_DIR/conda-environment.yml"

and use --verbose option for debug output to see what's going on.

I have an active phi-2 model (notice the * in conda env list above) and did not encounter such error.

PostArchitekt commented 9 months ago

STEP 2 Configure the model

Conda environment name matches Project name DAFineTune3. changed dataset to training dataset. Generate Project

Everything goes well until:

Information: WorkspaceAutomation.Catalog.HuggingFaceClientProvider[0]
   10:06:19.77 0 Download for model microsoft/phi-2 completed.
Debug: generate-project[0]
   10:06:19.77 0 RunProcess fileName:wsl arguments:[ --exec bash /mnt/c/Users/Dx/anaconda3/envs/DAFineTune3/setup/first_time_setup.sh] elevated:False
Debug: generate-project[0]
   10:06:45.35 0 RunProcess data:no change     /opt/miniconda/condabin/conda
Debug: generate-project[0]
   10:06:45.35 0 RunProcess data:no change     /opt/miniconda/bin/conda
Debug: generate-project[0]
   10:06:45.35 0 RunProcess data:no change     /opt/miniconda/bin/conda-env
Debug: generate-project[0]
   10:06:45.35 0 RunProcess data:no change     /opt/miniconda/bin/activate
Debug: generate-project[0]
   10:06:45.35 0 RunProcess data:no change     /opt/miniconda/bin/deactivate
Debug: generate-project[0]
   10:06:45.35 0 RunProcess data:no change     /opt/miniconda/etc/profile.d/conda.sh
Debug: generate-project[0]
   10:06:45.35 0 RunProcess data:no change     /opt/miniconda/etc/fish/conf.d/conda.fish
Debug: generate-project[0]
   10:06:45.35 0 RunProcess data:no change     /opt/miniconda/shell/condabin/Conda.psm1
Debug: generate-project[0]
   10:06:45.35 0 RunProcess data:no change     /opt/miniconda/shell/condabin/conda-hook.ps1
Debug: generate-project[0]
   10:06:45.35 0 RunProcess data:no change     /opt/miniconda/lib/python3.11/site-packages/xontrib/conda.xsh
Debug: generate-project[0]
   10:06:45.35 0 RunProcess data:no change     /opt/miniconda/etc/profile.d/conda.csh
Debug: generate-project[0]
   10:06:45.35 0 RunProcess data:no change     /home/dx/.bashrc
Debug: generate-project[0]
   10:06:45.35 0 RunProcess data:No action taken.
Debug: generate-project[0]
   10:06:45.39 0 RunProcess data:Creating or resetting Conda environment from /mnt/c/Users/Dx/anaconda3/envs/DAFineTune3/setup/conda-environment.yml...
Debug: generate-project[0]
   10:06:45.99 0 RunProcess data:Retrieving notices: ...working... done
Error: generate-project[0]
   10:06:46.01 0 RunProcess error:
**Error: generate-project[0]
   10:06:46.01 0 RunProcess error:CondaValueError: Cannot create a conda environment with ':' in the prefix. Aborting.**
Error: generate-project[0]
   10:06:46.01 0 RunProcess error:
Debug: generate-project[0]
   10:06:46.05 0 RunProcess data:Script execution completed.
Debug: generate-project[0]
   10:06:46.06 0 RunProcess exit code:0 elapsed:00:00:26.2828113
Debug: generate-project[0]
   10:06:46.06 0 ExecuteAsync Completed Elapsed:00:00:57.0817591
[2023-12-26T15:06:46.064Z] [INFO] Command executed to generate the project

Launched window into (WSL) workspace and received this notification in VSCode: This workspace is on the Windows file system (/mnt/). For best performance, we recommend moving the workspace to the Linux file system (~/home).

/mnt/c/Users/Dx/anaconda3/envs/DAFineTune3
conda env list
# conda environments:
#
                         /home/dx/anaconda3
                         /home/dx/anaconda3/envs/Seamless
base                  *  /opt/miniconda

cd into setup directory: /opt/miniconda/bin/conda env create -f "conda-environment.yml"

returns the same error: CondaValueError: Cannot create a conda environment with ':' in the prefix. Aborting.

/opt/miniconda/bin/conda env create -f "conda-environment.yml" --verbose CondaValueError: Cannot create a conda environment with ':' in the prefix. Aborting.

With nothing in the debug output

acube3 commented 9 months ago

I just tried the same procedure to create the environment manually and it worked fine for me. As a workaround you can manually create a conda environment and install each of the packages, followed by 'pip -r requirements.txt'