Can't create environment.yml (Windows 10 - Anaconda)

[petem266]

Tried running DeOldify on the latest Winx64 version of Anaconda. However, the environment.yml creation always fails with numerous conflicts concerning incompatible packages. I tried downgrading conda and various other workarounds I found via google. Any ideas as to what might be causing the issue?

---

Solving environment: \ Found conflicts! Looking for incompatible packages. This can take several minutes. Press CTRL-C to abort. Examining conflict for fastai jupyterlab: 80%|████████████████████████████████▊ | 8/10 [00:05<00:01, 1.38it/s]|failed | -

>>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

Traceback (most recent call last):
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda\exceptions.py", line 1079, in __call__
    return func(*args, **kwargs)
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda_env\cli\main.py", line 80, in do_call
    exit_code = getattr(module, func_name)(args, parser)
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda_env\cli\main_create.py", line 111, in execute
    result[installer_type] = installer.install(prefix, pkg_specs, args, env)
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda_env\installers\conda.py", line 32, in install
    prune=getattr(args, 'prune', False), update_modifier=UpdateModifier.FREEZE_INSTALLED)
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda\core\solve.py", line 117, in solve_for_transaction
    should_retry_solve)
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda\core\solve.py", line 158, in solve_for_diff
    force_remove, should_retry_solve)
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda\core\solve.py", line 281, in solve_final_state
    ssc = self._run_sat(ssc)
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda\common\io.py", line 88, in decorated
    return f(*args, **kwds)
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda\core\solve.py", line 808, in _run_sat
    should_retry_solve=ssc.should_retry_solve
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda\common\io.py", line 88, in decorated
    return f(*args, **kwds)
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda\resolve.py", line 1318, in solve
    self.find_conflicts(specs, specs_to_add, history_specs)
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda\resolve.py", line 347, in find_conflicts
    bad_deps = self.build_conflict_map(specs, specs_to_add, history_specs)
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda\resolve.py", line 507, in build_conflict_map
    root, search_node, dep_graph, num_occurances)
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda\resolve.py", line 369, in breadth_first_search_for_dep_graph
    last_spec = MatchSpec.union((path[-1], target_paths[-1][-1]))[0]
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda\models\match_spec.py", line 481, in union
    return cls.merge(match_specs, union=True)
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda\models\match_spec.py", line 475, in merge
    reduce(lambda x, y: x._merge(y, union), group) if len(group) > 1 else group[0]
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda\models\match_spec.py", line 475, in <lambda>
    reduce(lambda x, y: x._merge(y, union), group) if len(group) > 1 else group[0]
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda\models\match_spec.py", line 502, in _merge
    final = this_component.union(that_component)
  File "C:\Users\petem\Anaconda3\lib\site-packages\conda\models\match_spec.py", line 764, in union
    return '|'.join(options)
TypeError: sequence item 0: expected str instance, Channel found

$ C:\Users\petem\Anaconda3\Scripts\conda-env-script.py create -f environment.yml

environment variables: CIO_TEST= CONDA_AUTO_UPDATE_CONDA=false CONDA_DEFAULT_ENV=base CONDA_EXE=C:\Users\petem\Anaconda3\condabin..\Scripts\conda.exe CONDA_EXES="C:\Users\petem\Anaconda3\condabin..\Scripts\conda.exe" CONDA_PREFIX=C:\Users\petem\Anaconda3 CONDA_PROMPT_MODIFIER=(base) CONDA_PYTHON_EXE=C:\Users\petem\Anaconda3\python.exe CONDA_ROOT=C:\Users\petem\Anaconda3 CONDA_SHLVL=1 HOMEPATH=\Users\petem PATH=C:\Users\petem\Anaconda3;C:\Users\petem\Anaconda3\Library\mingw-w64\bi n;C:\Users\petem\Anaconda3\Library\usr\bin;C:\Users\petem\Anaconda3\Li brary\bin;C:\Users\petem\Anaconda3\Scripts;C:\Users\petem\Anaconda3\bi n;C:\Users\petem\Anaconda3;C:\Users\petem\Anaconda3\Library\mingw-w64\ bin;C:\Users\petem\Anaconda3\Library\usr\bin;C:\Users\petem\Anaconda3\ Library\bin;C:\Users\petem\Anaconda3\Scripts;C:\Users\petem\Anaconda3\ bin;C:\Users\petem\Anaconda3\condabin;C:\Windows\system32;C:\Windows;C :\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\ Windows\System32\OpenSSH;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files\Git\cmd;C:\Users\petem\AppData\Local\Microsoft\WindowsApps PSMODULEPATH=C:\Program Files\WindowsPowerShell\Modules;C:\Windows\system32\Windows PowerShell\v1.0\Modules REQUESTS_CA_BUNDLE= SSL_CERT_FILE=

 active environment : base
active env location : C:\Users\petem\Anaconda3
        shell level : 1
   user config file : C:\Users\petem\.condarc

populated config files : C:\Users\petem.condarc conda version : 4.8.2 conda-build version : 3.18.11 python version : 3.7.6.final.0 virtual packages : __cuda=10.1 base environment : C:\Users\petem\Anaconda3 (writable) channel URLs : https://repo.anaconda.com/pkgs/main/win-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/free/win-64 https://repo.anaconda.com/pkgs/free/noarch https://repo.anaconda.com/pkgs/r/win-64 https://repo.anaconda.com/pkgs/r/noarch https://repo.anaconda.com/pkgs/msys2/win-64 https://repo.anaconda.com/pkgs/msys2/noarch package cache : C:\Users\petem\Anaconda3\pkgs C:\Users\petem.conda\pkgs C:\Users\petem\AppData\Local\conda\conda\pkgs envs directories : C:\Users\petem\Anaconda3\envs C:\Users\petem.conda\envs C:\Users\petem\AppData\Local\conda\conda\envs platform : win-64 user-agent : conda/4.8.2 requests/2.22.0 CPython/3.7.6 Windows/10 Windows/10.0.18362 administrator : False netrc file : None offline mode : False

An unexpected error has occurred. Conda has prepared the above report.

Upload successful.

(base) C:\Users\petem\pythonstuff\DeOldify>

[jantic]

Hi @petem266 ! So unfortunately Windows support is one of those things we've drawn a line on and have the stance that it's beyond the scope of what we're willing to support for this project. There's many reasons for that, but one big one is that generally the deep learning ecosystem simply isn't tailored around Windows. Windows is basically neglected. So for us to chase issues stemming from that fact just seems to be not worth it. Granted, this alienates a lot of users and that's unfortunate, but it boils down to a simple question of time management for us. And sanity.

That all being said, I did make an honest effort to look at the error text you posted there. It's certainly not obvious what Anaconda is complaining about there, I can tell you that much.

If you can do this, your best bet by far would be to install a dual boot of Ubuntu 18.04 on your machine and run DeOldify from that. If I remember correctly, when you install Ubuntu it'll give you the option to make it a dual boot with Windows if you already have Windows on the machine. But I'd double check on that before proceeding.

[alexandrevicenzi]

Based on the traceback it seems to me that the bug is in Anaconda itself, maybe you can try to use another Anaconda version or file an issue in their project.

[alexandrevicenzi]

@petem266 take the official example from Anaconda docs and try to reproduce the bug.

[petem266]

Having tested things on an Ubuntu, I am starting to believe that Conda seems to be the issue.

I went back to Windows, downgraded my conda (to 4.6.14). This caused the environment creation to crash at FASTAI / PYTORCH. I edited the environment.yml and moved FASTAI under PIP just to see what happens. Well, this caused the environment to be created without a single error message. The environment was also activated by conda without any issues. However, FASTAI was not installed as it was "missing" when running the jupyter notebook.

Trying to install Fastai 1.0.51 into the environment however produced a "failed to build bottleneck" error and an error about Microsoft Visual C++ 14 missing. So, some progress was made, but new errors have occurred.

CORRECTION. Went back to the original ym.file just using a dowgraded CONDA. It produced an error when creating the environment. ResolvePackageNotFound:

• fastai=1.0.51 -> torchvision

Added PYTORCH into channels in the environment.yml file and now the environment was created successfully.

EDIT #3. After the previous environment creation, I installed the weights into the model directory and ran a test video. Everything runs smoothly in jupyter under Windows 10.

Edit #4. Tested this on my second Win10 laptop. This time I used the conda that came with the Anaconda package (4.7.12), no downgrades or updates to conda were done, however I again edited the environment.yml and added the channel name pytorch. Again, everything works just fine and the environment is created correctly.

[jantic]

Thanks @petem266 ! This is good to know.

[oyvigri]

My god Im glad I stumbled upon this issue. Note to self: Check the "Issues" tab ALOT sooner. Thank you for creating this issue and for continous feedback @petem266.

Im currently trying to setup this on my Windows 10 machine but have come across some of your mentioned errors. You have currently helped me to get past my previous errors atleast.

Could you take me through the process step by step in order to make this work ? (I understand if its a hassle, so no worries if you dont), but atleast please let me know if you wont so Im not wasting my life re-checking this thread every hour from now on.

Ive tried so much now and Im just completelly lost. Think I have 5 different python versions installed and anaconda aswell as miniconda installed.

Uninstalled Everything and started from scratch...

Where Im currently at:

installed Anaconda3-2019.10-Windows-x86_64 setup the following ENV variables

C:\users\xxx\Anaconda3 C:\users\xxx\Anaconda3\Scripts C:\users\xxx\Anaconda3\Library\bin

Copied libcrypto-1_1-x64. libssl-1_1-x64. from C:\users\xxx\Anaconda3\Library\bin to C:\users\xxx\Anaconda3\DLLs

(in order to get rid of the following error Error Caused by SSLError("Can\'t connect to HTTPS URL because the SSL module is not available. )

changed the environment.yml to the following, atleast thats what I figured from your previous comment.

name: deoldify
channels:
- fastai
- conda-forge
- defaults
- pytorch
dependencies:
- python>=3.7.3
- pytorch=1.0.1
- ffmpeg=4.1.1
- tensorboardX=1.6
- youtube-dl>=2019.4.17
- jupyterlab
- pillow=6.2.1
- pip:
  - ffmpeg-python==0.1.17
  - opencv-python>=3.3.0.10
  - wandb
  - fastai=1.0.51

After runing conda env create -f environment.yml Im now stuck with "endless" InvalidArchiveError Example:

InvalidArchiveError("Error with archive C:\\Users\\XXX\\Anaconda3\\pkgs\\prometheus_client-0.7.1-py_0.tar.bz2.  You probably need to delete and re-download or re-create this file.  Message from libarchive was:\n\nFailed to open 'C:\\Users\\XXX\\Anaconda3\\pkgs\\prometheus_client-0.7.1-py_0.tar.bz2'")

I should also mention if I eventually reach the following steps

source activate deoldify
jupyter lab

I understand that "source" is a unix specific command (have also no idea what jupyter is). What are the Windows equivalent commands ?

Any help would be GREATLY!!!! appreciated.

[oyvigri]

seeing this post is marked as "closed", of which I assume noone in this thread will recieve any further notifications, Im trying this -> @ping @petem266

[oyvigri]

Ok so this is how I got it running on Windows 10 Step by step: Should note that I didnt fully got it working because Im not running CUDA graphics card, although I got the application up and running and am currently seeing if theres an option to run on CPU instead.

1. Download Anaconda from here: https://www.anaconda.com/distribution/#download-section. I got it running with Anaconda version (4.7.12) with python version 3.7 installer. During the installation process, you will be presented with two checkboxes. Choose the "recomended one" (the bottom one).

2. Download WGet from here: http://gnuwin32.sourceforge.net/packages/wget.htm (I went for the Complete package, except sources one or with md5Sum b4679ac6f7757b35435ec711c6c8d912)

3. Add PATH Variables to Windows (If you are unsure what Im referring to, just google how to add system path variables to windows) This step is ofc dependant on where you installed your software. In my case it looked like this:

   C:\Users\XXX\AppData\Local\Continuum\anaconda3
   C:\Users\XXX\AppData\Local\Continuum\anaconda3\Scripts
   C:\Users\XXX\AppData\Local\Continuum\anaconda3\Library\bin
   C:\Program Files (x86)\GnuWin32\bin

4. open either CMD / powershell / git bash, in a folder where you would like to download deOldify and write the following: git clone https://github.com/jantic/DeOldify.git DeOldify (If you are unsure what GIT is, download git bash from here: https://git-scm.com/downloads).

5. now write cd DeOldify in cmd.

6. Open environment.yml file inside deOldify folder. change its content to the following:

   name: deoldify
   channels:
   - fastai
   - conda-forge
   - defaults
   - pytorch
   dependencies:
   - python>=3.7.3
   - fastai=1.0.51
   - pytorch=1.0.1
   - ffmpeg=4.1.1
   - tensorboardX=1.6
   - youtube-dl>=2019.4.17
   - jupyterlab
   - pillow=6.2.1
   - pip:
   - ffmpeg-python==0.1.17
   - opencv-python>=3.3.0.10
   - wandb

7. Go back to cmd / gitBash or whatever, and write the following: conda env create -f environment.yml (this takes a long time), and although after a while it looks like nothing is happening, wait some more. it should say when its finished.

8. when previous step is done, write the following in cmd: conda activate deoldify If this step provides with an error like Git isnt inited or similar, open separate cmd window and open in same folder (deOldify folder). Depending on which editor you open, write the following: conda init cmd (if you opened cmd) conda init bash for Git bash, conda init powershell etc. restart all cmds.

9. open new cmd and write jupyter lab this step should take a while and when done should open a webApp running on port localhost port 8888.

10. Open separate cmd and write the following: wget https://www.dropbox.com/s/mwjep3vyqk5mkjc/ColorizeStable_gen.pth?dl=0 -O ./models/ColorizeStable_gen.pth --no-check-certificate (This will only work given we provided the C:\Program Files (x86)\GnuWin32\bin to windows PATH) (Should note that I created the folder models inside the deOldify folder beforehand and unchecked the attribute read-only, although unsure if you need to do that yourself)

11. Enter localhost on port 8888 (it should open automatically) and open eg: imagecolorizer.ipynb and go through the steps.

For me this is where it ended because Im not running CUDA although if you have nvidia card this shouldnt be an issue.

[jantic]

@oyvigri Thanks for this very detailed writeup and research. I've been on vacation so I just got around to looking at this now.

This will be a great resource for those power users who really really want to run DeOldify on Windows. That being said, I don't plan on adding changes to support Windows, because that means we'll be promising support on that platform and opening a can of worms with maintenance on a platform that is generally just not well supported in the deep learning community. It's a practical decision more than anything.

Honestly, I secretly prefer Windows myself (heresy!). But not enough to open the flood gates on additional issues and corner cases to hunt down.

[oyvigri]

Hi again Jantic and thank you for your response. I was just wondering about where I can download the imagenet dataset file. Its worth noting I did ultimatelly convert to Linux as Windows wasnt the best OS for this. I did download it a while ago but have been having some issues which resulted me in starting over, and forgot where I downloaded it.

[jantic]

@oyvigri Join the competition (no obligations in doing so), then you can download, here

[oyvigri]

Yes that was the link. Thank you very much. Greatly appreciated. Have a nice day.