search

nyu-mll / jiant

jiant is an nlp toolkit
https://jiant.info
MIT License
1.64k stars 297 forks source link

Installation of jiant using pip fails #1319

Open zeno17 opened 3 years ago

zeno17 commented 3 years ago

Describe the bug I seem unable to install jiant because of its dependency of jsonnet. running pip install jiant returns the following traceback (following this thread:

(base) C:\WINDOWS\system32>python -m pip install jiant
WARNING: Ignoring invalid distribution -mpy (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -yflakes (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -umpy (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -qdm (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -mpy (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution - (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -mpy (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -yflakes (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -umpy (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -qdm (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -mpy (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution - (c:\programdata\anaconda3\lib\site-packages)
Collecting jiant
  Using cached jiant-2.2.0-py3-none-any.whl (253 kB)
Requirement already satisfied: tqdm==4.46.0 in c:\programdata\anaconda3\lib\site-packages (from jiant) (4.46.0)
Collecting jsonnet==0.15.0
  Using cached jsonnet-0.15.0.tar.gz (255 kB)
Requirement already satisfied: seqeval==0.0.12 in c:\programdata\anaconda3\lib\site-packages (from jiant) (0.0.12)
Collecting bs4==0.0.1
  Using cached bs4-0.0.1-py3-none-any.whl
Requirement already satisfied: sentencepiece==0.1.91 in c:\programdata\anaconda3\lib\site-packages (from jiant) (0.1.91)
Requirement already satisfied: pandas==1.0.3 in c:\programdata\anaconda3\lib\site-packages (from jiant) (1.0.3)
Requirement already satisfied: lxml==4.6.3 in c:\programdata\anaconda3\lib\site-packages (from jiant) (4.6.3)
Requirement already satisfied: python-Levenshtein==0.12.0 in c:\programdata\anaconda3\lib\site-packages (from jiant) (0.12.0)
Requirement already satisfied: numpy==1.18.4 in c:\programdata\anaconda3\lib\site-packages (from jiant) (1.18.4)
Requirement already satisfied: tokenizers==0.10.1 in c:\programdata\anaconda3\lib\site-packages (from jiant) (0.10.1)
Requirement already satisfied: scikit-learn==0.22.2.post1 in c:\programdata\anaconda3\lib\site-packages (from jiant) (0.22.2.post1)
Requirement already satisfied: torchvision==0.9.1 in c:\programdata\anaconda3\lib\site-packages (from jiant) (0.9.1+cu102)
Collecting attrs==19.3.0
  Using cached attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Requirement already satisfied: torch>=1.8.1 in c:\programdata\anaconda3\lib\site-packages (from jiant) (1.8.1+cu102)
Requirement already satisfied: transformers==4.5.0 in c:\programdata\anaconda3\lib\site-packages (from jiant) (4.5.0)
Requirement already satisfied: numexpr==2.7.1 in c:\programdata\anaconda3\lib\site-packages (from jiant) (2.7.1)
Collecting datasets==1.1.2
  Using cached datasets-1.1.2-py3-none-any.whl (147 kB)
Requirement already satisfied: nltk>=3.5 in c:\programdata\anaconda3\lib\site-packages (from jiant) (3.6.1)
Requirement already satisfied: scipy==1.4.1 in c:\programdata\anaconda3\lib\site-packages (from jiant) (1.4.1)
Requirement already satisfied: sacremoses==0.0.43 in c:\programdata\anaconda3\lib\site-packages (from jiant) (0.0.43)
Requirement already satisfied: beautifulsoup4 in c:\programdata\anaconda3\lib\site-packages (from bs4==0.0.1->jiant) (4.9.3)
Requirement already satisfied: filelock in c:\programdata\anaconda3\lib\site-packages (from datasets==1.1.2->jiant) (3.0.12)
Requirement already satisfied: dill in c:\USER\appdata\roaming\python\python38\site-packages (from datasets==1.1.2->jiant) (0.3.3)
Requirement already satisfied: multiprocess in c:\users\USER\appdata\roaming\python\python38\site-packages (from datasets==1.1.2->jiant) (0.70.11.1)
Requirement already satisfied: pyarrow>=0.17.1 in c:\users\USER\appdata\roaming\python\python38\site-packages (from datasets==1.1.2->jiant) (4.0.0)
Requirement already satisfied: xxhash in c:\users\USER\appdata\roaming\python\python38\site-packages (from datasets==1.1.2->jiant) (2.0.2)
Requirement already satisfied: requests>=2.19.0 in c:\programdata\anaconda3\lib\site-packages (from datasets==1.1.2->jiant) (2.25.1)
Requirement already satisfied: python-dateutil>=2.6.1 in c:\programdata\anaconda3\lib\site-packages (from pandas==1.0.3->jiant) (2.8.1)
Requirement already satisfied: pytz>=2017.2 in c:\programdata\anaconda3\lib\site-packages (from pandas==1.0.3->jiant) (2021.1)
Requirement already satisfied: setuptools in c:\programdata\anaconda3\lib\site-packages (from python-Levenshtein==0.12.0->jiant) (52.0.0.post20210125)
Requirement already satisfied: click in c:\programdata\anaconda3\lib\site-packages (from sacremoses==0.0.43->jiant) (7.1.2)
Requirement already satisfied: six in c:\programdata\anaconda3\lib\site-packages (from sacremoses==0.0.43->jiant) (1.15.0)
Requirement already satisfied: joblib in c:\programdata\anaconda3\lib\site-packages (from sacremoses==0.0.43->jiant) (1.0.1)
Requirement already satisfied: regex in c:\programdata\anaconda3\lib\site-packages (from sacremoses==0.0.43->jiant) (2021.4.4)
Requirement already satisfied: Keras>=2.2.4 in c:\programdata\anaconda3\lib\site-packages (from seqeval==0.0.12->jiant) (2.4.3)
Requirement already satisfied: pillow>=4.1.1 in c:\programdata\anaconda3\lib\site-packages (from torchvision==0.9.1->jiant) (8.2.0)
Requirement already satisfied: typing-extensions in c:\programdata\anaconda3\lib\site-packages (from torch>=1.8.1->jiant) (3.7.4.3)
Requirement already satisfied: packaging in c:\programdata\anaconda3\lib\site-packages (from transformers==4.5.0->jiant) (20.9)
Requirement already satisfied: h5py in c:\programdata\anaconda3\lib\site-packages (from Keras>=2.2.4->seqeval==0.0.12->jiant) (2.10.0)
Requirement already satisfied: pyyaml in c:\programdata\anaconda3\lib\site-packages (from Keras>=2.2.4->seqeval==0.0.12->jiant) (5.4.1)
Requirement already satisfied: chardet<5,>=3.0.2 in c:\programdata\anaconda3\lib\site-packages (from requests>=2.19.0->datasets==1.1.2->jiant) (4.0.0)
Requirement already satisfied: idna<3,>=2.5 in c:\programdata\anaconda3\lib\site-packages (from requests>=2.19.0->datasets==1.1.2->jiant) (2.10)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\programdata\anaconda3\lib\site-packages (from requests>=2.19.0->datasets==1.1.2->jiant) (1.26.4)
Requirement already satisfied: certifi>=2017.4.17 in c:\programdata\anaconda3\lib\site-packages (from requests>=2.19.0->datasets==1.1.2->jiant) (2020.12.5)
Requirement already satisfied: soupsieve>1.2 in c:\programdata\anaconda3\lib\site-packages (from beautifulsoup4->bs4==0.0.1->jiant) (2.2.1)
Requirement already satisfied: pyparsing>=2.0.2 in c:\programdata\anaconda3\lib\site-packages (from packaging->transformers==4.5.0->jiant) (2.4.7)
Building wheels for collected packages: jsonnet
  Building wheel for jsonnet (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: 'C:\ProgramData\Anaconda3\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-foxfaclo\\jsonnet_68441aa4070c4932a3725c37615908e5\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-foxfaclo\\jsonnet_68441aa4070c4932a3725c37615908e5\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\USER\AppData\Local\Temp\pip-wheel-e16omxyh'
       cwd: C:\Users\USER\AppData\Local\Temp\pip-install-foxfaclo\jsonnet_68441aa4070c4932a3725c37615908e5\
  Complete output (29 lines):
  running bdist_wheel
  running build
  running build_ext
  'grep' is not recognized as an internal or external command,
  operable program or batch file.
  process_begin: CreateProcess(NULL, uname -s, ...) failed.
  make: Makefile:122: pipe: Bad file descriptor
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/desugarer.cpp -o core/desugarer.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/formatter.cpp -o core/formatter.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/libjsonnet.cpp -o core/libjsonnet.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/lexer.cpp -o core/lexer.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/parser.cpp -o core/parser.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/pass.cpp -o core/pass.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/static_analysis.cpp -o core/static_analysis.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/string_utils.cpp -o core/string_utils.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/vm.cpp -o core/vm.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json third_party/md5/md5.cpp -o third_party/md5/md5.o
  building '_jsonnet' extension
  creating build
  creating build\temp.win-amd64-3.8
  creating build\temp.win-amd64-3.8\Release
  creating build\temp.win-amd64-3.8\Release\python
  C:\MinGW-w64\mingw64\bin\gcc.exe -mdll -O -Wall -DMS_WIN64 -Iinclude -Ithird_party/md5 -Ithird_party/json -IC:\ProgramData\Anaconda3\include -IC:\ProgramData\Anaconda3\include -c python/_jsonnet.c -o build\temp.win-amd64-3.8\Release\python\_jsonnet.o
  writing build\temp.win-amd64-3.8\Release\python\_jsonnet.cp38-win_amd64.def
  creating build\lib.win-amd64-3.8
  C:\MinGW-w64\mingw64\bin\g++.exe -shared -s build\temp.win-amd64-3.8\Release\python\_jsonnet.o core/desugarer.o core/formatter.o core/libjsonnet.o core/lexer.o core/parser.o core/pass.o core/static_analysis.o core/string_utils.o core/vm.o third_party/md5/md5.o build\temp.win-amd64-3.8\Release\python\_jsonnet.cp38-win_amd64.def -LC:\ProgramData\Anaconda3\libs -LC:\ProgramData\Anaconda3\PCbuild\amd64 -lpython38 -lmsvcr140 -o build\lib.win-amd64-3.8\_jsonnet.cp38-win_amd64.pyd
  C:/MinGW-w64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lmsvcr140
  collect2.exe: error: ld returned 1 exit status
  error: command 'C:\\MinGW-w64\\mingw64\\bin\\g++.exe' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for jsonnet
  Running setup.py clean for jsonnet
Failed to build jsonnet
WARNING: Ignoring invalid distribution -mpy (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -yflakes (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -umpy (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -qdm (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -mpy (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution - (c:\programdata\anaconda3\lib\site-packages)
Installing collected packages: jsonnet, datasets, bs4, attrs, jiant
    Running setup.py install for jsonnet ... error
    ERROR: Command errored out with exit status 1:
     command: 'C:\ProgramData\Anaconda3\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-foxfaclo\\jsonnet_68441aa4070c4932a3725c37615908e5\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-foxfaclo\\jsonnet_68441aa4070c4932a3725c37615908e5\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\USER\AppData\Local\Temp\pip-record-x82oidi9\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\ProgramData\Anaconda3\Include\jsonnet'
         cwd: C:\Users\USER\AppData\Local\Temp\pip-install-foxfaclo\jsonnet_68441aa4070c4932a3725c37615908e5\
    Complete output (29 lines):
    running install
    running build
    running build_ext
    'grep' is not recognized as an internal or external command,
    operable program or batch file.
    process_begin: CreateProcess(NULL, uname -s, ...) failed.
    make: Makefile:122: pipe: Bad file descriptor
    make: 'core/desugarer.o' is up to date.
    make: 'core/formatter.o' is up to date.
    make: 'core/libjsonnet.o' is up to date.
    make: 'core/lexer.o' is up to date.
    make: 'core/parser.o' is up to date.
    make: 'core/pass.o' is up to date.
    make: 'core/static_analysis.o' is up to date.
    make: 'core/string_utils.o' is up to date.
    make: 'core/vm.o' is up to date.
    make: 'third_party/md5/md5.o' is up to date.
    building '_jsonnet' extension
    creating build
    creating build\temp.win-amd64-3.8
    creating build\temp.win-amd64-3.8\Release
    creating build\temp.win-amd64-3.8\Release\python
    C:\MinGW-w64\mingw64\bin\gcc.exe -mdll -O -Wall -DMS_WIN64 -Iinclude -Ithird_party/md5 -Ithird_party/json -IC:\ProgramData\Anaconda3\include -IC:\ProgramData\Anaconda3\include -c python/_jsonnet.c -o build\temp.win-amd64-3.8\Release\python\_jsonnet.o
    writing build\temp.win-amd64-3.8\Release\python\_jsonnet.cp38-win_amd64.def
    creating build\lib.win-amd64-3.8
    C:\MinGW-w64\mingw64\bin\g++.exe -shared -s build\temp.win-amd64-3.8\Release\python\_jsonnet.o core/desugarer.o core/formatter.o core/libjsonnet.o core/lexer.o core/parser.o core/pass.o core/static_analysis.o core/string_utils.o core/vm.o third_party/md5/md5.o build\temp.win-amd64-3.8\Release\python\_jsonnet.cp38-win_amd64.def -LC:\ProgramData\Anaconda3\libs -LC:\ProgramData\Anaconda3\PCbuild\amd64 -lpython38 -lmsvcr140 -o build\lib.win-amd64-3.8\_jsonnet.cp38-win_amd64.pyd
    C:/MinGW-w64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lmsvcr140
    collect2.exe: error: ld returned 1 exit status
    error: command 'C:\\MinGW-w64\\mingw64\\bin\\g++.exe' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: 'C:\ProgramData\Anaconda3\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-foxfaclo\\jsonnet_68441aa4070c4932a3725c37615908e5\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-foxfaclo\\jsonnet_68441aa4070c4932a3725c37615908e5\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\USER\AppData\Local\Temp\pip-record-x82oidi9\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\ProgramData\Anaconda3\Include\jsonnet' Check the logs for full command output.
WARNING: Ignoring invalid distribution -mpy (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -yflakes (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -umpy (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -qdm (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -mpy (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution - (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -mpy (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -yflakes (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -umpy (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -qdm (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution -mpy (c:\programdata\anaconda3\lib\site-packages)
WARNING: Ignoring invalid distribution - (c:\programdata\anaconda3\lib\site-packages)

To Reproduce

Im using Windows 10, latest Anaconda version 2.0.3 with Python 3.8.8 Before this i repeatedly was getting errors as in the linked thread (google/jsonnet#476) Additional context I have tried most solutions that are available and the newer ones basically boil down to "dont use Windows". I actually succesfully installed jiant on a Linux GPU-instance but I would really like to do some local testing prior to pushing it to the computing cluster.

zeno17 commented 3 years ago

Additional information: I wiped my anaconda installation, set it up as new and it still fails with the following error:


Collecting jiant
  Using cached jiant-2.2.0-py3-none-any.whl (253 kB)
Collecting datasets==1.1.2
  Using cached datasets-1.1.2-py3-none-any.whl (147 kB)
Collecting seqeval==0.0.12
  Using cached seqeval-0.0.12-py3-none-any.whl
Collecting torch>=1.8.1
  Using cached torch-1.8.1-cp38-cp38-win_amd64.whl (190.5 MB)
Requirement already satisfied: lxml==4.6.3 in c:\users\USER\anaconda3\lib\site-packages (from jiant) (4.6.3)
Collecting torchvision==0.9.1
  Using cached torchvision-0.9.1-cp38-cp38-win_amd64.whl (852 kB)
Collecting sacremoses==0.0.43
  Using cached sacremoses-0.0.43-py3-none-any.whl
Collecting tokenizers==0.10.1
  Using cached tokenizers-0.10.1-cp38-cp38-win_amd64.whl (2.0 MB)
Collecting sentencepiece==0.1.91
  Using cached sentencepiece-0.1.91-cp38-cp38-win_amd64.whl (1.2 MB)
Collecting numpy==1.18.4
  Using cached numpy-1.18.4-cp38-cp38-win_amd64.whl (12.8 MB)
Collecting attrs==19.3.0
  Using cached attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Collecting numexpr==2.7.1
  Using cached numexpr-2.7.1-cp38-none-win_amd64.whl (90 kB)
Collecting transformers==4.5.0
  Using cached transformers-4.5.0-py3-none-any.whl (2.1 MB)
Collecting bs4==0.0.1
  Using cached bs4-0.0.1-py3-none-any.whl
Collecting jsonnet==0.15.0
  Using cached jsonnet-0.15.0.tar.gz (255 kB)
Collecting scipy==1.4.1
  Using cached scipy-1.4.1-cp38-cp38-win_amd64.whl (31.0 MB)
Collecting tqdm==4.46.0
  Using cached tqdm-4.46.0-py2.py3-none-any.whl (63 kB)
Collecting pandas==1.0.3
  Using cached pandas-1.0.3-cp38-cp38-win_amd64.whl (8.9 MB)
Requirement already satisfied: nltk>=3.5 in c:\users\USER\anaconda3\lib\site-packages (from jiant) (3.6.1)
Collecting python-Levenshtein==0.12.0
  Using cached python_Levenshtein-0.12.0-cp38-cp38-win_amd64.whl
Collecting scikit-learn==0.22.2.post1
  Using cached scikit_learn-0.22.2.post1-cp38-cp38-win_amd64.whl (6.6 MB)
Requirement already satisfied: beautifulsoup4 in c:\users\USER\anaconda3\lib\site-packages (from bs4==0.0.1->jiant) (4.9.3)
Requirement already satisfied: pyarrow>=0.17.1 in c:\users\USER\appdata\roaming\python\python38\site-packages (from datasets==1.1.2->jiant) (4.0.0)
Requirement already satisfied: xxhash in c:\users\USER\appdata\roaming\python\python38\site-packages (from datasets==1.1.2->jiant) (2.0.2)
Requirement already satisfied: dill in c:\users\USER\appdata\roaming\python\python38\site-packages (from datasets==1.1.2->jiant) (0.3.3)
Requirement already satisfied: multiprocess in c:\users\USER\appdata\roaming\python\python38\site-packages (from datasets==1.1.2->jiant) (0.70.11.1)
Requirement already satisfied: filelock in c:\users\USER\anaconda3\lib\site-packages (from datasets==1.1.2->jiant) (3.0.12)
Requirement already satisfied: requests>=2.19.0 in c:\users\USER\anaconda3\lib\site-packages (from datasets==1.1.2->jiant) (2.25.1)
Requirement already satisfied: pytz>=2017.2 in c:\users\USER\anaconda3\lib\site-packages (from pandas==1.0.3->jiant) (2021.1)
Requirement already satisfied: python-dateutil>=2.6.1 in c:\users\USER\anaconda3\lib\site-packages (from pandas==1.0.3->jiant) (2.8.1)
Requirement already satisfied: setuptools in c:\users\USER\anaconda3\lib\site-packages (from python-Levenshtein==0.12.0->jiant) (52.0.0.post20210125)
Requirement already satisfied: six in c:\users\USER\anaconda3\lib\site-packages (from sacremoses==0.0.43->jiant) (1.15.0)
Requirement already satisfied: click in c:\users\USER\anaconda3\lib\site-packages (from sacremoses==0.0.43->jiant) (7.1.2)
Requirement already satisfied: joblib in c:\users\USER\anaconda3\lib\site-packages (from sacremoses==0.0.43->jiant) (1.0.1)
Requirement already satisfied: regex in c:\users\USER\anaconda3\lib\site-packages (from sacremoses==0.0.43->jiant) (2021.4.4)
Collecting Keras>=2.2.4
  Using cached Keras-2.4.3-py2.py3-none-any.whl (36 kB)
Requirement already satisfied: pillow>=4.1.1 in c:\users\USER\anaconda3\lib\site-packages (from torchvision==0.9.1->jiant) (8.2.0)
Requirement already satisfied: typing-extensions in c:\users\USER\anaconda3\lib\site-packages (from torch>=1.8.1->jiant) (3.7.4.3)
Requirement already satisfied: packaging in c:\users\USER\anaconda3\lib\site-packages (from transformers==4.5.0->jiant) (20.9)
Requirement already satisfied: h5py in c:\users\USER\anaconda3\lib\site-packages (from Keras>=2.2.4->seqeval==0.0.12->jiant) (2.10.0)
Requirement already satisfied: pyyaml in c:\users\USER\anaconda3\lib\site-packages (from Keras>=2.2.4->seqeval==0.0.12->jiant) (5.4.1)
Requirement already satisfied: idna<3,>=2.5 in c:\users\USER\anaconda3\lib\site-packages (from requests>=2.19.0->datasets==1.1.2->jiant) (2.10)
Requirement already satisfied: chardet<5,>=3.0.2 in c:\users\USER\anaconda3\lib\site-packages (from requests>=2.19.0->datasets==1.1.2->jiant) (4.0.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\USER\anaconda3\lib\site-packages (from requests>=2.19.0->datasets==1.1.2->jiant) (1.26.4)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\USER\anaconda3\lib\site-packages (from requests>=2.19.0->datasets==1.1.2->jiant) (2020.12.5)
Requirement already satisfied: soupsieve>1.2 in c:\users\USER\anaconda3\lib\site-packages (from beautifulsoup4->bs4==0.0.1->jiant) (2.2.1)
Requirement already satisfied: pyparsing>=2.0.2 in c:\users\USER\anaconda3\lib\site-packages (from packaging->transformers==4.5.0->jiant) (2.4.7)
WARNING: The candidate selected for download or install is a yanked version: 'python-levenshtein' candidate (version 0.12.0 at https://files.pythonhosted.org/packages/42/a9/d1785c85ebf9b7dfacd08938dd028209c34a0ea3b1bcdb895208bd40a67d/python-Levenshtein-0.12.0.tar.gz#sha256=033a11de5e3d19ea25c9302d11224e1a1898fe5abd23c61c7c360c25195e3eb1 (from https://pypi.org/simple/python-levenshtein/))
Reason for being yanked: Insecure, upgrade to 0.12.1
Building wheels for collected packages: jsonnet
  Building wheel for jsonnet (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: 'C:\Users\USER\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-we64ywu8\\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-we64ywu8\\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\USER\AppData\Local\Temp\pip-wheel-h7jy5ttk'
       cwd: C:\Users\USER\AppData\Local\Temp\pip-install-we64ywu8\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\
  Complete output (29 lines):
  running bdist_wheel
  running build
  running build_ext
  'grep' is not recognized as an internal or external command,
  operable program or batch file.
  process_begin: CreateProcess(NULL, uname -s, ...) failed.
  make: Makefile:122: pipe: Bad file descriptor
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/desugarer.cpp -o core/desugarer.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/formatter.cpp -o core/formatter.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/libjsonnet.cpp -o core/libjsonnet.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/lexer.cpp -o core/lexer.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/parser.cpp -o core/parser.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/pass.cpp -o core/pass.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/static_analysis.cpp -o core/static_analysis.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/string_utils.cpp -o core/string_utils.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/vm.cpp -o core/vm.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json third_party/md5/md5.cpp -o third_party/md5/md5.o
  building '_jsonnet' extension
  creating build
  creating build\temp.win-amd64-3.8
  creating build\temp.win-amd64-3.8\Release
  creating build\temp.win-amd64-3.8\Release\python
  C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Iinclude -Ithird_party/md5 -Ithird_party/json -IC:\Users\USER\Anaconda3\include -IC:\Users\USER\Anaconda3\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt" /Tcpython/_jsonnet.c /Fobuild\temp.win-amd64-3.8\Release\python/_jsonnet.obj
  _jsonnet.c
  python/_jsonnet.c(440): warning C4090: 'function': different 'const' qualifiers
  creating C:\Users\USER\AppData\Local\Temp\pip-install-we64ywu8\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\build\lib.win-amd64-3.8
  C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\USER\Anaconda3\libs /LIBPATH:C:\Users\USER\Anaconda3\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x64" /EXPORT:PyInit__jsonnet build\temp.win-amd64-3.8\Release\python/_jsonnet.obj core/desugarer.o core/formatter.o core/libjsonnet.o core/lexer.o core/parser.o core/pass.o core/static_analysis.o core/string_utils.o core/vm.o third_party/md5/md5.o /OUT:build\lib.win-amd64-3.8\_jsonnet.cp38-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.8\Release\python\_jsonnet.cp38-win_amd64.lib
  desugarer.o : fatal error LNK1143: invalid or corrupt file: no symbol for COMDAT section 0x5
  error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.28.29910\\bin\\HostX86\\x64\\link.exe' failed with exit status 1143
  ----------------------------------------
  ERROR: Failed building wheel for jsonnet
  Running setup.py clean for jsonnet
Failed to build jsonnet
Installing collected packages: numpy, tqdm, scipy, torch, tokenizers, sacremoses, pandas, Keras, transformers, torchvision, seqeval, sentencepiece, scikit-learn, python-Levenshtein, numexpr, jsonnet, datasets, bs4, attrs, jiant
  Attempting uninstall: numpy
    Found existing installation: numpy 1.20.1
    Uninstalling numpy-1.20.1:
      Successfully uninstalled numpy-1.20.1
  Attempting uninstall: tqdm
    Found existing installation: tqdm 4.59.0
    Uninstalling tqdm-4.59.0:
      Successfully uninstalled tqdm-4.59.0
  Attempting uninstall: scipy
    Found existing installation: scipy 1.6.2
    Uninstalling scipy-1.6.2:
      Successfully uninstalled scipy-1.6.2
  Attempting uninstall: pandas
    Found existing installation: pandas 1.2.4
    Uninstalling pandas-1.2.4:
      Successfully uninstalled pandas-1.2.4
  Attempting uninstall: scikit-learn
    Found existing installation: scikit-learn 0.24.1
    Uninstalling scikit-learn-0.24.1:
      Successfully uninstalled scikit-learn-0.24.1
  Attempting uninstall: numexpr
    Found existing installation: numexpr 2.7.3
    Uninstalling numexpr-2.7.3:
      Successfully uninstalled numexpr-2.7.3
    Running setup.py install for jsonnet ... error
    ERROR: Command errored out with exit status 1:
     command: 'C:\Users\USER\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-we64ywu8\\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-we64ywu8\\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\USER\AppData\Local\Temp\pip-record-r5heueh3\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\USER\Anaconda3\Include\jsonnet'
         cwd: C:\Users\USER\AppData\Local\Temp\pip-install-we64ywu8\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\
    Complete output (29 lines):
    running install
    running build
    running build_ext
    'grep' is not recognized as an internal or external command,
    operable program or batch file.
    process_begin: CreateProcess(NULL, uname -s, ...) failed.
    make: Makefile:122: pipe: Bad file descriptor
    make: 'core/desugarer.o' is up to date.
    make: 'core/formatter.o' is up to date.
    make: 'core/libjsonnet.o' is up to date.
    make: 'core/lexer.o' is up to date.
    make: 'core/parser.o' is up to date.
    make: 'core/pass.o' is up to date.
    make: 'core/static_analysis.o' is up to date.
    make: 'core/string_utils.o' is up to date.
    make: 'core/vm.o' is up to date.
    make: 'third_party/md5/md5.o' is up to date.
    building '_jsonnet' extension
    creating build
    creating build\temp.win-amd64-3.8
    creating build\temp.win-amd64-3.8\Release
    creating build\temp.win-amd64-3.8\Release\python
    C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Iinclude -Ithird_party/md5 -Ithird_party/json -IC:\Users\USER\Anaconda3\include -IC:\Users\USER\Anaconda3\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt" /Tcpython/_jsonnet.c /Fobuild\temp.win-amd64-3.8\Release\python/_jsonnet.obj
    _jsonnet.c
    python/_jsonnet.c(440): warning C4090: 'function': different 'const' qualifiers
    creating C:\Users\USER\AppData\Local\Temp\pip-install-we64ywu8\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\build\lib.win-amd64-3.8
    C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\USER\Anaconda3\libs /LIBPATH:C:\Users\USER\Anaconda3\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x64" /EXPORT:PyInit__jsonnet build\temp.win-amd64-3.8\Release\python/_jsonnet.obj core/desugarer.o core/formatter.o core/libjsonnet.o core/lexer.o core/parser.o core/pass.o core/static_analysis.o core/string_utils.o core/vm.o third_party/md5/md5.o /OUT:build\lib.win-amd64-3.8\_jsonnet.cp38-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.8\Release\python\_jsonnet.cp38-win_amd64.lib
    desugarer.o : fatal error LNK1143: invalid or corrupt file: no symbol for COMDAT section 0x5
    error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.28.29910\\bin\\HostX86\\x64\\link.exe' failed with exit status 1143
    ----------------------------------------
ERROR: Command errored out with exit status 1: 'C:\Users\USER\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-we64ywu8\\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-we64ywu8\\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\USER\AppData\Local\Temp\pip-record-r5heueh3\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\USER\Anaconda3\Include\jsonnet' Check the logs for full command output.
zeno17 commented 3 years ago

If I remove my MinGW installation it cannot compile at all because it seems to be missing some compiler in path.

Building wheels for collected packages: jsonnet
  Building wheel for jsonnet (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: 'C:\Users\USER\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-n35v5nsc\\jsonnet_ad2aa1e6f8c6402981889d6e4f24d754\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-n35v5nsc\\jsonnet_ad2aa1e6f8c6402981889d6e4f24d754\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\USER\AppData\Local\Temp\pip-wheel-x8qn1rxp'
       cwd: C:\Users\USER\AppData\Local\Temp\pip-install-n35v5nsc\jsonnet_ad2aa1e6f8c6402981889d6e4f24d754\
  Complete output (4 lines):
  running bdist_wheel
  running build
  running build_ext
  error: [WinError 2] The system cannot find the file specified
  ----------------------------------------
  ERROR: Failed building wheel for jsonnet
  Running setup.py clean for jsonnet
Failed to build jsonnet
Installing collected packages: jsonnet, datasets, bs4, attrs, jiant
    Running setup.py install for jsonnet ... error
    ERROR: Command errored out with exit status 1:
     command: 'C:\Users\USER\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-n35v5nsc\\jsonnet_ad2aa1e6f8c6402981889d6e4f24d754\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-n35v5nsc\\jsonnet_ad2aa1e6f8c6402981889d6e4f24d754\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\USER\AppData\Local\Temp\pip-record-83ne6wxg\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\USER\Anaconda3\Include\jsonnet'
         cwd: C:\Users\USER\AppData\Local\Temp\pip-install-n35v5nsc\jsonnet_ad2aa1e6f8c6402981889d6e4f24d754\
    Complete output (4 lines):
    running install
    running build
    running build_ext
    error: [WinError 2] The system cannot find the file specified
    ----------------------------------------
ERROR: Command errored out with exit status 1: 'C:\Users\USER\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-n35v5nsc\\jsonnet_ad2aa1e6f8c6402981889d6e4f24d754\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-n35v5nsc\\jsonnet_ad2aa1e6f8c6402981889d6e4f24d754\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\USER\AppData\Local\Temp\pip-record-83ne6wxg\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\USER\Anaconda3\Include\jsonnet' Check the logs for full command output.
zeno17 commented 3 years ago

Any help would be greatly appreciated given that as of now, I cant use the library in its entirity

zeno17 commented 3 years ago

I have not been able to fix the issue itself but there is something you can do to get jiant running on a Windows-machine.

See thread: https://github.com/google/jsonnet/issues/912#issuecomment-845872430

sid-detasecure commented 2 years ago

I fixed it by installing jiant via Commenting out the line that contains jsonnet and then installing via pip3 install -r requirement.txt

Then install jsonnet-binary. This should solve the problem.