search

google / jsonnet

Jsonnet - The data templating language
http://jsonnet.org
Apache License 2.0
7.02k stars 441 forks source link

Completely unable to install jsonnet using pip install #912

Open zeno17 opened 3 years ago

zeno17 commented 3 years ago

I have tried installing jsonnet given that its a dependency for jiant on Windows 10

when trying to run pip install jsonnet I encounter the following error:

Building wheels for collected packages: jsonnet
  Building wheel for jsonnet (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: 'C:\Users\USER\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-n35v5nsc\\jsonnet_ad2aa1e6f8c6402981889d6e4f24d754\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-n35v5nsc\\jsonnet_ad2aa1e6f8c6402981889d6e4f24d754\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\USER\AppData\Local\Temp\pip-wheel-x8qn1rxp'
       cwd: C:\Users\USER\AppData\Local\Temp\pip-install-n35v5nsc\jsonnet_ad2aa1e6f8c6402981889d6e4f24d754\
  Complete output (4 lines):
  running bdist_wheel
  running build
  running build_ext
  error: [WinError 2] The system cannot find the file specified
  ----------------------------------------
  ERROR: Failed building wheel for jsonnet
  Running setup.py clean for jsonnet
Failed to build jsonnet
Installing collected packages: jsonnet, datasets, bs4, attrs, jiant
    Running setup.py install for jsonnet ... error
    ERROR: Command errored out with exit status 1:
     command: 'C:\Users\USER\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-n35v5nsc\\jsonnet_ad2aa1e6f8c6402981889d6e4f24d754\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-n35v5nsc\\jsonnet_ad2aa1e6f8c6402981889d6e4f24d754\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\USER\AppData\Local\Temp\pip-record-83ne6wxg\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\USER\Anaconda3\Include\jsonnet'
         cwd: C:\Users\USER\AppData\Local\Temp\pip-install-n35v5nsc\jsonnet_ad2aa1e6f8c6402981889d6e4f24d754\
    Complete output (4 lines):
    running install
    running build
    running build_ext
    error: [WinError 2] The system cannot find the file specified
    ----------------------------------------
ERROR: Command errored out with exit status 1: 'C:\Users\USER\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-n35v5nsc\\jsonnet_ad2aa1e6f8c6402981889d6e4f24d754\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-n35v5nsc\\jsonnet_ad2aa1e6f8c6402981889d6e4f24d754\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\USER\AppData\Local\Temp\pip-record-83ne6wxg\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\USER\Anaconda3\Include\jsonnet' Check the logs for full command output.
zeno17 commented 3 years ago

I have tried solutions such as this specific solution from #476.

But that does not seem to work given that I get other errors:


Collecting jiant
  Using cached jiant-2.2.0-py3-none-any.whl (253 kB)
Collecting datasets==1.1.2
  Using cached datasets-1.1.2-py3-none-any.whl (147 kB)
Collecting seqeval==0.0.12
  Using cached seqeval-0.0.12-py3-none-any.whl
Collecting torch>=1.8.1
  Using cached torch-1.8.1-cp38-cp38-win_amd64.whl (190.5 MB)
Requirement already satisfied: lxml==4.6.3 in c:\users\USER\anaconda3\lib\site-packages (from jiant) (4.6.3)
Collecting torchvision==0.9.1
  Using cached torchvision-0.9.1-cp38-cp38-win_amd64.whl (852 kB)
Collecting sacremoses==0.0.43
  Using cached sacremoses-0.0.43-py3-none-any.whl
Collecting tokenizers==0.10.1
  Using cached tokenizers-0.10.1-cp38-cp38-win_amd64.whl (2.0 MB)
Collecting sentencepiece==0.1.91
  Using cached sentencepiece-0.1.91-cp38-cp38-win_amd64.whl (1.2 MB)
Collecting numpy==1.18.4
  Using cached numpy-1.18.4-cp38-cp38-win_amd64.whl (12.8 MB)
Collecting attrs==19.3.0
  Using cached attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Collecting numexpr==2.7.1
  Using cached numexpr-2.7.1-cp38-none-win_amd64.whl (90 kB)
Collecting transformers==4.5.0
  Using cached transformers-4.5.0-py3-none-any.whl (2.1 MB)
Collecting bs4==0.0.1
  Using cached bs4-0.0.1-py3-none-any.whl
Collecting jsonnet==0.15.0
  Using cached jsonnet-0.15.0.tar.gz (255 kB)
Collecting scipy==1.4.1
  Using cached scipy-1.4.1-cp38-cp38-win_amd64.whl (31.0 MB)
Collecting tqdm==4.46.0
  Using cached tqdm-4.46.0-py2.py3-none-any.whl (63 kB)
Collecting pandas==1.0.3
  Using cached pandas-1.0.3-cp38-cp38-win_amd64.whl (8.9 MB)
Requirement already satisfied: nltk>=3.5 in c:\users\USER\anaconda3\lib\site-packages (from jiant) (3.6.1)
Collecting python-Levenshtein==0.12.0
  Using cached python_Levenshtein-0.12.0-cp38-cp38-win_amd64.whl
Collecting scikit-learn==0.22.2.post1
  Using cached scikit_learn-0.22.2.post1-cp38-cp38-win_amd64.whl (6.6 MB)
Requirement already satisfied: beautifulsoup4 in c:\users\USER\anaconda3\lib\site-packages (from bs4==0.0.1->jiant) (4.9.3)
Requirement already satisfied: pyarrow>=0.17.1 in c:\users\USER\appdata\roaming\python\python38\site-packages (from datasets==1.1.2->jiant) (4.0.0)
Requirement already satisfied: xxhash in c:\users\USER\appdata\roaming\python\python38\site-packages (from datasets==1.1.2->jiant) (2.0.2)
Requirement already satisfied: dill in c:\users\USER\appdata\roaming\python\python38\site-packages (from datasets==1.1.2->jiant) (0.3.3)
Requirement already satisfied: multiprocess in c:\users\USER\appdata\roaming\python\python38\site-packages (from datasets==1.1.2->jiant) (0.70.11.1)
Requirement already satisfied: filelock in c:\users\USER\anaconda3\lib\site-packages (from datasets==1.1.2->jiant) (3.0.12)
Requirement already satisfied: requests>=2.19.0 in c:\users\USER\anaconda3\lib\site-packages (from datasets==1.1.2->jiant) (2.25.1)
Requirement already satisfied: pytz>=2017.2 in c:\users\USER\anaconda3\lib\site-packages (from pandas==1.0.3->jiant) (2021.1)
Requirement already satisfied: python-dateutil>=2.6.1 in c:\users\USER\anaconda3\lib\site-packages (from pandas==1.0.3->jiant) (2.8.1)
Requirement already satisfied: setuptools in c:\users\USER\anaconda3\lib\site-packages (from python-Levenshtein==0.12.0->jiant) (52.0.0.post20210125)
Requirement already satisfied: six in c:\users\USER\anaconda3\lib\site-packages (from sacremoses==0.0.43->jiant) (1.15.0)
Requirement already satisfied: click in c:\users\USER\anaconda3\lib\site-packages (from sacremoses==0.0.43->jiant) (7.1.2)
Requirement already satisfied: joblib in c:\users\USER\anaconda3\lib\site-packages (from sacremoses==0.0.43->jiant) (1.0.1)
Requirement already satisfied: regex in c:\users\USER\anaconda3\lib\site-packages (from sacremoses==0.0.43->jiant) (2021.4.4)
Collecting Keras>=2.2.4
  Using cached Keras-2.4.3-py2.py3-none-any.whl (36 kB)
Requirement already satisfied: pillow>=4.1.1 in c:\users\USER\anaconda3\lib\site-packages (from torchvision==0.9.1->jiant) (8.2.0)
Requirement already satisfied: typing-extensions in c:\users\USER\anaconda3\lib\site-packages (from torch>=1.8.1->jiant) (3.7.4.3)
Requirement already satisfied: packaging in c:\users\USER\anaconda3\lib\site-packages (from transformers==4.5.0->jiant) (20.9)
Requirement already satisfied: h5py in c:\users\USER\anaconda3\lib\site-packages (from Keras>=2.2.4->seqeval==0.0.12->jiant) (2.10.0)
Requirement already satisfied: pyyaml in c:\users\USER\anaconda3\lib\site-packages (from Keras>=2.2.4->seqeval==0.0.12->jiant) (5.4.1)
Requirement already satisfied: idna<3,>=2.5 in c:\users\USER\anaconda3\lib\site-packages (from requests>=2.19.0->datasets==1.1.2->jiant) (2.10)
Requirement already satisfied: chardet<5,>=3.0.2 in c:\users\USER\anaconda3\lib\site-packages (from requests>=2.19.0->datasets==1.1.2->jiant) (4.0.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\USER\anaconda3\lib\site-packages (from requests>=2.19.0->datasets==1.1.2->jiant) (1.26.4)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\USER\anaconda3\lib\site-packages (from requests>=2.19.0->datasets==1.1.2->jiant) (2020.12.5)
Requirement already satisfied: soupsieve>1.2 in c:\users\USER\anaconda3\lib\site-packages (from beautifulsoup4->bs4==0.0.1->jiant) (2.2.1)
Requirement already satisfied: pyparsing>=2.0.2 in c:\users\USER\anaconda3\lib\site-packages (from packaging->transformers==4.5.0->jiant) (2.4.7)
WARNING: The candidate selected for download or install is a yanked version: 'python-levenshtein' candidate (version 0.12.0 at https://files.pythonhosted.org/packages/42/a9/d1785c85ebf9b7dfacd08938dd028209c34a0ea3b1bcdb895208bd40a67d/python-Levenshtein-0.12.0.tar.gz#sha256=033a11de5e3d19ea25c9302d11224e1a1898fe5abd23c61c7c360c25195e3eb1 (from https://pypi.org/simple/python-levenshtein/))
Reason for being yanked: Insecure, upgrade to 0.12.1
Building wheels for collected packages: jsonnet
  Building wheel for jsonnet (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: 'C:\Users\USER\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-we64ywu8\\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-we64ywu8\\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\USER\AppData\Local\Temp\pip-wheel-h7jy5ttk'
       cwd: C:\Users\USER\AppData\Local\Temp\pip-install-we64ywu8\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\
  Complete output (29 lines):
  running bdist_wheel
  running build
  running build_ext
  'grep' is not recognized as an internal or external command,
  operable program or batch file.
  process_begin: CreateProcess(NULL, uname -s, ...) failed.
  make: Makefile:122: pipe: Bad file descriptor
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/desugarer.cpp -o core/desugarer.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/formatter.cpp -o core/formatter.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/libjsonnet.cpp -o core/libjsonnet.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/lexer.cpp -o core/lexer.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/parser.cpp -o core/parser.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/pass.cpp -o core/pass.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/static_analysis.cpp -o core/static_analysis.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/string_utils.cpp -o core/string_utils.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json core/vm.cpp -o core/vm.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 -Ithird_party/json third_party/md5/md5.cpp -o third_party/md5/md5.o
  building '_jsonnet' extension
  creating build
  creating build\temp.win-amd64-3.8
  creating build\temp.win-amd64-3.8\Release
  creating build\temp.win-amd64-3.8\Release\python
  C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Iinclude -Ithird_party/md5 -Ithird_party/json -IC:\Users\USER\Anaconda3\include -IC:\Users\USER\Anaconda3\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt" /Tcpython/_jsonnet.c /Fobuild\temp.win-amd64-3.8\Release\python/_jsonnet.obj
  _jsonnet.c
  python/_jsonnet.c(440): warning C4090: 'function': different 'const' qualifiers
  creating C:\Users\USER\AppData\Local\Temp\pip-install-we64ywu8\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\build\lib.win-amd64-3.8
  C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\USER\Anaconda3\libs /LIBPATH:C:\Users\USER\Anaconda3\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x64" /EXPORT:PyInit__jsonnet build\temp.win-amd64-3.8\Release\python/_jsonnet.obj core/desugarer.o core/formatter.o core/libjsonnet.o core/lexer.o core/parser.o core/pass.o core/static_analysis.o core/string_utils.o core/vm.o third_party/md5/md5.o /OUT:build\lib.win-amd64-3.8\_jsonnet.cp38-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.8\Release\python\_jsonnet.cp38-win_amd64.lib
  desugarer.o : fatal error LNK1143: invalid or corrupt file: no symbol for COMDAT section 0x5
  error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.28.29910\\bin\\HostX86\\x64\\link.exe' failed with exit status 1143
  ----------------------------------------
  ERROR: Failed building wheel for jsonnet
  Running setup.py clean for jsonnet
Failed to build jsonnet
Installing collected packages: numpy, tqdm, scipy, torch, tokenizers, sacremoses, pandas, Keras, transformers, torchvision, seqeval, sentencepiece, scikit-learn, python-Levenshtein, numexpr, jsonnet, datasets, bs4, attrs, jiant
  Attempting uninstall: numpy
    Found existing installation: numpy 1.20.1
    Uninstalling numpy-1.20.1:
      Successfully uninstalled numpy-1.20.1
  Attempting uninstall: tqdm
    Found existing installation: tqdm 4.59.0
    Uninstalling tqdm-4.59.0:
      Successfully uninstalled tqdm-4.59.0
  Attempting uninstall: scipy
    Found existing installation: scipy 1.6.2
    Uninstalling scipy-1.6.2:
      Successfully uninstalled scipy-1.6.2
  Attempting uninstall: pandas
    Found existing installation: pandas 1.2.4
    Uninstalling pandas-1.2.4:
      Successfully uninstalled pandas-1.2.4
  Attempting uninstall: scikit-learn
    Found existing installation: scikit-learn 0.24.1
    Uninstalling scikit-learn-0.24.1:
      Successfully uninstalled scikit-learn-0.24.1
  Attempting uninstall: numexpr
    Found existing installation: numexpr 2.7.3
    Uninstalling numexpr-2.7.3:
      Successfully uninstalled numexpr-2.7.3
    Running setup.py install for jsonnet ... error
    ERROR: Command errored out with exit status 1:
     command: 'C:\Users\USER\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-we64ywu8\\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-we64ywu8\\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\USER\AppData\Local\Temp\pip-record-r5heueh3\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\USER\Anaconda3\Include\jsonnet'
         cwd: C:\Users\USER\AppData\Local\Temp\pip-install-we64ywu8\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\
    Complete output (29 lines):
    running install
    running build
    running build_ext
    'grep' is not recognized as an internal or external command,
    operable program or batch file.
    process_begin: CreateProcess(NULL, uname -s, ...) failed.
    make: Makefile:122: pipe: Bad file descriptor
    make: 'core/desugarer.o' is up to date.
    make: 'core/formatter.o' is up to date.
    make: 'core/libjsonnet.o' is up to date.
    make: 'core/lexer.o' is up to date.
    make: 'core/parser.o' is up to date.
    make: 'core/pass.o' is up to date.
    make: 'core/static_analysis.o' is up to date.
    make: 'core/string_utils.o' is up to date.
    make: 'core/vm.o' is up to date.
    make: 'third_party/md5/md5.o' is up to date.
    building '_jsonnet' extension
    creating build
    creating build\temp.win-amd64-3.8
    creating build\temp.win-amd64-3.8\Release
    creating build\temp.win-amd64-3.8\Release\python
    C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Iinclude -Ithird_party/md5 -Ithird_party/json -IC:\Users\USER\Anaconda3\include -IC:\Users\USER\Anaconda3\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt" /Tcpython/_jsonnet.c /Fobuild\temp.win-amd64-3.8\Release\python/_jsonnet.obj
    _jsonnet.c
    python/_jsonnet.c(440): warning C4090: 'function': different 'const' qualifiers
    creating C:\Users\USER\AppData\Local\Temp\pip-install-we64ywu8\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\build\lib.win-amd64-3.8
    C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\USER\Anaconda3\libs /LIBPATH:C:\Users\USER\Anaconda3\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x64" /EXPORT:PyInit__jsonnet build\temp.win-amd64-3.8\Release\python/_jsonnet.obj core/desugarer.o core/formatter.o core/libjsonnet.o core/lexer.o core/parser.o core/pass.o core/static_analysis.o core/string_utils.o core/vm.o third_party/md5/md5.o /OUT:build\lib.win-amd64-3.8\_jsonnet.cp38-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.8\Release\python\_jsonnet.cp38-win_amd64.lib
    desugarer.o : fatal error LNK1143: invalid or corrupt file: no symbol for COMDAT section 0x5
    error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.28.29910\\bin\\HostX86\\x64\\link.exe' failed with exit status 1143
    ----------------------------------------
ERROR: Command errored out with exit status 1: 'C:\Users\USER\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-we64ywu8\\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-we64ywu8\\jsonnet_cf10a061cf6a4e9e812da8e693e7a90b\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\USER\AppData\Local\Temp\pip-record-r5heueh3\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\USER\Anaconda3\Include\jsonnet' Check the logs for full command output.
zeno17 commented 3 years ago

Any help would be greatly appreciated given that as of now, I cannot use the jiant library in its entirity.

zeno17 commented 3 years ago

I also tried to install jiant and its jsonnet dependency on a computer cluster which is Linux-based which went without any problems, but I would really like to be able to run a local version on my windows machine.

sbarzowski commented 3 years ago

Thank you for the detailed report. Not sure why but it looks like the compilation uses g++, but linking uses MSVC, hence the error. Also we use grep somewhere in the Makefile.

Not sure what's the right way to address this. I'm open to suggestions.

zeno17 commented 3 years ago

Personally I wouldn't know how to fix it. I only ran into this issue because of jiants dependency on jsonnet. I dont have the knowledge about C++/Makefile/Make/etc. to solve this in general.

In terms of trying to install jsonnet so it works as a dependency I tried:

  1. This entire thread: https://github.com/google/jsonnet/issues/476#issuecomment-488656947
  2. Installing jsonnetbin (works but doesnt allow jiant to use it)
  3. Cloning the jsonnetbin repo to a local directory and calling python setup.py install. This makes a cloned version of jsonnet (i get a file called _jsonnet.cp38-win_amd64.pyd) which is actually importable in python itself (e.g. import jsonnet) but is not a valid option to use as a dependency for jiant.

I also raised the issue at jiant itself (theres more detailed error reports there incase you are interested) but so far no cigar.

Do you have any ideas on how I could get either jsonnet or jsonnetbin to work as a dependency for jiant?

sbarzowski commented 3 years ago

Well, the jsonnet package is importable as _jsonnet (with an underscore) unlike jsonnetbin which uses jsonnet. So a hacky quick solution would be to get jsonnetbin, modify the line here, so that the name matches https://github.com/mcovalt/jsonnetbin/blob/master/setup.py#L68 and install it. This should be a drop in replacement.

Then you'll need to install jiant without it complaining about lack of jsonnet package. Probably the easiest way would be to clone it locally and drop the dependency line here: https://github.com/nyu-mll/jiant/blob/master/setup.py#L62. Then you can install it with python setup.py install.

Of course this flow sucks. It may still be worth trying as a temporary workaround and as a way to verity if using jsonnetbin is a viable solution. Once we have that confirmed we can think about fixing it for everyone, so that installing jsonnet from pip works out of the box.

zeno17 commented 3 years ago

Alright, I have tested this and I am able to run jiant by having jsonnetbin installed (does not require alteration) and then making a local copy of jiant and removing indeed the specific dependency line: https://github.com/nyu-mll/jiant/blob/master/setup.py#L62

image

Ive taken your suggestions and made it more step-by-step plan for anyone in the future on how you can get jiant to run on a windows machine. Ill reference it on the jiant issue I made there aswell.

Thanks for your help.

Steps

  1. run pip install jsonnetbin (https://github.com/mcovalt/jsonnetbin)
  2. Clone jiant to a local repo (e.g. C:\Users\YOUR_USERNAME\Documents\Github\jiant ) using Github Desktop or another solution
  3. Open C:\Users\YOUR_USERNAME\Documents\Github\jiant\setup.py with an editor and remove "jsonnet == 0.15.0", from the variable install_required. At time of writing it is this specific line (https://github.com/nyu-mll/jiant/blob/master/setup.py#L62)
  4. Open CMD/Anaconda prompt and change directories to where you stored your jiantclone:cd C:\Users\YOUR_USERNAME\Documents\Github\jiant
  5. Run python setup.py install

This should now allow you to run jiant in a windows machine. (This may break in the future incase jiant will start using a later version of jsonnet which isnt covered by jsonnetbin)

srichandana23 commented 1 year ago

Can someone just the way to run the same in colab as well?

sparkprime commented 1 year ago

You might have better luck depending on this instead: https://pypi.org/project/gojsonnet/

It will still need compiling but as it's based on the Go version of Jsonnet the only C code needed to build is the python adapter.