baptisteroziere
commented
3 years ago Hi. Your issue comes from treesitter, which we use to parse the source code. It seems to be close to this issue, which they fixed on github but I think not on the version we install with pip https://github.com/tree-sitter/py-tree-sitter/issues/43. It seems to affect new versions of python. Although our code probably works with other versions of python, I updated the environment installation script and the CI to use python 3.6.9, which is the one we used to train the models, to avoid this kind of issues in the future. I double checked and tree-sitter works on a clean 3.6.9 installation.
You can either run the install_env script again to install everything with python 3.6.9 or try to install the new version of tree-sitter which probably fixes this issue.
mostafa-saad
I am trying to use the pretrained model to transform from C++ to Java, but with no luck after installations with this error
` python -m codegen_sources.model.translate --src_lang cpp --tgt_lang java --model_path models/TransCoder_model_1.pth --beam_size 1 < zcpp_sample.cpp adding to path <>/workspaces/git_web/facebook_transcoder/CodeGen INFO - 10/02/21 08:26:08 - 0:00:04 - ============ Model Reloading INFO - 10/02/21 08:26:08 - 0:00:04 - Reloading encoder from models/TransCoder_model_1.pth ... WARNING - 10/02/21 08:26:10 - 0:00:06 - Lang cpp_sa matched to pretrained cpp_sa lang embedding. WARNING - 10/02/21 08:26:10 - 0:00:06 - Lang java_sa matched to pretrained java_sa lang embedding. WARNING - 10/02/21 08:26:10 - 0:00:06 - Lang python_sa matched to pretrained python_sa lang embedding. WARNING - 10/02/21 08:26:10 - 0:00:06 - The size of position embeddings in current model is 2048, the size of reloaded is 1024. need to repeat last positions 1024 times. INFO - 10/02/21 08:26:10 - 0:00:07 - Reloading decoders from models/TransCoder_model_1.pth ... WARNING - 10/02/21 08:26:11 - 0:00:08 - Lang cpp_sa matched to pretrained cpp_sa lang embedding. WARNING - 10/02/21 08:26:11 - 0:00:08 - Lang java_sa matched to pretrained java_sa lang embedding. WARNING - 10/02/21 08:26:11 - 0:00:08 - Lang python_sa matched to pretrained python_sa lang embedding. WARNING - 10/02/21 08:26:11 - 0:00:08 - The size of position embeddings in current model is 2048, the size of reloaded is 1024. need to repeat last positions 1024 times. INFO - 10/02/21 08:26:11 - 0:00:08 - Number of parameters (encoder): 143239641 INFO - 10/02/21 08:26:11 - 0:00:08 - Number of parameters (decoders): 168442329 INFO - 10/02/21 08:26:11 - 0:00:08 - Number of decoders: 1
Input cpp function:
include
using namespace std;
int main() { cout<<"My First Program. Helllllllo."<<endl;
} /usr/bin/ld: cannot find -lc++ collect2: error: ld returned 1 exit status Traceback (most recent call last): File "<>/anaconda3/envs/pyt/lib/python3.9/distutils/unixccompiler.py", line 206, in link self.spawn(linker + ld_args) File "<>/anaconda3/envs/pyt/lib/python3.9/distutils/ccompiler.py", line 910, in spawn spawn(cmd, dry_run=self.dry_run) File "<>/anaconda3/envs/pyt/lib/python3.9/distutils/spawn.py", line 91, in spawn raise DistutilsExecError( distutils.errors.DistutilsExecError: command '/usr/bin/cc' failed with exit code 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "<>/anaconda3/envs/pyt/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "<>/anaconda3/envs/pyt/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "<>/workspaces/git_web/facebook_transcoder/CodeGen/codegen_sources/model/translate.py", line 254, in
output = translator.translate(
File "<>/workspaces/git_web/facebook_transcoder/CodeGen/codegen_sources/model/translate.py", line 139, in translate
src_lang_processor = LangProcessor.processors[lang1](
File "<>/workspaces/git_web/facebook_transcoder/CodeGen/codegen_sources/preprocessing/lang_processors/cpp_processor.py", line 25, in init
super().init(
File "<>/workspaces/git_web/facebook_transcoder/CodeGen/codegen_sources/preprocessing/lang_processors/tree_sitter_processor.py", line 40, in init
self.create_treesiter_parser()
File "<>/workspaces/git_web/facebook_transcoder/CodeGen/codegen_sources/preprocessing/lang_processors/tree_sitter_processor.py", line 48, in create_treesiter_parser
Language.build_library(
File "<>/anaconda3/envs/pyt/lib/python3.9/site-packages/tree_sitter/init.py", line 72, in build_library
compiler.link_shared_object(object_paths, output_path)
File "<>/anaconda3/envs/pyt/lib/python3.9/distutils/ccompiler.py", line 713, in link_shared_object
self.link(CCompiler.SHARED_OBJECT, objects,
File "<>/anaconda3/envs/pyt/lib/python3.9/distutils/unixccompiler.py", line 208, in link
raise LinkError(msg)
distutils.errors.LinkError: command '/usr/bin/cc' failed with exit code 1
`
I run the following command in my environment ' cc --version cc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 '