search

saltudelft / type4py

Type4Py: Deep Similarity Learning-Based Type Inference for Python
Apache License 2.0
61 stars 13 forks source link

Crash when trying to infer single file with freshly trained model using ManyTypes4Py #4

Closed fmease closed 2 years ago

fmease commented 3 years ago

Hello, thank you for creating and providing this great project! I plan to use this project for my bachelor thesis. Therefore, I am mainly interested in the inference functionality provided with infer.py on branch server (branch infer seems to be outdated). I am aware of the VS Code extension and the public JSON API. I, however, prefer to use this project locally.

Since infer.py takes a pre-trained model as a program argument, I followed all the steps in the README to train such a model. Unfortunately, the script crashes with the following message (excerpt):

onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: tok for the following indices
 index: 0 Got: 7 Expected: 1
 Please fix either the inputs or the model.

Below you can find a link to a Google Colab notebook with all the steps from start (downloading the ManyTypes4Py dataset, pip-installing type4py, preprocessing) to finish (training a model, trying to infer the types of a single file) and the corresponding output from when I ran it the last time (including the full error backtrace on the bottom):

https://colab.research.google.com/drive/1kRIffMlgGCeW55wXelksGrXfSd0WjhKQ?usp=sharing

It should be relatively self-explanatory. Evidently, I use a fork of this project and not the project itself. The differences are minor though: In learn.py, I just re-uncommented the .to(DEVICE)-calls (c42144d) as otherwise it would lead to a crash in the notebook (vectors are on different devices). The remaining changes don't affect Python files and are not relevant to this issues. Further, I am using venv, although I doubt this has any negative influence on the execution of this project.

My question is, how can I successfully use infer.py? How can I obtain a proper compatible model for it? Are any of those steps in the linked notebook incorrect?

mir-am commented 3 years ago

Thanks for your interest in Type4Py. I cannot see the Google Colab as it requires permission. I sent a request to gain access. Unfortunately, the infer module lacks documentation, which we plan to add later on.

Looking at the error, something may be wrong with the input. I still need to see the notebook.

fmease commented 3 years ago

Thank you for your reply. I forgot to make the notebook public. It should be now.

mir-am commented 3 years ago

Thank you for your reply. I forgot to make the notebook public. It should be now.

I have managed to reproduce the error locally on our server. I will investigate the issue very soon and keep you updated here. In the meantime, you can try other Python examples or files.

mir-am commented 2 years ago

I am going to close this issue now. Feel free to open it if the issue persists.