search

vered1986 / HypeNET

Integrated path-based and distributional method for hypernymy detection
Other
85 stars 13 forks source link

dynet verson #6

Closed Lee-handsome closed 5 years ago

Lee-handsome commented 5 years ago

Hi,I got some error like this: terminate called after throwing an instance of 'std::invalid_argument' what(): Attempting to define parameters before initializing DyNet. Be sure to call dynet::initialize() before defining your model. Aborted (core dumped) What's your dynet verson?How can I fix it?

vered1986 commented 5 years ago

I'm using DyNet version 1.0 in this repository (in the master branch, not in V1 which uses pycnn).

If you are using it and still have this error, can you find out in which line it happens?

Lee-handsome commented 5 years ago

It works,thank you! By the way,can you offer the code to construct the dataset, especially negative pairs?plz!

vered1986 commented 5 years ago

Great!

The code for constructing the dataset relies on knowledge resources in a specific format built for previous work, so I don't think the scripts would be very helpful. I suggest you use the dataset provided in the repository. If you want to recreate a version of it, you need to:

  1. Extract tuples (subject, relation, object) from knowledge resources (DBPedia, Yago, Wikidata, and WordNet) where relation appears in table 1 in the paper: http://aclweb.org/anthology/P/P16/P16-1226.pdf.

  2. Create negative examples, of two types: (1) other (non-hypernym) relations of the subjects; (2) random negative instances.

  3. Filter out pairs that don't have any paths in the resource, using filter_noun_pairs.py.

  4. Downsample instances such that there are 4 times more negative instances than positive instances.

  5. Split the dataset to train, test, and validation, either randomly or lexically.

zhixiaochuan12 commented 5 years ago

I met this Aborted (core dumped) error too. The content of log is:

Loading the dataset... Done! Initializing word embeddings... Loading path files... Loading the corpus... Done! Pairs without paths: 1549 , all dataset: 20314 Done! Number of lemmas 400001, number of pos tags: 17, number of dependency labels: 60 , number of directions: 6 Creating the network...

There are no other details, so I guess it may be caused by create_computation_graph function in paths_lstm_classifier.py.

@Lee-handsome Could you please tell us how you installed the 1.0 version?

I tried pip install dynet==1.1.0 and manual installation introduced here. I add -b v1 to git clone https://github.com/clab/dynet.git to install v1 version, but both of them failed with lots of errors.

Then I tried not adding -b v1, I installed dynet 2.0.3 successfully, but still met the Aborted (core dumped) error.

vered1986 commented 5 years ago

Yes, core dumps come from DyNet and it's not easy to figure out why. You can follow the solution here to make it actually dump the core file and then look at it to figure out what happened.

With respect to installing an older DyNet version, I suggest asking in their github issues.

And the easiest solution would probably be to use LexNet (our follow up project) which should be compatible with the latest DyNet version, to the best of my knowledge.