Running applyrules.py and runrulemine.py

[vishrantsaagar]

Hey, I was actually trying to understand your paper and code because I'm currently in an AI class and it was quite interesting to me to analyze how you went through this process. I am currently stuck as it appears that the only times that running applyrules.py and runrulemine.py works is when I use se14l to find aspect terms, but the required data for the other files seems to be missing. How do you generate the data so that we can get the aspect and opinion terms for all the datasets involved?

[hldai]

Hi, you need to get the corresponding dependency parsing and POS tagging results. I used Stanford NLP.

[vishrantsaagar]

Oh, thanks a lot for the prompt reply. So in your config.py file you have the following code:

SE14L_FILES = {
    'train_sents_file': join(SE14_DIR, 'laptops/laptops_train_sents.json'),
    'train_tok_texts_file': join(SE14_DIR, 'laptops/laptops_train_texts_tok_pos.txt'),
    'train_valid_split_file': join(SE14_DIR, 'laptops/laptops_train_valid_split.txt'),
    'test_sents_file': join(SE14_DIR, 'laptops/laptops_test_sents.json'),
    'test_tok_texts_file': join(SE14_DIR, 'laptops/laptops_test_texts_tok_pos.txt'),
    'train_dep_tags_file': join(SE14_DIR, 'laptops/laptops-train-rule-dep.txt'),
    'train_pos_tags_file': join(SE14_DIR, 'laptops/laptops-train-rule-pos.txt'),
    'word_cnts_file': join(SE14_DIR, 'laptops/word_cnts.txt'),
    'aspect_rule_patterns_file': join(DATA_DIR, 'rule_patterns/se14l_aspect_mined_rule_patterns.txt'),
    'aspect_term_filter_file': join(SE14_DIR, 'laptops/se14l_aspect_term_filter_vocab.txt'),
    'aspect_term_hit_rate_file': join(SE14_DIR, 'laptops/se14l_aspect_term_hit_rate_file.txt')
} 

About half of these files are already present in the GOOGLE DRIVE dataset given but the other half aren't. Are you saying that you used the Stanford CoreNLP separately to generate those files and dragged and dropped them into the folder for use?

[vishrantsaagar]

Could you possibly make the code available to obtain the corresponding dependency parsing and POS tagging results? Also beyond that, after the term_filer_file and hit_rate_file are generated, it requires certain other files which I believe may not be available,

[hldai]

Hi, I see your problems. But since I am working for a company now and don't have much time on weekdays, I will see what I can do this weekend.

[vishrantsaagar]

Oh yes, that would be great, thank you so much! Hope your work is going good as well!

[vishrantsaagar]

Hey, just wanted to send this as a reminder. Don't want to seem demanding. Just wanted to make sure. Really appreciate it.

[hldai]

I've update the java code for generating dependency parsing, pos tagging, and tokenization results.

[vishrantsaagar]

Hey, yeah, thank you so much. I've been messing around with it a little but just can't get it to run. For 1, could you let me know if I have the order of execution correct: Tokenize.java depParse.java runrulemine.py applyrules.py trainrinante.py and then repeat for each dataset available?

If the above is correct. I am stuck on figuring out how to get Stanford NLP to work on the java files given, did you use any arguments? what command did you use on terminal to run the 2 java files to generate the desired output?

[hldai]

This repo already contains the tokenized semeval data, so you don't need to run Tokenize.java. You run DepParse.java to get dependency parsing and pos tagging results, runrulemine.py to mine the rules. For the java code, I used StanforNLP 4.1.0. Before running it, you should change the file paths in the main function and the path to the model file. No arguments needed. If you cannot run the java code. Try google the problem or post the error here.

You run applyrules.py to apply the mined rules, but you probably need to get the Yelp data, or other restaurant review data yourself, since it seems I am not allowed to distribute Yelp data.

[vishrantsaagar]

Oh alright, I understand, I really appreciate the help so far.

[vishrantsaagar]

Hey, I was setting all of this up on IntelliJ because VSCode seemed hopeless with StanfordNLP

The following is my pom.xml file:

I have the DepParse file in the same project

I am running into this error:

It is confusing because I included the models in my pom.xml file.

Do you have any idea what might be going wrong?

The error point is at line 96

[hldai]

Try setting taggerPath to the path of the tagger model file.

[vishrantsaagar]

Where could I find the tagger model file? I downloaded the following folder "stanford-corenlp-4.3.1" from the site and it doesn't seem to have it

[hldai]

It should be in stanford-corenlp-4.3.1-models.jar, jar files are actually zip files. Alternatively, you can try adding this file to class path with -cp when you run the java code.

[vishrantsaagar]

do i need to have the stanford-corenlp-4.3.1-models.jar in the same folder as well? which in this case would be com.vishrant.nlp? because even after moving it there and setting cp using command "javac -cp stanford-corenlp-4.3.1-models.jar DepParse.java" I still get these errors:

and more saying the packages doesn't exist.

I even tried to just run the file by making the taggerPath this:

I directly used the "copy address text" feature to get the right folder and stuff. And then I get the following error:

Really trying it all here

[vishrantsaagar]

Hey, you can disregard the prior messages, but is there any particular reason you picked 150 epochs? did it just give you the most valid f1 score?

[vishrantsaagar]

And the final question: You mention shared, double alt and pre configurations for RINANTE, in the rinante.py code you have 2 places that mention: lstm_l2_src = False lstm_l2_tar = False

Can I get context on which is which? Thank you so much for all your help!

[hldai]

150 epochs: no particular reason, just so that a good f1 score on the validation set can be obtained.

lstm_l2_src and lstm_l2_tar are about whether or not to add l2 regularization to the LSTM parapemters. I believe they are always set to False so you can ignore them. share_lstm is the argument that controls shared and double.