ask some question about main_inference.py

[WISH-WEI]

While running the code，the following problem occurs： from utils import load_data, build_vocab, preview_data, get_batches, load_inference_data ImportError: cannot import name 'load_inference_data' from 'utils'
I don't see the function method in the utils file, how should I modify it?

[WISH-WEI]

Which code is used to generate the file emorynlp_features_roberta_discourse.pkl？

[WISH-WEI]

Hello! Thank you very much for sharing!

I have tried again according to the method you said and found some new problems。 in the  main_inference.py  ops = model.step(batch) This line has an error while running：for relation in dialog["relations"] The keyword of relation is not displayed in the data。 I guess maybe there was an error in the file  main_inference.py  or there is a problem with data  data_train = load_data('data/STAC/train.json', map_relations) data_test = load_inference_data('./erc_data/meld/test.json')

Could you please provide it again  main_inference.py 

I am very sorry for the delay of your time, but if you can help me further answer my doubts, I really appreciate it

[yeegirl]

Did you successfully run main_inference.py?

[circle-hit]

Hello! Thank you very much for sharing! I have tried again according to the method you said and found some new problems。 in the  main_inference.py  ops = model.step(batch) This line has an error while running：for relation in dialog["relations"] The keyword of relation is not displayed in the data。 I guess maybe there was an error in the file  main_inference.py  or there is a problem with data  data_train = load_data('data/STAC/train.json', map_relations) data_test = load_inference_data('./erc_data/meld/test.json') Could you please provide it again  main_inference.py  I am very sorry for the delay of your time, but if you can help me further answer my doubts, I really appreciate it

I am sorry for the delay of my reply. The load_inference_data() function is:

def load_inference_data(filename): print "Loading data:", filename f_in = open(filename) inp = f_in.readline() data = json.loads(inp) num_sent = 0 cnt_multi_parents = 0 for dialog in data: last_speaker = None turn = 0 for edu in dialog["edus"]: edu["text_raw"] = edu["text"] + " " text = edu["text"]

        while text.find("http") >= 0:
            i = text.find("http")
            j = i
            while (j < len(text) and text[j] != ' '): j += 1
            text = text[:i] + " [url] " + text[j + 1:]

        invalid_chars = ["/", "\*", "^", ">", "<", "\$", "\|", "=", "@"]
        for ch in invalid_chars:
            text = re.sub(ch, "", text)
        tokens = []
        cur = 0
        for i in range(len(text)):
            if text[i] in "',?.!()\": ":
                if (cur < i):
                    tokens.append(text[cur:i])
                if text[i] != " ":
                    if len(tokens) == 0 or tokens[-1] != text[i]:
                        tokens.append(text[i])
                cur = i + 1
        if cur < len(text):
            tokens.append(text[cur:])
        tokens = [token.lower() for token in tokens]
        for i, token in enumerate(tokens):
            if re.match("\d+", token): 
                tokens[i] = "[num]"
        edu["tokens"] = tokens

        if edu["speaker"] != last_speaker:
            last_speaker = edu["speaker"]
            turn += 1
        edu["turn"] = turn
return data

And please refer to the DialogueParsing repo for implement main_inference.py. We only replace the inference data for our ERC purpose and all the configuration is same with that in the original DialogueParsing repo.

[WISH-WEI]

Sorry to bother you, but can you provide a complete file for processing erc data into parse? (The file version provided does not correspond, or part of the code is missing) I have tried many times without success, if you can give me a hand, I would be very grateful！

[circle-hit]

Sorry to bother you, but can you provide a complete file for processing erc data into parse? (The file version provided does not correspond, or part of the code is missing) I have tried many times without success, if you can give me a hand, I would be very grateful！

Hi, could you describe more details of the problems you are facing now and I would check them correspondingly.

[WISH-WEI]

Thank you for your patience! My current problems are as follows:

1. Tensorflow 1.3, the required environment for DiscourseParsing, cannot be installed. As a result, an error message will be displayed when the program is run on the def _build_embedding function in NonStructured_Encoder. MutableHashTable is not available in versions older than 2.0, making it difficult for the program to continue at this stage
2. How is the test.json data in the data processed? By which function or file?
3. How to judge the accuracy of relations results in parsed documents?