search

zychen423 / KE-VIST

The code and output of our AAAI paper "Knowledge-Enriched Visual Storytelling"
GNU General Public License v3.0
40 stars 9 forks source link

数据集 #9

Closed Thinking-more closed 2 years ago

Thinking-more commented 2 years ago

您好，请问ROC_replace_coref_gender_mapped_frame_noun.json和ROC_train,ROC_valid,ROC_test有什么区别呢？

zychen423 commented 2 years ago

@joe32140 @ethan5437 還有人記得嗎....

Thinking-more commented 2 years ago

哈哈哈，感谢回复，还有个问题就是在stage3，执行python inference_added_story.py -model save_model_finetuneLDPE_pretrain_vist/trained_accu_55.397.chkpt -device 3 -positional LDPE -term_path ../../stage2/visual_genome_language_model_terms_add_lowest.json时，跑不通该程序，添加参数term_path也会报错，而且我看数据集的处理好像没有用到图片间新增加的term？

zychen423 commented 2 years ago

@RENJZ1026 你可以把跑不過的 error message 貼上來嗎？不過年代實在非常久遠，我也不是很有信心能幫你除錯QQ

joe32140 commented 2 years ago

在 ROC_replace_coref_gender_mapped_frame_noun.json 我們使用 coreference resolution 去把故事中的 pronouns 改成對應的 enitity. For example, "There is a dog. It's running" -> "There is a dog. The dog is running". 可以參考 paper Table 1.

joe32140 commented 2 years ago

https://github.com/zychen423/KE-VIST/tree/4cf3062dbc60731492ee81e5c0cd8491caeacd80/src/stage1/image2term 可以看這邊

Thinking-more commented 2 years ago

https://github.com/zychen423/KE-VIST/tree/4cf3062dbc60731492ee81e5c0cd8491caeacd80/src/stage1/image2term 可以看這邊

这里说的data/preprocess_src好像没有提供耶

joe32140 commented 2 years ago

只能問 @ethan5437 還有沒有權限了

Thinking-more commented 2 years ago

只能問 @ethan5437 還有沒有權限了

好滴，感谢您的帮助！

Thinking-more commented 2 years ago

@RENJZ1026 你可以把跑不過的 error message 貼上來嗎？不過年代實在非常久遠，我也不是很有信心能幫你除錯QQ

这是我遇到的问题： 当我直接执行readme提供的:python inference_added_story.py -model save_model_finetuneLDPE_pretrain_vist/trained_accu_55.397.chkpt -device 3 -positional LDPE -term_path ../../stage2/visual_genome_language_model_terms_add_lowest.json 报错：inference_added_story.py: error: the following arguments are required: -insert, -relation 然后我执行:python inference_added_story.py -model save_model_finetuneLDPE_pretrain_vist/trained_accu_55.397.chkpt -device 3 -positional LDPE -term_path ../../stage2/visual_genome_language_model_terms_add_lowest.json -insert 1 -relation vg 报错：inference_added_story.py: error: unrecognized arguments: -term_path ../../stage2/visual_genome_language_model_terms_add_lowest.json 然后我在inference_added_story.py的parser中加入： parser.add_argument(‘-term_path, type=str)后运行：python inference_added_story.py -model save_model_finetuneLDPE_pretrain_vist/trained_accu_55.397.chkpt -device 3 -positional LDPE -term_path ../../stage2/visual_genome_language_model_terms_add_lowest.json -insert 1 -relation vg 报错：Traceback (most recent call last):
File "inference_added_story.py", line 84, in main() File "inference_added_story.py", line 60, in main all_hyp, all_scores = translator.translate_batch(frame, frame_pos, frame_sen_pos) File "/mnt/ssd/zrj/KE-VIST-master/src/stage3/term2story/transformer/Translator.py", line 210, in translate_batch active_inst_idx_list = beam_decode_step( File "/mnt/ssd/zrj/KE-VIST-master/src/stage3/term2story/transformer/Translator.py", line 165, in beam_decode_step active_inst_idx_list = collect_active_inst_idx_list( File "/mnt/ssd/zrj/KE-VIST-master/src/stage3/term2story/transformer/Translator.py", line 149, in collect_active_inst_idx_list is_inst_complete = inst_beams[inst_idx].advance(word_prob[inst_position]) File "/mnt/ssd/zrj/KE-VIST-master/src/stage3/term2story/transformer/Beam.py", line 94, in advance tmp_pos[i] = self.next_pos[-1][self.prev_ks[-1][i]]+1 IndexError: tensors used as indices must be long, byte or bool tensors 然后我将Beam.py文件的报错位置的type改成torch.long再执行上面的命令，就会报错keyerror

无论您是否还能回答出，我都十分感激您的耐心帮助！

zychen423 commented 2 years ago

@RENJZ1026 我看了看 code 的確跟 README 寫的不一樣，但年代真的太久了我也回想不起來，真是抱歉幫不上忙

Thinking-more commented 2 years ago

@RENJZ1026 我看了看 code 的確跟 README 寫的不一樣，但年代真的太久了我也回想不起來，真是抱歉幫不上忙

没关系，非常感谢您的帮助！我再研究研究。

zychen423 commented 2 years ago

那我把這個 issue 先關了。如果之後還有別的問題，再麻煩你開新的 issue。謝謝

ethan5437 commented 2 years ago

我也沒有權限了 抱歉幫不上忙...

On Fri, Jan 14, 2022 at 5:41 PM Zi-Yuan Chen @.***> wrote:

Closed #9 https://github.com/zychen423/KE-VIST/issues/9.

— Reply to this email directly, view it on GitHub https://github.com/zychen423/KE-VIST/issues/9#event-5893588376, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHYDWQ6KIKFDH2ECC7YXPHDUV7VTHANCNFSM5L3CUT6A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>