search

pooyafayyaz / Fingerspelling-PoseNet

3 stars 0 forks source link

i have 2 new questions #4

Closed wjhuang0113 closed 1 day ago

wjhuang0113 commented 3 weeks ago

Hi, Your suggestion above is useful to me.But I have 2 new questions, please help me!

  1. In train_with_lm.py, the dataset’s "train" and "test" are placed into dataset_train and dataset_test, respectively, while in eval_model_lm.py, the "test" portion of the dataset is placed into dataset_test. I would like to ask whether this is written incorrectly. My understanding is that train_with_lm.py should place the dataset’s "train" and "dev" portions into dataset_train and dataset_dev, respectively.

  2. After modifying loss_token = (cls_token, gt_label_size) to loss_token = loss_cls(cls_token, gt_label_size), I was able to successfully run train_with_lm.py to line 173, where lev_acc = compute_acc(preds, gt_labels), but the program encountered an error at line 67 of utils.py: subs[ord(s[row])-97][ord(t[col])-97] += 1. After tracing the source code, I suspect that the issue lies in line 66 of utils.py, where the condition if s[row] not in [' ','.'] and t[col] not in [' ','.']: may be problematic. This is because the characters that need to be processed include "$' &.@acbedgfihkjmlonqpsrutwvyxz", but the code only accounts for the two non-alphabetic characters ' ' and '.'. The other six characters ,'$', '\'', '&', '@', after being converted using ASCII and subtracting 97, result in values less than 0, which causes the program to throw an error. Therefore, I modified that line to if s[row] not in [' ','.','$', '\'', '&', '@'] and t[col] not in [' ','.','$', '\'', '&', '@']:. I would like to ask if my assumption here is correct.

Thanks.

pooyafayyaz commented 3 weeks ago

Hi,

  1. Correct, you should use the train and dev sets for training and the test set for evaluation. This is how the reported results were obtained, though I may experiment further before releasing the code. So ignore this.

  2. I haven't encountered any issues with that, but I'll check and rerun the code to verify. There may be some package updates that are not compatible with the current version of my code.

wjhuang0113 commented 3 weeks ago

Hi, thank you for your reply.

When I ran your code, I encountered an error at line 251 of ctc_decoder.py: lm_scores = decoder.return_scores(poses, strings, self.char_to_int) After tracing the code, I suspect that return_scores comes from line 154 of model.py: def return_scores(self, poses, strings, vocab_map): I'm not sure if this part is correct. Could you clarify?

wjhuang0113 commented 2 weeks ago

Hi,

I'm currently running into an issue with train_with_lm.py. Each training epoch takes about 30 minutes, but the evaluation phase takes nearly 480 minutes. Could you let me know how long it actually takes in your environment when you tested it?

Thanks!!

pooyafayyaz commented 4 days ago

The testing phase is expected to take longer because it passes all candidates to the decoder. In my case, though, it took around 30 minutes or so.