awslabs / handwritten-text-recognition-for-apache-mxnet

This repository lets you train neural networks models for performing end-to-end full-page handwriting recognition using the Apache MXNet deep learning frameworks on the IAM Dataset.
Apache License 2.0
488 stars 189 forks source link

get_denoised adding spaces in output so half sentances is not coming in output #17

Open kbrajwani opened 5 years ago

kbrajwani commented 5 years ago

[AM] Torserane TVcom in Bankiner wo Finnnce [D ] T o r e v e n g e T o m i n B a

see am is full sentance but [D ] is cutting down sentance. tell me where can i change the code to get correct output.

jonomon commented 5 years ago

Hi kbrajwani, [AM] is the correct sentence?

kbrajwani commented 5 years ago

AM is not the correct sentence but my concern is not about correct prediction but i want to know why [D] is adding spaces in character. Sometimes [D] is giving correct prediction

jonomon commented 5 years ago

Which code are you running and on what dataset?

kbrajwani commented 5 years ago

Latest code of your repository with pretrained models.

kbrajwani commented 5 years ago

Tesing on iam dataset images.

jonomon commented 5 years ago

are all predictions like this or this particular one?

kbrajwani commented 5 years ago

No, only first and last line of prediction is come like that.

jonomon commented 5 years ago

Did you make any changes? I am do not see the lines you mentioned in https://github.com/awslabs/handwritten-text-recognition-for-apache-mxnet/blob/master/0_handwriting_ocr.ipynb.

kbrajwani commented 5 years ago

Yes i made lots of change in your code, See following code generating output which i am talking about decoded_line_denoiser = get_denoised(line_character_probs, ctc_bs=False) print("[D ]",decoded_line_denoiser)

jonomon commented 5 years ago

This file presents the methods the denoiser was trained. The data was modelled after the noise associated with the previous steps of our model.

Judging from your example output "Torserane TVcom in Bankiner wo Finnnce", the noise seems quite different from our model and the words are not really recognisable. Most likely the pretrained denoiser wont work well.

I found that the better the handwriting recognition, the better the denoiser. It might be beneficial for you to first improve the handwriting (i.e., focus on improving the output [AM]) then work on the denoiser.

Please note that the output for our handwriting recognition is "Can't go lighting bonfites on this bus," and the denoiser only changed bonfites to bonfires.

kbrajwani commented 5 years ago

Screenshot (7)

see in this image print(generator.generate_sequences(inputs, states, sentence)) this line is generating output like this This sentnce has an eror Choice T h i s s e n t e n c e

output last line is putting one space after every character its your notebook file

jonomon commented 4 years ago

Did you edit any of the functions (denoiser.encode, generator.generate_sequences etc)?

devbaseh commented 4 years ago

Same Issue With me, the final prediction has spaces and gives only half of the sentence like this,

"This sentnce has an eror Choice T h i s s e n t e n c e "

I have just changed "ctx_nlp = mx.gpu(3)" to "ctx_nlp = mx.gpu(0) if mx.context.num_gpus() > 0 else mx.cpu()" I couldn't figure out why this happens...

raghav-menon commented 4 years ago

Hi, I am having the same problem too. Does this require more than one GPU to do the job. If I am not wrong ctx_nlp = mx.gpu(3) command indicates that the variable is being assigned to the 4th GPU in case there are 4 GPUs. If the line is left as such it gives out an error telling that the number should be 1 less than the number of GPU devices. Since I have only one, I had assigned it as ctx_nlp = mx.gpu(0). But that in turn cuts out half the sentence adding a space between the alphabets as already mentioned by a few here. Not sure whether it is exactly a GPU problem. Was wondering whether this can be run on a CPU rather !! Any help is appreciated!!

jonomon commented 4 years ago

@raghav-menon It shouldn't matter whether it's on GPU or CPU. Are you using the IAM dataset?

@devbaseh Which notebook are you running?

raghav-menon commented 4 years ago

@raghav-menon It shouldn't matter whether it's on GPU or CPU. Are you using the IAM dataset?

Thank you for the quick reply. I am using the IAM dataset. I have tried using both CPU and GPU and the problem persists. The demoing output only gives partial sentences with a space in between. The system I am using is AWS with 64 GB of RAM and a Tesla K80 attached.

jonomon commented 4 years ago

@ThomasDelteil ?

raghav-menon commented 4 years ago

Would be grateful if you could advise me on how to solve it!! Thanks

ThomasDelteil commented 4 years ago

I'll try to have a look this weekend, have you tried retraining from the denoising notebook?

raghav-menon commented 4 years ago

I'll try to have a look this weekend, have you tried retraining from the denoising notebook?

Thanks Thomas. I have only used the trained model provided and ran the code. Have not tried retraining!!

mahin003 commented 4 years ago

If anybody executed it on Google colab ,please sharethe edited iam_dataset.py it with me , mahinqureship1@gmail.com

yangyingxiang commented 4 years ago

Same issue here, didn't modify any code besides changing gpu from 3 to 0:

"This sentnce has an eror Choice T h i s s e n t e n c e "

mahin003 commented 4 years ago

i'm facing this error ... Can u help me on this ?? i didnt find the problem

On Sun, Oct 4, 2020 at 3:36 AM yangyingxiang notifications@github.com wrote:

Same issue here, didn't modify any code besides changing gpu from 3 to 0:

"This sentnce has an eror Choice T h i s s e n t e n c e "

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/awslabs/handwritten-text-recognition-for-apache-mxnet/issues/17#issuecomment-703167728, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMR2YQOOWDI4QEGW44MPWYTSI6KNHANCNFSM4JEQEEXQ .

jalvathi commented 3 years ago

@jonomon @ThomasDelteil Is there any update on the space issue we are getting in denoiser?? An update from you guys will make my day.

Thanks for this wonderful repository. :+1:

jalvathi commented 3 years ago

@mahin003 @yangyingxiang @raghav-menon @devbaseh @kbrajwani

I had initiated a merge here

I hope, this solves your issues as well. I just got it solved in my code. :)

cc: @jonomon @ThomasDelteil