WordNGramJaccardMeasure is case insensitive - Githubissues

dkpro / dkpro-similarity

Word and text similarity measures

https://dkpro.github.io/dkpro-similarity

Other

53 stars 22 forks source link

WordNGramJaccardMeasure is case insensitive #12

Closed nicolaierbs closed 9 years ago

nicolaierbs commented 9 years ago

Original issue 12 created by dkpro on 2013-08-03T09:35:08.000Z:

What steps will reproduce the problem?

I've noticed that WordNGramJaccardMeasure and WordNGramContainmentMeasure (which inherits the behavior) always transform the tokens to lower case.
Just check line 105 in the currently implementation (revision 311).

What is the expected output? What do you see instead?

I expected the application to distinguish between upper and lower case but it didn't.
Alternatively, there could be a parameter to trigger "case insensitive".

What version of the product are you using? On what operating system?

Revision 311 from trunk (version of today).
Operating system doesn't matter.
Please provide any additional information below.

nicolaierbs commented 9 years ago

Comment #1 originally posted by dkpro on 2013-08-03T09:50:47.000Z:

Fixed in recent snapshot by adding a parameter. New default behaviour is not to convert to lower case. This might change the outcome of experiments using the latest snapshot.