JohnSnowLabs / spark-nlp

State of the Art Natural Language Processing
https://sparknlp.org/
Apache License 2.0
3.77k stars 705 forks source link

Symmetric SpellChecker:Missing default values #1121

Open ibychkov007 opened 3 years ago

ibychkov007 commented 3 years ago

Description

SymmetricDeleteModel.load("//spellcheck_sd_en_2.0.2_2.4_1556604489934") .setInputCols(Array(context.inputField)) .setOutputCol(context.outputField + "_token") .setDupsLimit(2) //error .setFrequencyThreshold(0) .setMaxEditDistance(3) .setDeletesThreshold(0) .setMaxFrequency(10) //error .setMinFrequency(1) //error Unless i set values for setDupsLimit/setMaxFrequency/setMinFrequency it fails with error complaining that default value is missing. For rest of parameters I am using default values I've discovered in source code. How should i set those parameters causing error? SHoudl spark NLP have default values for them?

Expected Behavior

setDupsLimit/setMaxFrequency/setMinFrequency should be used to fine tune default values - SymmetricDeleteModel should work using defauts values for them

Current Behavior

Failures becasue of missinf default values

Possible Solution

Steps to Reproduce

1. 2. 3. 4.

Context

Your Environment

maziyarpanahi commented 3 years ago

Thanks for reporting this, every annotator should work without any parameters. So the default values are missing and they are required. I'll take a look for the next release