search

zinggAI / zingg

Scalable identity resolution, entity resolution, data mastering and deduplication using ML
GNU Affero General Public License v3.0
919 stars 110 forks source link

build generic junits #728

Open sonalgoyal opened 7 months ago

sonalgoyal commented 7 months ago

a lot of junits can be generalized so that they can be used for Spark as well as other frameworks. create a list here of junits which use SparkFrame and let us see which ones we can make generic.

gnanaprakash-ravi commented 7 months ago

1.TestSparkFrameBase.java

  1. TestSparkFrame.java
  2. TestBlock.java
  3. TestTrainer.java
  4. TestStopWordsRecommender.java
  5. TestModelDocumenter.java
  6. TestDSUtil.java
  7. TestStopWords.java
sonalgoyal commented 5 days ago

Start with TestSparkFrame. Make it generic. Data should be fed from an array and POJO and you can populate that in common

sonalgoyal commented 1 hour ago

TestTrainer, TestDSUtil you can skip. See if stopwords stuff needs generic impl.