snorkel-team / snorkel

A system for quickly generating training data with weak supervision
https://snorkel.org
Apache License 2.0
5.81k stars 857 forks source link

Modify test to permit higher cardinality cases for label model #1637

Closed fredsala closed 3 years ago

fredsala commented 3 years ago

Description of proposed changes

The test_label_model_basic and test_label_model_sparse tests are specific to binary class; increasing the cardinality of the problem beyond 2 (which increases the number of parameters) prevents the tests from passing.

This change modifies the tests to use average error (which is invariant to the number of parameters) instead of maximum error (which makes higher-cardinality tests increasingly difficult).

Related issue(s)

Fixes #1631

Test plan

I ran the scenario in #1631 with cardinality 3,4,...,10 and the tests pass.

Checklist

Need help on these? Just ask!

codecov[bot] commented 3 years ago

Codecov Report

Merging #1637 (70d7208) into master (1adccb6) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1637   +/-   ##
=======================================
  Coverage   97.31%   97.31%           
=======================================
  Files          68       68           
  Lines        2157     2157           
  Branches      348      348           
=======================================
  Hits         2099     2099           
  Misses         30       30           
  Partials       28       28           
fredsala commented 3 years ago

Ready for merge.