Encounter error running tutorial code

[ishare]

Encounter error running "tutorials/workshop/Workshop_3_Generative_Model_Training.ipynb" "1. Training the Model" step. All previous steps look good.

searcher = RandomSearch(GenerativeModel, param_ranges, L_train, n=5, model_class_params=model_class_params)
%time gen_model, run_stats = searcher.fit(L_dev, L_gold_dev)

Here's the output:

============================================================
[1] Testing step_size = 1.00e-05, decay = 9.00e-01, epochs = 100, reg_param = 1.00e-03
============================================================
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/miniconda3/envs/snorkel/lib/python3.6/site-packages/snorkel/learning/utils.py in _fit_st(self, X_valid, Y_valid, b, beta, set_unlabeled_as_neg, eval_batch_size)
    401                 model.train(*train_args, X_dev=X_valid, Y_dev=Y_valid, 
--> 402                     save_dir=self.save_dir, **hps)
    403             except:

TypeError: train() got an unexpected keyword argument 'X_dev'

During handling of the above exception, another exception occurred:

IndexError                                Traceback (most recent call last)
<timed exec> in <module>

~/miniconda3/envs/snorkel/lib/python3.6/site-packages/snorkel/learning/utils.py in fit(self, X_valid, Y_valid, b, beta, set_unlabeled_as_neg, n_threads, eval_batch_size)
    359             opt_model, run_stats = self._fit_st(X_valid, Y_valid, b=b, 
    360                 beta=beta, set_unlabeled_as_neg=set_unlabeled_as_neg,
--> 361                 eval_batch_size=eval_batch_size)
    362         return opt_model, run_stats
    363 

~/miniconda3/envs/snorkel/lib/python3.6/site-packages/snorkel/learning/utils.py in _fit_st(self, X_valid, Y_valid, b, beta, set_unlabeled_as_neg, eval_batch_size)
    402                     save_dir=self.save_dir, **hps)
    403             except:
--> 404                 model.train(*train_args, **hps)
    405 
    406             # Test the model

~/miniconda3/envs/snorkel/lib/python3.6/site-packages/snorkel/learning/gen_learning.py in train(self, L, deps, LF_acc_prior_weights, LF_acc_prior_weight_default, labels, label_prior_weight, init_deps, init_class_prior, epochs, step_size, decay, reg_param, reg_type, verbose, truncation, burn_in, cardinality, timer, candidate_ranges, threads)
    135 
    136         # Check to make sure matrix is int-valued
--> 137         element_type = type(L[0,0])
    138         # Note: Other simpler forms of this check often don't work; still not
    139         # sure why...

~/miniconda3/envs/snorkel/lib/python3.6/site-packages/snorkel/annotations.py in __getitem__(self, key)
     90 
     91     def __getitem__(self, key):
---> 92         X = super(csr_AnnotationMatrix, self).__getitem__(key)
     93 
     94         # If X is an integer or float value, just return it

~/miniconda3/envs/snorkel/lib/python3.6/site-packages/scipy/sparse/csr.py in __getitem__(self, key)
    287             # [i, j]
    288             if isintlike(col):
--> 289                 return self._get_single_element(row, col)
    290             # [i, 1:2]
    291             elif isinstance(col, slice):

~/miniconda3/envs/snorkel/lib/python3.6/site-packages/scipy/sparse/compressed.py in _get_single_element(self, row, col)
    854         if not (0 <= row < M) or not (0 <= col < N):
    855             raise IndexError("index out of bounds: 0<=%d<%d, 0<=%d<%d" %
--> 856                              (row, M, col, N))
    857 
    858         major_index, minor_index = self._swap((row, col))

IndexError: index out of bounds: 0<=0<22254, 0<=0<0

[skk123123]

I encounter an error in Workshop 3 as well - when I run the same lines of code mentioned in the issue above:

searcher = RandomSearch(GenerativeModel, param_ranges, L_train, n=5, model_class_params=model_class_params)

%time gen_model, run_stats = searcher.fit(L_dev, L_gold_dev)

----- ERROR DETAILS -----

[1] Testing step_size = 1.00e-05, decay = 9.00e-01, epochs = 100, reg_param = 1.00e-03

TypeError Traceback (most recent call last) ~/anaconda2/envs/snorkel/lib/python3.6/site-packages/snorkel/learning/utils.py in _fit_st(self, X_valid, Y_valid, b, beta, set_unlabeled_as_neg, eval_batch_size) 401 model.train(*train_args, X_dev=X_valid, Y_dev=Y_valid, --> 402 save_dir=self.save_dir, **hps) 403 except:

TypeError: train() got an unexpected keyword argument 'X_dev'

During handling of the above exception, another exception occurred:

ValueError Traceback (most recent call last)

in ~/anaconda2/envs/snorkel/lib/python3.6/site-packages/snorkel/learning/utils.py in fit(self, X_valid, Y_valid, b, beta, set_unlabeled_as_neg, n_threads, eval_batch_size) 359 opt_model, run_stats = self._fit_st(X_valid, Y_valid, b=b, 360 beta=beta, set_unlabeled_as_neg=set_unlabeled_as_neg, --> 361 eval_batch_size=eval_batch_size) 362 return opt_model, run_stats 363 ~/anaconda2/envs/snorkel/lib/python3.6/site-packages/snorkel/learning/utils.py in _fit_st(self, X_valid, Y_valid, b, beta, set_unlabeled_as_neg, eval_batch_size) 402 save_dir=self.save_dir, **hps) 403 except: --> 404 model.train(*train_args, **hps) 405 406 # Test the model ~/anaconda2/envs/snorkel/lib/python3.6/site-packages/snorkel/learning/gen_learning.py in train(self, L, deps, LF_acc_prior_weights, LF_acc_prior_weight_default, labels, label_prior_weight, init_deps, init_class_prior, epochs, step_size, decay, reg_param, reg_type, verbose, truncation, burn_in, cardinality, timer, candidate_ranges, threads) 140 if not issubclass(element_type, np.integer): 141 raise ValueError("""Label matrix must have int-type elements, --> 142 but elements have type %s""" % element_type) 143 144 # Automatically infer cardinality ValueError: Label matrix must have int-type elements, but elements have type

[ajratner]

@jason-fries any ideas here?

[paroma]

We're working on a re-design of the code which will address this issue!

[bhancock8]

Snorkel v0.9 was released today with a fresh batch of tutorials that have been thoroughly tested! Give these a try. We recommend starting with the "Getting Started" walkthrough: https://www.snorkel.org/get-started/.