Inference from the box?

[Dzz1th]

Hi! Thank you for posting the model. Could you please provide how to make inference from the box? If I understood correctly, model from dropbox has already been fitted, so we should be able to run it, but by the design model require original data and building of optimisers to run

class CorefModel:
    Attributes:
        config (coref.config.Config): the model's configuration,
            see config.toml for the details
        epochs_trained (int): number of epochs the model has been trained for
        trainable (Dict[str, torch.nn.Module]): trainable submodules with their
            names used as keys
        training (bool): used to toggle train/eval modes

    Submodules (in the order of their usage in the pipeline):
        tokenizer (transformers.AutoTokenizer)
        bert (transformers.AutoModel)
        we (WordEncoder)
        rough_scorer (RoughScorer)
        pw (PairwiseEncoder)
        a_scorer (AnaphoricityScorer)
        sp (SpanPredictor)
    """
    def __init__(self,
                 config_path: str,
                 section: str,
                 epochs_trained: int = 0):
        """
        A newly created model is set to evaluation mode.

        Args:
            config_path (str): the path to the toml file with the configuration
            section (str): the selected section of the config file
            epochs_trained (int): the number of epochs finished
                (useful for warm start)
        """
        self.config = CorefModel._load_config(config_path, section)
        self.epochs_trained = epochs_trained
        self._docs: Dict[str, List[Doc]] = {}
        self._build_model()
        self._build_optimizers()
        self._set_training(False)
        self._coref_criterion = CorefLoss(self.config.bce_loss_weight)
        self._span_criterion = torch.nn.CrossEntropyLoss(reduction="sum")

So maybe there is an option to run it without all this stuff?

[vdobrovolskii]

Hi! Thanks for the suggestion! I never thought about that, because I always had the original data. I think you might be good to go if you comment out the self._build_optimizers() bit (you might need to manually initialize self.optimizers and self.schedulers as empty dictionaries). Let me know how it works out for you!

I will fix this later myself in the master branch, thanks again.

[Dzz1th]

@vdobrovolskii I found out following error:

Traceback (most recent call last):
  File "predict.py", line 71, in <module>
    result = model.run(doc)
  File "/home/mlb/Job/nlp_trading/src/coreference_resolution/wl-coref/coref/coref_model.py", line 220, in run
    words, cluster_ids = self.we(doc, self._bertify(doc))
  File "/home/mlb/Job/nlp_trading/src/coreference_resolution/wl-coref/coref/coref_model.py", line 359, in _bertify
    return out[subword_mask_tensor]
TypeError: only integer tensors of a single element can be converted to an index

I printed out subword_mask_tensor and found that it is tensor of bool. So, what should be there to done it correctly? I used sample_input.jsonline as input for model.

[vdobrovolskii]

Hmmm. Are you using the same versions of dependencies as listed in requirements.txt? Boolean masks can be freely used as indices in pytorch, so I am a little confused. Please, make sure that you install the versions from requirements.txt, if it doesn't work, send me the output of your pip freeze and I'll try to reproduce your error.

[Dzz1th]

@vdobrovolskii I use the latest pytorch version(1.9), so the trouble was here:

out, _ = self.bert(
            subwords_batches_tensor,
            attention_mask=torch.tensor(
                attention_mask, device=self.config.device))

In latest pytorch versions self.bert return <class 'transformers.modeling_outputs.BaseModelOutputWithPoolingAndCrossAttentions'>, so you need just to get only out and then to extract 'last_hidden_state' manually. After this code is running successfully.

[mhillebrand]

I will fix this later myself in the master branch, thanks again.

Looking forward to it, @vdobrovolskii. Thanks for your awesome research!

[ShahzebPurelogics]

@vdobrovolskii I use the latest pytorch version(1.9), so the trouble was here:

out, _ = self.bert(
            subwords_batches_tensor,
            attention_mask=torch.tensor(
                attention_mask, device=self.config.device))

In latest pytorch versions self.bert return <class 'transformers.modeling_outputs.BaseModelOutputWithPoolingAndCrossAttentions'>, so you need just to get only out and then to extract 'last_hidden_state' manually. After this code is running successfully.

@Dzz1th I am stuck at this can you please describe it? Thank you

[vdobrovolskii]

@ShahzebPurelogics The output of AutoModel is now different, so try downgrading or using this line of code as a replacement:

out = self.bert(subwords_batches_tensor, attention_mask=torch.tensor(attention_mask, device=self.config.device)).last_hidden_state