search

ncbi-nlp / bluebert

BlueBERT, pre-trained on PubMed abstracts and clinical notes (MIMIC-III).
https://arxiv.org/abs/1906.05474
Other
558 stars 78 forks source link

Finetuning ELMO. Variable seqlen is not defined #17

Closed caspillaga closed 4 years ago

caspillaga commented 4 years ago

In the file elmoft.py line 114, the variable seqlen is used but it is not previously defined.

class ELMoClfHead(BaseClfHead):
    def __init__(...):
        ...
        if task_type == 'nmt':
            ...
            self.norm = NORM_TYPE_MAP[norm_type](seqlen)

so the code crashes in the line 114: self.norm = NORM_TYPE_MAP[norm_type](seqlen)

caspillaga commented 4 years ago

I hard-coded seqlen to 128, and now that part of the code runs OK (in the default NER experiment). My guess is that seqlen should be replaced by self.maxlen... but I'm not sure because I only inspected the code superficially

cskyan commented 4 years ago

I hard-coded seqlen to 128, and now that part of the code runs OK (in the default NER experiment). My guess is that seqlen should be replaced by self.maxlen... but I'm not sure because I only inspected the code superficially

The bug is fixed. Please pull the newest version. You are right that the code is not consistent. Please also note that it is not recommended to hard-code the maxlen parameter here. You could set the parameter through the parameter task_params={'maxlen':MAXLEN_VALUE} that is suitable for your dataset.