Use Pretrained Model

[Ofir408]

Hi, How can I do training from a pre-trained model?

For example, instead of using:

from pyhealth.models import Transformer, RNN, RETAIN

model = Transformer(
    dataset=mimic3_task_ds,
    # look up what are available for "feature_keys" and "label_keys" in dataset.samples[0]
    feature_keys=["conditions", "procedures", "drugs"],
    label_key="label",
    mode="binary",

I would like to use a pre-trained Transformer or pre-trained bert model instead. Is it possible? @pat-jj

[pat-jj]

Hi, we currently do not have pre-trained models, but instead, we have pre-trained embedding of medical code. You can import those pre-trained embeddings by calling pretrained_emb="LM/clinicalbert" for example when you initialize models. Please check this notebook for details: pre-trained embedding tutorial. You can further train this model after importing the embedding.