xiangking / ark-nlp

A private nlp coding package, which quickly implements the SOTA solutions.
Apache License 2.0
311 stars 64 forks source link

New Feature : Adding Pipeline and refine Introduction of doc. #43

Open Zrealshadow opened 2 years ago

Zrealshadow commented 2 years ago

Description

The example in Introduction docs is too long and complex. Many settings can be wrappered into a default configuration, such as epoch, batchsize, optimizer, etc. users can custom their own configuration through some string options instead of declaring a specific class.

The all process can also be wrappered into a default class (called pipeline in huggingface) This is an example

>>> from transformers import pipeline
>>> unmasker = pipeline('fill-mask', model='bert-base-uncased')
>>> unmasker("Hello I'm a [MASK] model.")

[{'sequence': "[CLS] hello i'm a fashion model. [SEP]",
  'score': 0.1073106899857521,
  'token': 4827,
  'token_str': 'fashion'},
 {'sequence': "[CLS] hello i'm a role model. [SEP]",
  'score': 0.08774490654468536,
  'token': 2535,
  'token_str': 'role'},
 {'sequence': "[CLS] hello i'm a new model. [SEP]",
  'score': 0.05338378623127937,
  'token': 2047,
  'token_str': 'new'},
 {'sequence': "[CLS] hello i'm a super model. [SEP]",
  'score': 0.04667217284440994,
  'token': 3565,
  'token_str': 'super'},
 {'sequence': "[CLS] hello i'm a fine model. [SEP]",
  'score': 0.027095865458250046,
  'token': 2986,
  'token_str': 'fine'}]

By this way, we can refine README docs, make it clear and easy-understanding. As for some users who want to further custom their model, we can provide more complex example scripts under test directory.