search

in-context-learning-2024 / in-context

5 stars 1 forks source link

Rework hybrid model layer definition #79

Open nelson-lojo opened 3 months ago

nelson-lojo commented 3 months ago

The current scheme for adding layers is very repetitive. It would be nice if it could be done entirely from yaml.

Proposal: use some python meta programming magic to auto-import layers from yaml

example:

layers:
  - module: "transformers.Mamba"
    name: "MambaMixer"
    config:
      module: "transformers.Mamba"
      name: "MambaConfig"
    positional_args:
      - "arg1"
      - "arg2
    keyword_args:
      some_value: 4