search

delphi-suite / delphi

small language models training made easy
Apache License 2.0
9 stars 1 forks source link

load config files in order, later overrides earlier, lots more testing #108

Closed jaidhyani closed 6 months ago

jaidhyani commented 6 months ago

The main functional impact of this PR is to kill the "priority" config arg and just load configs in the order in which they're passed in to the training script, with the later overriding the former. Alongside this we do some refactoring to support testing to ensure that everything works as expected. We also improve type-casting override argument strings to better match the actual config types and make it significantly less dangerous.

jettjaniak commented 6 months ago

why the merge commit?

jaidhyani commented 6 months ago

why the merge commit?

Default behavior in the github UI, clicked without thinking. I think we can configure this to default to rebase/ff.

image
jettjaniak commented 6 months ago

https://github.com/orgs/community/discussions/12032 not configurable unfortunately :/

jaidhyani commented 6 months ago
  • filter_config_to_actual_config_values

I think we actually don't need this anymore, going to optimistically delete.

jaidhyani commented 6 months ago
  • in general I'm quite overwhelmed by train/config/utils.py, I didn't realize we'll need all of this to have the functionality I imagined
  • could you add some docstrings? like what are the functions doing and why we need them

I added some more docstrings explaining how some things are used, why we need them, and how they work. I also deleted a few things that we don't need anymore.