Y-IAB / axolotl

Go ahead and axolotl questions
Apache License 2.0
0 stars 0 forks source link

Integrate Axolotl Training Metrics with Vessl AI #4

Open rifqiyan opened 8 months ago

rifqiyan commented 8 months ago

⚠️ Please check that this feature request hasn't been suggested before.

🔖 Feature description

Axolotl currently only supports W&B to display training metrics. It will be good if we can also send the metrics to VESSL AI so it can be displayed on the UI.

✔️ Solution

Add metrics integration with VESSL AI SDK

❓ Alternatives

No response

📝 Additional Context

No response

Acknowledgements

seungduk-yanolja commented 7 months ago

All PRs are supposed to be merged into the original repository. However, Vessl is not popular and only a few people are using it. Therefore, I propose adding a new config that imports a custom library and runs the given callback on every step. For example, in the YAML config,

callbacks_on_step:
  - iab.vessl

and we declare a library like vessl.py in iab module,

def on_init():
  pass

def on_step(log):
  pass