Integrate Axolotl Training Metrics with Vessl AI

Y-IAB / axolotl

Go ahead and axolotl questions

Apache License 2.0

0 stars 0 forks source link

Integrate Axolotl Training Metrics with Vessl AI #4

Open rifqiyan opened 8 months ago

rifqiyan commented 8 months ago

⚠️ Please check that this feature request hasn't been suggested before.

[X] I searched previous Ideas in Discussions didn't find any similar feature requests.
[X] I searched previous Issues didn't find any similar feature requests.

🔖 Feature description

Axolotl currently only supports W&B to display training metrics. It will be good if we can also send the metrics to VESSL AI so it can be displayed on the UI.

✔️ Solution

Add metrics integration with VESSL AI SDK

❓ Alternatives

No response

📝 Additional Context

No response

Acknowledgements

[X] My issue title is concise, descriptive, and in title casing.
[X] I have searched the existing issues to make sure this feature has not been requested yet.
[X] I have provided enough information for the maintainers to understand and evaluate this request.

seungduk-yanolja commented 6 months ago

All PRs are supposed to be merged into the original repository. However, Vessl is not popular and only a few people are using it. Therefore, I propose adding a new config that imports a custom library and runs the given callback on every step. For example, in the YAML config,

callbacks_on_step:
  - iab.vessl

and we declare a library like vessl.py in iab module,

def on_init():
  pass

def on_step(log):
  pass