Utilise Hetzner for LLM Fine-tuning

[grayJiaaoLi]

User story

1. As a ML engineer
2. I want / need to utilise the two Hetzner machines provided by our industry partner
3. So that we can use them for the fine-tuning of our LLM.

Acceptance criteria

• Get familiar with how to Utilise the Hetzner machines
• Read the information provided by Kubernetes (credentials etc.)
• Get familiar with details of configuration
• Set Up the Environment
  • Access the server
  • Install the drivers, frameworks, dependencies...
  • Make the LLM fine-tuning script runnable on the two Hetzner machines

Further information

• Useful resources:
  • Official documentation
  • Tutorials

Definition of done (DoD)

• Bill of Materials in the planning document has been updated
• All the complex logics have been tested
• All feature branches have been merged and closed
• New feature code has been documented
• Potential new licenses have been checked
• All GitHub Actions are passing
• The requirement.txt is updated

DoD general criteria

• Feature has been fully implemented
• Feature has been merged into the mainline
• All acceptance criteria were met
• Product owner approved features
• All tests are passing
• Developers agreed to release

[grayJiaaoLi]

Considering still no resources on Hetzner, we have get to try it on HPC again for now.

[dominic0df]

The credentials were now provided by our industry partner (see discord chat)

[dominic0df]

already invested in HPC SP02, SP05 is still remaining as we had an issue with the credentials