search

pyannote / pyannote-audio

Neural building blocks for speaker diarization: speech activity detection, speaker change detection, overlapped speech detection, speaker embedding
http://pyannote.github.io
MIT License
6.09k stars 762 forks source link

Investigate a CI with GPU Access #478

Closed mogwai closed 3 years ago

mogwai commented 3 years ago

Issue by mogwai Thursday Oct 22, 2020 at 18:16 GMT Originally opened as https://github.com/hbredin/pyannote-audio-v2/issues/39

For testing inference and training it would be nice to check that they are on the correct device. Luckily lightning takes care of this mostly but it would be good to verify for out custom inference code.

This would also mean that we could train a few models and verify results are as expected.

mogwai commented 3 years ago

Comment by hbredin Friday Oct 23, 2020 at 09:06 GMT

Would be nice to to fully re-train all models after each commit. Something like https://cml.dev/

mogwai commented 3 years ago

Comment by mogwai Friday Oct 23, 2020 at 16:33 GMT

Maybe something like https://dvc.org/ would be useful here too

aktech commented 3 years ago

Hi @mogwai I am the creator of Cirun.io, "GPU" and "CI" caught my eye.

FWIW I'll share my two cents. I created a service for problems like these, which is basically running custom machines (including GPUs) in GitHub Actions: https://cirun.io/

It is used in multiple open source projects needing GPU support like the following:

https://github.com/pystatgen/sgkit/ https://github.com/qutip/qutip-cupy

It is fairly simple to setup, all you need is a cloud account (AWS or GCP) and a simple yaml file describing what kind of machines you need and Cirun will spin up ephemeral machines on your cloud for GitHub Actions to run. It's native to GitHub ecosystem, which mean you can see logs/trigger in the Github's interface itself, just like any Github Action run.

Also, note that Cirun is free for Open source projects. (You only pay to your cloud provider for machine usage)

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.