search

mlcommons / ck

Collective Knowledge (CK) is an educational community project to learn how to run AI, ML and other emerging workloads in the most efficient and cost-effective way across diverse models, data sets, software and hardware using MLCommons CM (Collective Mind workflow automation framework)
https://cKnowledge.org
Apache License 2.0
602 stars 113 forks source link

Proof of concept journal IPOL #617

Closed jhergon closed 1 year ago

jhergon commented 1 year ago

The source code of ML/AI algorithms for image processing is published in the open journal IPOL. The idea is to be able to convert this type of published code to the CK Framework, so that it can be interoperable, automatable, portable, reproducible and easily incorporated into MLops workflows. This As a proof of concept and the generation of a guide tutorial adapted for this type of publication for future autors

for example: http://www.ipol.im/pub/art/2022/439/ which uses a deep learning algorithm

gfursin commented 1 year ago

Thank you very much, Jose, for opening this ticket and Happy New Year! We are a bit swamped working with the community on MLPerf inference v3.0 submissions but I have added to our ToDo list and want to have a look at it as soon as we stabilize everything for the submission! Will keep in touch!

gfursin commented 1 year ago

Hi Jose, It was great to meet you in person at ENS yesterday and thank you for a cool hackathon. It was cool to see that we can quite quickly reproduce a journal article and provide a unified CK2 (CM) interface to it to run on different machines. I need to clean it up a bit and I will share it with you for further discussions ... Will keep in touch!

gfursin commented 1 year ago

Hi Jose, We are finishing the release of the MLPerf inference v3.0 results (next week) and will resume our community developments. I will commit our CK2/CM workflow for the IPOL paper then and will be happy to resume our discussions. In the meantime, we have just released an open-source platform for optimization and reproducibility challenges that we will be developing as a community effort too. I have added our "challenge" there: https://access.cknowledge.org/playground/?action=challenges&name=reproduce-and-automate-ipol-paper ... Hope everything is fine on your side and will keep in touch!

gfursin commented 1 year ago

Hi Jose,

I have finally managed to finish 2 CM scripts and document examples to reproduce IPOL paper:

Please feel free to check it out and tell me if it works. We can discuss it during our community conf-call on Thursday evening, if you have a chance to join!

Thanks!

gfursin commented 1 year ago

Based on our last conf-call, we have completed this challenge! We will add a tutorial to add CM interface to research projects and papers for artifact evaluation based on this challenge shortly!