search

sisap-challenges / challenge2023

Repository for handling issues, Q & A, pre-registration, registration, and show partial ranks
0 stars 1 forks source link

Questions: How to submit our solution? #9

Open kondounagi opened 1 year ago

kondounagi commented 1 year ago

team

mti-lab

task

TaskA, TaskB, TaskC

question

Dear @sadit and @maumueller ,

I hope this message finds you well.

I am writing to you on behalf of our team, mti-lab, who is participating in the SISAP'23 challenge. We are eager to compete and have been diligently preparing our solutions for submission.

However, we are unclear about the exact process of solution submission for the competition. We have referred to the information provided on your website and in the competition guidelines, but we are still a little confused about certain details.

Could you kindly provide some guidance on how exactly should we submit our solutions?

We deeply appreciate your assistance and prompt response to our queries as it would help us ensure that our solutions are submitted correctly and on time.

Thank you for organizing this stimulating competition, and for your support in making this a successful experience for all participants.

Best regards, Yusuke Kondo

sadit commented 1 year ago

Dear @kondounagi

We understand your concern. We are working on the precise procedure.

The general idea is that your team should have a working system on GitHub. The system should work with GitHub actions. In your pre-registration issue, you will comment with a precise commit hash and your repo.

We should be able to clone and reproduce your solution (using a Linux setup) using the specified datasets and hyperparameters (all setups should be indicated in your running script). Please check the site's solution examples for a more precise idea. We will follow the steps you specify in the GH actions, and if we find any problems, we will contact you.

In parallel, you must prepare a report in the SISAP conference format using the experimental results you computed in your system and the GitHub actions machine (perhaps only with the 300K subset) using the public query set and the gold standard.

We will use a different query set for running your system. We will rank all systems and share the final rank with all teams before the camera-ready dateline.

Please check the https://sisap-challenges.github.io/ site for checking the important dates and any modifications.

I appreciate your patience Eric

kondounagi commented 1 year ago

Dear @sadit

Thank you for your detailed explanation. Now my understanding is clear. I'll share your expectations with my teammates.

Yusuke