Accept submission meta attributes in evalai submit command

[Ram81]

Description

Currently, when a participant makes a submission using evalai-cli they can't pass submission_meta_attributes from cli. The submit command should allow the participant to enter all the submission_meta_attributes fields when making a submission using evalai cli.

Approach

1. Make an API call to EvalAI backend to check if challenge phase has any submission meta attributes
2. If challenge phase has non null submission_meta_attributes ask participant to enter the fields.
3. Add checks for required meta attributes fields
4. Pass the value of submission_meta_attributes when making submission API call

[burnerlee]

@Ram81 could you please assign me this issue. I am just fixing all the issues in open PRs right now. 1 of them is merged. 2 PR are open which I'll fix just now. Otherwise, please assign this to me when I am able to get those merged

[Ram81]

@burnerlee Sure, you can go through the evalai-cli codebase meanwhile and set it up on your local machine.

@RishabhJain2018 can we assign this issue to @burnerlee.

[burnerlee]

@Ram81 @RishabhJain2018 can't log in in the project in terminal using evalai login using username:host, password:password

[burnerlee]

(venv) --- local/evalai-cli ‹master› » evalai login             
username: host
Enter password: 

Unable to log in with provided credentials.

[RishabhJain2018]

Hey @burnerlee , Please change the host to localhost:8000 using this command -- evalai host -sh https://eval.ai

[burnerlee]

yeah! works. Thank you. Should I add this in the readme as well? @RishabhJain2018

[RishabhJain2018]

Yes, would be good to add. Thanks!

[burnerlee]

@RishabhJain2018 @Ram81, I was looking for an API to fetch the submission_meta_attributes of a challenge_phase. I got one(http://localhost:8000/api/challenges/phase/alexander-phase-2021-294c180c81/) but it uses the slug for the challenge_phase rather than the id. So should I make one for this?

[burnerlee]

Also each challenge_phase has submission_meta_attributes and default_submission_meta_attributes. We have to use the submission_meta_attributes right? Just a confirmation

[Ram81]

@burnerlee default_submission_meta_attributes are already being used when a participant makes a submission.

@RishabhJain2018 @Ram81, I was looking for an API to fetch the submission_meta_attributes of a challenge_phase. I got one(http://localhost:8000/api/challenges/phase/alexander-phase-2021-294c180c81/) but it uses the slug for the challenge_phase rather than the id. So should I make one for this?

We already have an API to get challenge phase details using challenge phase id. You can find it here

[burnerlee]

oh! thank you so much. Also, would the default and non-default attributes be different in the backend? or would I have to add a check to remove the duplication of attributes?

[burnerlee]

Also, there are no submission_meta_attributes in any of the sample Challenge Phases stored in the database. Could you show some example JSON Object for submission_meta_attributes which I can save for testing?

[Ram81]

oh! thank you so much. Also, would the default and non-default attributes be different in the backend? or would I have to add a check to remove the duplication of attributes?

They're different you don't have to check for duplicates.

Also, there are no submission_meta_attributes in any of the sample Challenge Phases stored in the database. Could you show some example JSON Object for submission_meta_attributes which I can save for testing?

You can create a challenge to use it for testing and understanding how it works. To create a challenge go to https://github.com/Cloud-CV/EvalAI-starters and clone it. After cloning just do ./run.sh in the repo and you'll get a challenge_config.zip. Upload it to your local evalai by going to create challenge page. Now you'll have a challenge with submission meta attributes and you can check the format from django admin for the newly created challenge phases.

[burnerlee]

oh thanks! this would do

[Ram81]

@burnerlee any update on this task?

[burnerlee]

Yes @Ram81 , already made the PR. @RishabhJain2018 asked for your review on the PR. Waiting for the same

[burnerlee]

315 made a day before yesterday, will now work on cyclic import issue

[Ram81]

@burnerlee left some comments on the PR