Codabench submissions: parameters.json

[MaximeLee]

If I want to only submit a custom model and no augmented simulator,

what would be the arguments related to the simulator in the parameters.json file (and the config.ini file in consequence)?

Is there a more complete documentation where every possible arguments for each keyword in the parameters.json file can be found?

[Mleyliabadi]

I don't understand what you would refer by a custom model and no augmented simulator ?! All the implementations (custom or not) required for the submission in the challenge are somehow augmented simulators, as we want to improve the performance (computation time) of the existing physical simulator.

However, if you refer to the third solution of participation explained in the fourth notebook. The procedure of submission remains the same. We are working to provide various submission examples with different configurations and to explain parameters.json with respect to each example.

We are going also to organize a webinar by explaining in more details the submission procedure. An email will be sent soon to all the participants very soon.

[MaximeLee]

I don't understand what you would refer by a custom model and no augmented simulator ?! All the implementations (custom or not) required for the submission in the challenge are somehow augmented simulators, as we want to improve the performance (computation time) of the existing physical simulator.

The augmented simulator class has model(Torch) and _model(TF) attributes. I was thinking that in case I only provide a custom model, the augmented simulator class isnt necessary if the evaluation is done with the AirfRANSEvaluation function.

However, if you refer to the third solution of participation explained in the fourth notebook. The procedure of submission remains the same. We are working to provide various submission examples with different configurations and to explain parameters.json with respect to each example.

Yes this is the situation I am refering to.

[jpbrunetsx]

Hi,

We have now implemented new examples that should cover this use case : https://github.com/IRT-SystemX/ml4physim_startingkit/blob/main/7_Submission_examples.ipynb example 3 and 4. Example 3 correspond to the submission of a custom model and example 4 to the evaluation of a pre-trained custom model. Please note that in order to be eligible for the final ranking all model will need to be trained and evaluated through the submission process.

In case you would need packages that are not available at the moment don't hesitate to contact us and we will evaluate the feasability.

Best regards