search

tumaer / lagrangebench

LagrangeBench: A Lagrangian Fluid Mechanics Benchmarking Suite
https://lagrangebench.readthedocs.io
MIT License
57 stars 6 forks source link

Config files for testing #8

Closed hanjq17 closed 10 months ago

hanjq17 commented 1 year ago

Dear authors,

Thanks for the great work!

I have a little concern---The config files provided in configs seem to be the training configs. Could you share the evaluation (testing) configs so as to reproduce the results in the paper (e.g., Table 2)?

I changed mode into infer and added model_dir while keeping all the rest the same as training configs. It is correct?

Thanks again and I look forward to hearing from you!

Best, Jiaqi

arturtoshev commented 1 year ago

Dear Jiaqi,

thanks for trying our code!

In general, your approach to change mode to infer and add the model_dir path is correct. However, we used all available trajectories during testing (infer), so your results might not be the same.

For the camera-ready version, we are running all experiments with more seeds and actually, we already modified the configs to have one more mode called all, i.e. train and test in one go (see https://github.com/tumaer/lagrangebench/pull/7). This new setup then has all the configs for training and testing and we will update the paper in around 10 days (camera-ready deadline). We also changed some model-related hyperparameters, which we will also push in around 10 days. I'll ping you once we've made these updates. Thanks for your understanding.

Best, Artur

hanjq17 commented 1 year ago

Hi Artur,

Great to hear this! I look forward to the update---let me know when you make it!

Best, Jiaqi

arturtoshev commented 11 months ago

Hi Jiaqi,

we just made the step and released the newest version: https://github.com/tumaer/lagrangebench/releases/tag/0.0.2 (We also updated the paper on arXiv, if you want to have our latest benchmarking results - we essentially reran all experiments)

We will be improving the documentation further, but a combination of the README and the notebooks should now provide some insights into your questions. If your question is answered, please close this issue and open new ones if you have further doubts. All feedback is highly appreciated :)

Best, Artur

arturtoshev commented 10 months ago

I close this issue after almost two months with no activity. Feel free to open a new one anytime.