search

nextsimhub / nextsimdg

neXtSIM_DG : next generation sea-ice model with DG
https://nextsim-dg.readthedocs.io/en/latest/?badge=latest
Apache License 2.0
10 stars 13 forks source link

Benchmark test case doesn't run anymore #547

Open einola opened 4 months ago

einola commented 4 months ago

The benchmark test case doesn't run correctly after merging into develop (PR #532). The concentration field starts to deteriorate within 4 hours, with "spots" of open water appearing, which then spread. All the ice is gone, or NaN, within a day. In some versions, this happens more quickly.

I've tried running git bisect, but I had to skip a lot of commits because they didn't compile. I also skipped commits where the model didn't run but for seemingly different reasons. The final result of git bisect gives a large number of commits to try - but it's a start!

--- 8< ---

There are only 'skip'ped commits left to test. The first bad commit could be any of: 2ac46487f7a9b01d4493128b588d3ce029534e72 853146f4d9fea4699aaccdb54c2580dbe2d51d5d d224799aadbc699db9a0c2bd509406af085cac56 7a651646c85bf3b3429e5d905e62e1ee9d3f8367 f60ba75c873d1b65b149978d87c3a975f379bd61 97a10f24144848b9dd34fe922279d3f47ac55bda db22b7f3e703d604a7d368b339027fdd027c24a7 c34931bb1e2e1dd3e5816fed115d9d2b56d9bff6 fa34f4e6dbdf9b5c50e5b03e78815fb3ba19835f 2e96f0a2ca58e627866f53d22a332c387167c784 6a164c40d2741614648a0ca4facc7b79f35d183f 0c5fc1cfcdb0bff709b87ec729ab60b98f09be8f 100771b43a8e511ce764087e3c5762e9f84efcf9 237b2de476b320141c2c4f1b25a08e6e0f43a480 4021f2fbe788dc46fb3160548f712bd3729fc651 aade7a0fb5797e2ee1a6f0bccfae6144835c4295 f28ef281975f7fb8dab427b0f4a440d2b431ac72 7e38a2ed43da9074796eacbb535957cfe5ea6b4a 8e63e990d9e0b71dc6b7561c1e51eec1b5541251 2762f25053a7aca13aea3e07a3d1f163e76a414c b9ecf32b487b7e1565d0ba51d9afbbafb1890184 cfeab24636196588f591c33c0d39eb585f8798c9 ee2d4711e4ffec4e12d647e710f2da57ef17e63a 9b4b86f8b495aaf12a90b6da1ade5ad16712c93f f9b617de8baa591540bf8b2394afb0a4bc2ec609 525dd92e82ac1e718f29837380b586657b08e1b6 e867b76f6f6e56f8e3a6c51f191c04f14105ad17 2b8dc009accd0c84469cd627838b64cf8103552e fbea2b3d2aa7287510d75fb308dd2227a4d674e3 9209e9b779e8d2d190c7b0c2b6ff5a1e19d689ef da30f5f0e923fc9d0b15c0a1127ed8ef9342694c 281915017acc8e659a688d78bc9e7c3a2a2d4697 6bbb1cd44b79afceb56a2352d5ca9e655c78a50d 3ab51acf068ed7f93bfb476ff4c962c05b71f6de d755a2f06c79f241a5a4a17ddb67ad430f7d69a3 63e33852927ef06aa62b1aff593077655733014d e485245b0ecc19cad37f78456749fbfd61ad17b9 ca44575339badb5f1d6527d7ae01ce6d7488e477 We cannot bisect more!

timspainNERSC commented 4 months ago

./nextsim --config-file config_benchmark.cfg?

einola commented 4 months ago

./nextsim --config-file config_benchmark.cfg?

Yes. And you need to run make_init_benchmark.py first, although in develop, that should happen as part of the cmake run.

einola commented 3 months ago

It turns out that the model now has a problem with missing values in the init file. A simple workaround is to not use the missing_value attribute. See #564

The hotfix #564 doesn't solve the issue properly, so I'm leaving it open.