search

ibpsa / project1-boptest-gym

Other
39 stars 20 forks source link

Update boptest-gym-service to BOPTEST 0.3.0 #111

Closed javiarrobas closed 1 year ago

javiarrobas commented 2 years ago

Issue https://github.com/ibpsa/project1-boptest-gym/issues/109 updated BOPTEST-Gym master to BOPTEST v0.3.0. This issue is to update BOPTEST-Gym-Service and tutorial to the same version. @dhblum @kbenne @icupeiro FYI.

javiarrobas commented 2 years ago

The tutorial will be updated in a different issue.

kbenne commented 2 years ago

@JavierArroyoBastida and @dhblum

boptest.net is now running 0.3.0 https://api.boptest.net/version

FYI @tijcolem

javiarrobas commented 2 years ago

Nice! I will let you know once I get the tutorial running.

dhblum commented 2 years ago

Wow congratulations @kbenne and @tijcolem! This is a great milestone! I'll update the webpage to reflect this soon.

dhblum commented 2 years ago

Probably not the best place to put this, but @kbenne when I run curl api.boptest.net/testcases my response is:

[{"testcaseid":"bestest_air"},{"testcaseid":"bestest_hydronic"},{"testcaseid":"bestest_hydronic_heat_pump"},{"testcaseid":"multizone_residential_hydronic"},{"testcaseid":"singlezone_commercial_hydronic"},{"testcaseid":"testcase1"},{"testcaseid":"testcase2"},{"testcaseid":"testcase3"},{"testcaseid":"64018"},{"testcaseid":"228106"},{"testcaseid":"109265"},{"testcaseid":"383154"},{"testcaseid":"38791"},{"testcaseid":"28945"},{"testcaseid":"42331"},{"testcaseid":"446650"},{"testcaseid":"338822"},{"testcaseid":"357336"},{"testcaseid":"529961"},{"testcaseid":"471662"},{"testcaseid":"179364"},{"testcaseid":"515291"},{"testcaseid":"132930"},{"testcaseid":"258546"},{"testcaseid":"175195"},{"testcaseid":"453060"},{"testcaseid":"480482"},{"testcaseid":"335712"},{"testcaseid":"13433"},{"testcaseid":"462583"},{"testcaseid":"416029"},{"testcaseid":"216570"},{"testcaseid":"519255"},{"testcaseid":"200797"},{"testcaseid":"390841"},{"testcaseid":"360023"},{"testcaseid":"281822"},{"testcaseid":"304966"}]
  1. I don't think there should be all those numbered test cases.
  2. I don't see an option for multizone_office_simple_air which is part of 0.3.0.
kbenne commented 2 years ago
  1. @jmythms is using the server for MegaBOP runs. Those are ResStock building IDs. I'd like to add per user testcases to avoid this in FY23.
  2. I need to add multizone_office_simple_air
kbenne commented 1 year ago

Friendly reminder @JavierArroyoBastida. I hope this is still on your radar. I think the notebook is especially popular.

javiarrobas commented 1 year ago

Thanks for the reminder @kbenne! and sorry this got delayed... I'm testing the Colab notebook against service now, but it gets hanging forever when running the following:

image I get an answer when using curl in the same notebook though:

image

Any idea why this could be happening?

javiarrobas commented 1 year ago

Gentle reminder about this @kbenne. I've just tried once again but it's still hanging.

kbenne commented 1 year ago

Hello @JavierArroyoBastida. I was away on vacation for a little bit, but I'm back now.

Sometimes I see PyFMI hang on a difficult model. I think the algorithm tries to downstep infinitesimally smaller and smaller resulting in a situation where calls to advance never return. In the service architecture the worker is rendered unavailable and while there is a timeout mechanism, the timeout will not interrupt an active call into pyfmi. We are using the service deployment to run a wide range custom testcases so this possibility is all the more common.

I'm running the service with only two workers and I found both occupied by hung jobs, so I went ahead and killed the jobs and now the workers are idle and available for requests.

Very soon we will be doing some updates to address this situation. 1. we will implement a watchdog to automatically kill hung jobs. 2. We will implement autoscaling to add more workers, when there are jobs in the queue and not enough workers to pick them up.

kbenne commented 1 year ago

@dhblum I added multizone_office_simple_air.

javiarrobas commented 1 year ago

@kbenne sorry for my delay on this. Nice to see the plans to update that situation! I'm trying to update the tutorial but I'm afraid I've run into the same situation as before. Probably the workers are idle again. If so, could you kill them? Hopefully next time I can run the tutorial at once.

javiarrobas commented 1 year ago

Closing since BOPTEST-Gym service and tutorial have been updated to BOPTEST 0.3.0 already.