search

UM-Bridge / umbridge

UM-Bridge (the UQ and Model Bridge) provides a unified interface for numerical models that is accessible from virtually any programming language or framework.
https://um-bridge-benchmarks.readthedocs.io/en/docs/
MIT License
33 stars 14 forks source link

Modified CI to allow vector input #76

Closed chun9l closed 6 months ago

chun9l commented 6 months ago

Some benchmark fails in CI because the inputs are unrealistic. Atm users can only specify one value to fill the whole vector. Changes introduce here will allow users to set an input vector; the original behavior is retained.

Specify them like this --input_value="[input1, input2,...]"

linusseelinger commented 6 months ago

Looks good! Does this also work with models that have multiple input vectors?

chun9l commented 6 months ago

Ah good point, I didn't thought of that. Let me see if there's an existing model with this type of input

chun9l commented 6 months ago

The answer to your question was no sadly:(

But this new push should do the trick. I ditched json.loads and used eval instead. Also, altered the for loop in test_evaluate to account for nested lists.

For some reason though, spaces are not allowed when specifying input_value. May be due to how Python/Docker parses the string