aws-samples / foundation-model-benchmarking-tool

Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.
MIT No Attribution
151 stars 21 forks source link

Loosen dependencies for more flexible installation #105

Open athewsey opened 1 month ago

athewsey commented 1 month ago

As discussed here on StackOverflow:

  1. Applications should generally lock dependencies to exact versions, for reliable deployment
  2. Libraries should generally support broad dependency version ranges where practical, to accommodate installing them on a range of environments and using them in a range of downstream applications

I suggest that poetry.lock accomplishes (1) for users wishing to download fmbench from source, but users installing fmbench from PyPI fall in the camp of (2), and would like fmbench to play nicely with whatever other dependencies might be in their environment.

For stable libraries that follow semver, it seems like we should be able to trust caret requirements? For e.g. specifically I would think something like the below (which I haven't fully tested):

For unstable libraries (seaborn, tomark, kaleido), maybe we could at least use tilde requirements to allow patch versions?

aarora79 commented 4 weeks ago

Thanks for the detailed work on this. I do think that the developer workflow should create a new conda env with Python 3.11 and then build the fmbench package using poetry. I realize that it would be nice to not have to do that and just install it an existing python venv but am not sure if that would lead us down a rabbit hole of dependency version mismatch. I can try it and see if it works with the versions you have provided and see how far we get.