melange396
commented
4 months ago To this end, we may want to get more information about the context of running environments to help us explain and identify discrepancies in performance/behavior/output... We should reexamine https://github.com/cmu-delphi/covidcast-indicators/issues/1702 with an eye on the later comments in https://github.com/cmu-delphi/delphi-epidata/issues/987 .
We should probably do some version pinning. It is good for consistency and reproducibility. There is essentially none in this repo, which is dangerous. Having completely unpinned versions risks inadvertent introduction of breaking changes without warning. We were bitten by this when the default
cvxpysolver engine changed and produced complications in debugging (see #1966). On the other hand, tight/hard pinning to exact and specific versions can leave us stuck with vulnerabilities or bugs or performance problems that might get fixed without us knowing (dependabothelps with that somewhat though). Also, when everything is hard-pinned, upgrading the version of ONE thing often means youll hafta go find and re-pin updated versions of others for compatibility with shared or dependent packages. Ideally, we could leave versions unpinned for development, and then hard-pin the exact versions as part of release process, but that relies on trusting the unit/integration testing process to expose all breaking changes or new bugs, which isn't a guarantee by any means. So... The best answer is somewhere in between.Perhaps we should pin our requirements/dependencies to just a particular "minor" version which should allow "patch" revisions for fixes but not introduce breaking changes. We might even consider specifying a strict minimum version along side that.
See also: