search

deephaven / deephaven-core

Deephaven Community Core
Other
254 stars 80 forks source link

Produce requirements.txt release artifacts #3725

Open devinrsmith opened 1 year ago

devinrsmith commented 1 year ago

It would be good to produce a set of requirements.txt release artifacts, with the knowledge that these requirements.txt files have all been through our unit and integration testing frameworks (and potential further QA).

It is likely that some matrix of requirements.txt would need to be produced. At a minimum, it would need to be per python version, and potentially as an addition, multiplied by each "extra" desired set of packages (all-ai, nltk, pytorch, etc..).

The downstream consumers of these files would be:

This is not a trivial task.

Firstly, we don't test a matrix of python versions at the moment - only 3.10. https://github.com/deephaven/deephaven-core/issues/3724

Secondly, we have a sort of cyclical dependency between https://github.com/deephaven/deephaven-server-docker and this repository. https://github.com/deephaven/deephaven-core/issues/3723

Related https://github.com/deephaven/deephaven-core/issues/3426

devinrsmith commented 1 year ago

As an additional upside, this might be a nice pattern for advanced integrators who want to specify their own requirements file and ensure all of the DHC unit / integration testing works.

devinrsmith commented 1 year ago

There may be architecture-specific flavors that need to be accounted for as well (amd64 v arm64).

devinrsmith commented 11 months ago

There may be value in publishing the docker/server-jetty/src/main/server-jetty/requirements.txt, even though it's a bit limited in scope (only tested on amd64, only python 3.10, etc...).

It's up for debate whether we want / need to strip out the "optional" dependencies from this list (related https://github.com/deephaven/deephaven-server-docker/issues/71) such as turbodbc which needs to be purpose-built. That said, deephaven-server-docker as a downstream consumer does have the logic to build that wheel as necessary.