[api] [python] [r] Implement nightly package builds to test integrity of Python and R APIs with fuzzy dependency pins

[pablo-gar]

We want to ensure that any patch-level updates to TileDB-SOMA will be readily tested against the Census API, to catch potential regressions. This will allow the TileDB-SOMA package dependency to more confidently use a fuzzy version pin for the patch level. E.g. tiledbsoma~=1.4.0.

Schedule the Python unit tests GHA workflow and R unit tests GHA workflow to run on a nightly schedule

Notes:

• It does not seem necessary to test the Census API against all previous patch levels (e.g. using a GHA matrix), since users installing the Census anew will always get the latest patch level of tiledbsoma. We only need to ensure the latest patch, as of "today", is working correctly. (Also, we do not expect > 1 patch level TileDB-SOMA release per day, sou nightly testing should suffice.)

Ticket can be closed when work is completed for the builds of:

• [x] Python
• [x] R

[atolopko-czi]

@pablo-gar I think we agreed this shouldn't be high priority. That said, it's almost no work...

[atolopko-czi]

Should we run the R tests on a nightly basis as well? Does the tiledbsoma r-universe dependency get pinned at build time or user installation time? If the latter, then we should.

[atolopko-czi]

Should we run the R tests on a nightly basis as well? Does the tiledbsoma r-universe dependency get pinned at build time or user installation time? If the latter, then we should.

Pablo responds:

R-universe makes everything a little confusing (imo). But generally speaking as far as I know R API grabs the latests versions of its uninstalled dependencies at time of installation. This means that: For a complete clean environment, it will get all the latests dependency versions at installation time. For an existing environment, at time of installation it will use the existing installed dependencies unless the user opts-in for updating dependencies, if there are uninstalled dependencies it will grab the latest as described above. And @mlin included in the CI to test for both: Fresh environment recreating user journey as described in 1) above. Fresh environment recreating development installation — latest tiledbsoma installation from main github branch https://github.com/chanzuckerberg/cellxgene-census/blob/main/.github/workflows/r-check.yml (edited)

[pablo-gar]

@pablo-gar I think we agreed this shouldn't be high priority. That said, it's almost no work...

I see it as high priority but I did not consider it a blocker for our last release.

[bkmartinjr]

@pablo-gar - AFAIK, this is complete. Can you close if you and @mlin agree?

[pablo-gar]

Agreed! @mlin ?

[bkmartinjr]

Closing this, as it appears all necessary changes have landed