Remove Dask integration - Githubissues

biocore / BIRDMAn

Bayesian Inferential Regression for Differential Microbiome Analysis

BSD 3-Clause "New" or "Revised" License

22 stars 5 forks source link

Remove Dask integration #48

Closed gibsramen closed 3 years ago

gibsramen commented 3 years ago

After talking with @mortonjt, we decided that it was probably best if we remove explicit Dask integration. It is becoming fairly cumbersome to debug and manage and would likely only get more convoluted as the project evolves.

To this end, I am working on streamlining the API to support single feature fitting such that a user can adapt a parallelization scheme that supports their needs best.

TO DO:

[ ] Update documentation to reflect this change
[ ] Better expose single feature fitting
[ ] Add unit tests for single feature fitting
[ ] Add documentation example for parallelization on PBS/Slurm/etc. (array jobs probably)

gibsramen commented 3 years ago

Going to remove references to Dask for this PR and merge. Addressing #51 will be a pretty large refactor that I'd like to keep separate. Several of the TODOs will be addressed in that PR.