BLMM main with dask - Githubissues

TomMaullin / BLMM

This repository contains all code for the BLMM toolbox.

19 stars 5 forks source link

BLMM main with dask #61

Closed vinferrer closed 2 years ago

vinferrer commented 2 years ago

Close #28

This PR prepares the BLMM.py main script for job submission with dask

Changes proposed in this pull request:

use dask to submit setup, batch, concatenation, result and cleanup as jobs with dask
delete unnecessary bash scripts
now blmm_setup returns nb and nvb as arguments so we don't need to write them in a .txt

vinferrer commented 2 years ago

Hello, @TomMaullin. I used your code in #63 to generate some dummy data and use BLMM in my cluster. The pipeline finishes without problems right now. Would you like to review my changes and once we merge with #63 we can do integration testing

TomMaullin commented 2 years ago

Hi @vinferrer, Ah great! Apologies I think I must have got my wires crossed slightly as I thought you weren't currently looking at this and started building my own branch based on this. My current working version is here:

https://github.com/TomMaullin/BLMM/blob/dask_update/BLMM/BLMM.py

It seems there are probably things in both PR's we might want to keep; perhaps we could video call at some point and discuss how best to proceed from here?