merge allc - Githubissues

lhqing / whole_mouse_brain

misc code for whole mouse brain analysis

MIT License

5 stars 0 forks source link

merge allc #9

Closed lhqing closed 2 years ago

lhqing commented 2 years ago

I will create

[x] L4Region label for mC and m3C
[x] prepare ALLC file path list
[x] Merge CEMBA mC ALLC: ecker-rachel-analysis/merge_allc/result
[x] Merge CEMBA m3C ALLC: ecker-rachel-analysis/m3c_merge_allc
[x] Prepare BaseDS for mC ALLC files
[x] Prepare BaseDS for m3C ALLC files
[x] Check ALLC merge and BaseDS correctness by comparing with MCDS at 100Kb bins

lhqing commented 2 years ago

@rachelzeng98 I created the L4Region label for all datasets, see #2

Here is all the information needed for merging ALLC, update wmb package

CEMBA_MC:
- Label: cemba.get_mc_annot()['L4Region']
- Single Cell ALLC Path: cemba.get_allc_path(dataset='mc', allc_type='full')
CEMBA_M3C
- Label: cemba.get_m3c_annot()['L4Region']
- Single Cell ALLC Path: cemba.get_allc_path(dataset='m3c', allc_type='full')

Before starting, please also check that the labels are as described and that all files are accessible to you.

lhqing commented 2 years ago

I'd suggest we start with a small cell cluster, prepare a sky-pliot YAML file, try spot job. Once worked, we can easily parallel all clusters.

Right now the allcools merge can NOT resume if the spot VM is killed. The resubmission will just let the merge job start again from the beginning, which is fine - if the spot-vm kill doesn't occur too frequently.