-
## Description
Lately I have been working with PartitionedDataset a lot in a setting where I am processing many small files (think 30k+ files), all together > 30GB. Processing them sequentially in a …
-
### What is your issue?
Given the following situation:
- a small Dataset with a few variables and a single dimension `dim1` , backed by Dask
- a large Dataset with a single variable and a singl…
-
In the part of “Clean datasets from excluded primaryids” and “Rule-based deduplication
”,We performed independent data cleaning on the latest data 23Q1 or 23Q2 if we should perform the same operation…
-
SOAP seems to put a big load on the Lustre file system on Cosma so we aren't able to run many instances at the same time. With recent improvements in HDF5 we might be able to fix this.
SOAP splits …
-
Kerchunk allows "scanning" FITS datasets and representing them as zarr to be loaded by xarray, including concatenating or otherwise combining multiple HDUs or files into a single logical dataset. What…
-
Issue: I am currently running into an issue when trying to run a process with Task/Dask Vine, wherein the process inevitably fails when the input dataset is too large. The symptoms are:
* The proc…
-
### Milestones:
- Study 3DGS implementations and identify best option, risks and blockers for integration
- Study [filament renderer](https://google.github.io/filament/Filament.html) and Open3D-fi…
-
Hello SCIMAP developers,
I wanted to know if it is possible to use either MERFISH or Visium data with this tool, and how would one go about doing so. Because, I keep encountering an error with rega…
-
Do these algorithms calculate the overall maximum of the data set? or just the maximum in each thread block?
-
I follow [these example scripts](https://github.com/HaixinShi/fmov_pose?tab=readme-ov-file#running) and face the following error:
```
Hello FMOV
Use global conf...
Load data: Begin
scale_mats_np …