-
Hello,
I am trying to read a matrix from a file with Matrix Market (MM) format within a distributed computing framework using MPI.
I familiarized with the methodology explained in the following e…
-
### Application contact emails
maintainers@drasi.io, alljones@microsoft.com, nvalsan@microsoft.com, Smith.Jonathan@microsoft.com
### Project Summary
Drasi is a change data processing platform tha…
-
I would like to specify in my application what dependency injection should be used on the server side of the dataportal, using a composite pattern.
Problem is, how do i get my dataportal to take 'an …
-
A FAQ in the community is a structured roadmap for learning Data Engineering and it's about time we start addressing it. We currently have a getting started guide but it's not detailed enough and was …
-
Subscribe to this issue and stay notified about new [weekly trending repos in Go](https://github.com/trending/go?since=weekly)!
-
### Feature Type
- [X] Adding new functionality to pandas
- [ ] Changing existing functionality in pandas
- [ ] Removing existing functionality in pandas
### Problem Description
Currently, panda…
-
### 🚀 Feature
I propose enhancing the `SubprocVecEnv` to support multiple clusters. To achieve this, I have created a new class called `DistVecEnv` that is fully compatible with `SubprocVecEnv` as …
-
Hi! I have noticed that you utilize the "accelerate" repo to train the model only on a single GPU. I change it to multi-gpus and use "model.module" to replace the corresponding code. However, there is…
-
- 1 Overview of StoneDB V2.0
- 2 SQL Syntaxes
- 3 Massive Parallel Processing
- 3.1 Data Distribution Approaches
- 3.2 Redistribution Approaches
- 3.3 Metadata Management
- 4 Query Engine
…
-
I am trying to do data analysis on the 9900 parquet files that in total they have 100GB size.
After 70K garbage collections warning:
`distributed.utils_perf - WARNING - full garbage collections …