-
### Blog Post
Balancing reproducibility and parallelization in data science presents a trade-off due to several factors. Managing consistent RNG states and process synchronization in parallel tasks…
-
Hello,
Currently, I am trying to convert a large dataset into NWB, i.e., we have extracellular recordings of length 60 minutes from 256 electrodes with a sampling frequency of 30 k. Directly loadin…
-
I have a script that periodically runs some heavy computation in 5 processes, and I'm timing the time it takes for each run to complete.
The computation involves querying from a Postgres DB and a L…
-
Dear Fluid TOC members and maintainers,
I am proposing to have Zhiqiang Liu (GithubID: xliuqq) as a new committer.
Zhiqiang Liu, is a software engineer. He currently is engaged in work related t…
-
While downloading data using PySEP, some steps of the procedure seem to be very slow, and if the flags requiring these steps are turned on, the data download can take a long time. I was wondering if i…
-
# A OverView of Parallel Computing Libraries in Python [Not Finished]
## 1. [multiprocessing](https://docs.python.org/3/library/multiprocessing.html) Module
The [multiprocessing](https://docs.p…
-
ghost updated
2 years ago
-
After a long journey of configuring different software versions on HPC (cf. #333 ), I ended up finding countless and inconsistent errors across nodes and sessions in HPC. Now I am trying to move on to…
-
Dear YangZhong,
I am writing to you regarding a memory issue I am encountering while calculating the band structure of a system with 3903 atoms using `band_cal_parallel.py`. With parameters set to …
-
Recognizing Steps to controlling the size of the data going into storage
Currently the system has 4 TB or storage. It is important to contoll the size of the data that is used for the model when sor…