-
### Informations
- **Qiskit Aer version**: 0.14.0
- **Python version**: 3.11.6
- **Operating system**: Ubuntu 23.10
### What is the current behavior?
I am running a code to create GHZ s…
-
This can be reproduced by cloning latest Megatron-LM and enabling transformer_engine for `--transformer-impl` instead of using local implementation.
The experiments are run in a `nvcr.io/nvidia/pyt…
-
背景:
环境:
机器: 8 P40物理机
docker: paddlecloud镜像: iregistry.baidu-int.com/paddlecloud/base-images:paddlecloud-ubuntu20.04-gcc8.2-cuda11.8-cudnn8.9-openmpi4.1.5-codelab1.6.1.5-bccl2.15.5.4-hadoop2.2.4.2-a…
-
Hey guys,
When I try to train a new gpt2 model using pretrain_gpt.sh I get the following error
--------------------------------------------------
DeepSpeed C++/CUDA extension op report
------…
-
As you do this list, if it doesn't match the description here hop on build page and double check the description wasn't rewritten. We've been tweaking quite a bit, especially mana regen abilities and …
-
Parallel distributed lazy runs out of memory during compilation on a 3M element mesh, 3D, second order.
Node counts of 1, 2, 4, 8 were tried on lassen. With 4 ranks (gpus) utilized on each node.
…
-
I was trying to train Polycoder using the [preconfigured dataset](https://github.com/frankxu2004/gpt-neox#datasets), from the checkpoint `checkpoints-2-7B`, I used the following command as per the [in…
-
环境：nvidia a10 24g显存，docker：nvidia/cuda:11.8.0-cudnn8-devel-ubuntu20.04，cpu：Intel® Xeon® Silver 4314×2，mem：256G
日志如下：
NCCL_DEBUG=info NCCL_IB_DISABLE=0 NCCL_NET_GDR_LEVEL=2 deepspeed --master_port 16…
-
Hi,
I am trying to reproduce the results of ZeRO on GPT2 and BERT pretraining.
I followed this tutorial:
https://github.com/microsoft/DeepSpeedExamples/blob/master/Megatron-LM-v1.1.5-ZeRO3/READM…
-
**Summary:**
本文分析了文本搜索中混合搜索的融合函数，具体探讨了词汇搜索和语义搜索的融合方法，包括凸结合（Convex Combination, CC）和相互排名融合（Reciprocal Rank Fusion, RRF），并揭示了它们各自的优点和潜在问题。研究发现，RRF对参数敏感，而CC对分数正则化的选择一般不敏感，CC在域内和域外设置中均优于RRF。此外，CC在少量训…