karpathy build-nanogpt issues

karpathy / build-nanogpt

Video+code lecture on building nanoGPT from scratch

3.44k stars 473 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

How is the autoregressive loss handled?

#82 BabyCNM closed 15 hours ago
2
Avoid tiktoken.decode panic on unknown tokens.

#81 IggShaman opened 1 month ago
1
Avoid tiktoken.decode panic on unknown tokens.

#80 IggShaman opened 1 month ago
0
torch.compile-d models do not work with example generation and hellaswag eval

#79 IggShaman opened 1 month ago
1
Make example generation work when the model is torch.compile-d

#78 IggShaman opened 1 month ago
0
fix: Progress bar not complete after starting a new shard

#77 sanbaiw opened 1 month ago
0
Update train_gpt2.py with compile mode for torch.compile

#73 manu-chauhan closed 1 month ago
1
Ensure Consistency Between GPTConfig.block_size and Sequence Length T

#72 Benetti-Hub opened 1 month ago
1
TTS

#67 yukiarimo opened 2 months ago
8
Refactor: Replace Class Name with cls in from_pretrained Method- Fix subclassing

#66 mohsenhariri opened 2 months ago
0
Add `torch_xla` support to `build-nanogpt`

#64 miladm closed 2 months ago
0
Enable torch compile with DDP

#62 stevebako opened 2 months ago
0
Fix torch.compile Issue - Error with HellaSwag eval and Generation

#60 ML-Guy closed 2 months ago
0
Run the script on MacOS

#59 amjadmajid opened 2 months ago
0
Run fineweb.py on MacOS

#57 amjadmajid closed 2 months ago
0
Text generation can use raw_model instead of model

#56 sapphire008 opened 2 months ago
0
consistently change model to raw_model

#54 hdocmsu opened 2 months ago
0
Dataloader now shuffles the shards and documents within

#52 fraserlove opened 2 months ago
1
Different inference results between flash attention and manually implemented attention appeared.

#50 Jaeckel-d opened 3 months ago
0
How to support padding in the train dataset for training ?

#49 mrhimanshu opened 3 months ago
2
Integrating GPT-2 with deepspeed Zero-1, Zero-2 and Zero-3

#48 Devadeut opened 3 months ago
1
Cannot get the log file "log124M_40B/log.txt"?

#47 dtdo90 opened 3 months ago
5
Fineweb sharding multi-process bugfix

#46 GrahamTheCoder closed 2 months ago
1
Running codes on Windows issues

#45 gerardaristizabalpla4 opened 3 months ago
2
RuntimeError: User specified an unsupported autocast device_type 'cuda:0'

#44 0smboy closed 3 months ago
1
Update README.md

#43 sriramgkn opened 3 months ago
0
Fix out of bounds check

#42 lukasugar opened 3 months ago
0
Executing with 1 GPU raises "OutOfMemory Exception", executing with 2 GPUs "RuntimeError: CUDA error: invalid device ordinal"

#41 nmerkle closed 3 months ago
2
add requirements

#40 mchen610 opened 3 months ago
0
add requirements

#39 mchen610 closed 3 months ago
0
564

#38 testwithtesty closed 3 months ago
0
Add .gitignore

#36 lutzroeder closed 1 week ago
0
Fix typo in configure_optimizers

#32 lukasugar opened 3 months ago
0
Is dataloader making optimal batches?

#31 paraschopra closed 3 months ago
1
fix sync issue that results in incorrect gradient accumulation and incorrect loss

#30 WilsonCWu closed 3 months ago
1
NO dropout in MLP and CausalSelfAttention

#29 peter-ni-noob closed 3 weeks ago
2
fix `Attempted to set non-positive bottom ylim` UserWarning in play.ipynb

#28 WilsonCWu closed 3 months ago
0
fix script for non-cuda devices

#27 adamskrodzki closed 3 months ago
1
current position should be 0 at the start a shard

#26 eliebak closed 3 months ago
2
Sharding the dataset not completing?

#25 dustinwloring1988 opened 3 months ago
7
Extending GPT2 for audio generation

#24 nivibilla opened 3 months ago
0
fix the double-jump for out-of-bound check in DataLoaderLite

#23 fatemi closed 2 months ago
3
Chunking method in the original GPT-2 training dataset

#22 rasbt closed 3 months ago
2
Fix typo in README.md

#20 awesomebytes closed 3 months ago
2
Propose validation loss calculation to imporve accuracy by reducing floating-point errors

#19 aakashapoorv opened 3 months ago
2
Embeddings are initialized with std of 0.02

#18 eryk-mazus opened 3 months ago
2
Implement tensor parallelism

#17 marib00 closed 3 months ago
4
Fix device type in autocast

#16 banyan-god closed 3 months ago
0
Update train_gpt2.py

#9 zhangfaen closed 3 months ago
0
Update README.md

#8 aepiotti closed 3 months ago
0