issues
search
ReaLLMASIC
/
nanoGPT
The simplest, fastest repository for training/finetuning medium-sized GPTs.
MIT License
20
stars
16
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Add support for Audio Tokenization and Voice Emulation via SNAC Tokenization
#198
klei22
closed
1 week ago
0
Add wte factorization
#197
klei22
opened
2 weeks ago
1
Add scripts compatible with the Newswire Dataset
#196
klei22
closed
1 week ago
0
a draft of gated attention and its modification
#195
Hrancheng
opened
2 weeks ago
0
golden gen for decoder
#194
Buck008
closed
1 week ago
3
Add nan handling strategy for run_vizier.py
#193
klei22
opened
3 weeks ago
0
Add quantized krmsnorm
#192
klei22
closed
1 week ago
3
Rubiks cube improvements
#191
klei22
closed
3 weeks ago
1
Add matrix factorization and visuals
#190
klei22
opened
3 weeks ago
1
Add linear wrapper and kan feedback
#189
klei22
closed
4 weeks ago
1
Remove duplicate block in model.py
#188
gkielian
closed
4 weeks ago
0
Add scripts creating compatibility for additional dataset
#187
klei22
closed
1 month ago
0
Add vizier optimization
#186
klei22
closed
1 month ago
0
added core_array code and activation function code
#185
zymeng3001
opened
1 month ago
0
Quantization and Binarization Implementations for Linear Layers
#184
mmoffatt2
opened
1 month ago
3
Add scripts compatible with Zyda and FineWeb-Edu pre-training datasets
#183
klei22
closed
1 month ago
0
Add kan and hyperparams
#182
gkielian
closed
4 weeks ago
4
try of implementing cross-layer attention
#181
Hrancheng
opened
1 month ago
3
Add hardware component verilog from tapeout repo
#180
zymeng3001
closed
1 month ago
0
Add embedding table factorization
#179
klei22
opened
1 month ago
1
Add Option for Memory Optimized Training via Gradient Checkpointing
#178
klei22
closed
1 month ago
1
Add settings log, graphs, and model augmentation to sample.py
#177
klei22
closed
1 month ago
0
Add and improve scripts for dataset processing
#176
klei22
closed
1 month ago
1
Add template for parquet datasets
#175
klei22
closed
1 month ago
0
Added boxplots
#174
mmoffatt2
closed
1 month ago
1
replace linear with KAN
#173
SenmiaoORZ
closed
1 month ago
3
Add interactivity to sample.py
#172
klei22
closed
1 month ago
0
Fixed typo in consmax_sweep config
#171
karthik-sunil
closed
1 month ago
0
Add fix for torchrun multigpu training
#170
gkielian
closed
1 month ago
0
Add dim to init functions for softmax variations
#169
gkielian
closed
1 month ago
0
Add option to use SwiGLU FFN Option
#168
gkielian
closed
1 month ago
0
Softmax parameter sweep
#167
karthik-sunil
closed
1 month ago
1
Softmax formulation pass
#166
gkielian
closed
1 month ago
1
merged with master and enabled plotting of strongermax/polymax
#165
Hrancheng
closed
1 month ago
0
Make n_kv_group 6 by default to enable flash attn
#164
gkielian
closed
2 months ago
0
Modified train.py to enable plotting of input/output statistics for constantmax
#163
Hrancheng
closed
2 months ago
1
Add Softplus activation and update inspect script
#162
gkielian
closed
2 months ago
0
Add scripts compatible wtih the korean parallel corpora
#161
klei22
closed
2 months ago
0
Fix non euler base calculation for exppolymax
#160
gkielian
closed
2 months ago
0
Add polymax with relu2 forward pass (PolymaxQuan)
#159
gkielian
closed
2 months ago
1
Add scripts for testing out-of-distribution addition
#158
klei22
closed
2 months ago
0
Add scripts compatible with lichess dataset
#157
klei22
closed
1 month ago
0
Tidy implementation
#156
gkielian
closed
2 months ago
1
Add softmax variation dictionary
#155
gkielian
closed
2 months ago
0
Quest: Add chess dataset for chess bot
#154
gkielian
closed
1 month ago
1
Add partial RMSnorm "pRMSNorm" variation
#153
gkielian
closed
2 months ago
1
Add table customization
#152
gkielian
closed
2 months ago
1
Add checkpoint inspector
#151
gkielian
closed
3 months ago
0
Add vector tokenization
#150
gkielian
opened
3 months ago
0
Fix multigpu training for train.py script
#149
gkielian
closed
1 month ago
1
Next