issues
search
ReaLLMASIC
/
nanoGPT
The simplest, fastest repository for training/finetuning medium-sized GPTs.
MIT License
23
stars
17
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Add scripts compatible wtih the korean parallel corpora
#161
klei22
closed
4 months ago
0
Fix non euler base calculation for exppolymax
#160
gkielian
closed
4 months ago
0
Add polymax with relu2 forward pass (PolymaxQuan)
#159
gkielian
closed
4 months ago
1
Add scripts for testing out-of-distribution addition
#158
klei22
closed
4 months ago
0
Add scripts compatible with lichess dataset
#157
klei22
closed
3 months ago
0
Tidy implementation
#156
gkielian
closed
4 months ago
1
Add softmax variation dictionary
#155
gkielian
closed
4 months ago
0
Quest: Add chess dataset for chess bot
#154
gkielian
closed
3 months ago
1
Add partial RMSnorm "pRMSNorm" variation
#153
gkielian
closed
4 months ago
1
Add table customization
#152
gkielian
closed
4 months ago
1
Add checkpoint inspector
#151
gkielian
closed
4 months ago
0
Add vector tokenization
#150
gkielian
opened
4 months ago
0
Fix multigpu training for train.py script
#149
gkielian
closed
3 months ago
1
Add scripts compatible with MMLU Benchmark
#148
klei22
closed
5 months ago
1
Add option for Mixture of Experts
#147
klei22
closed
4 weeks ago
1
Qilong::Add new dataset - guanaco
#146
Mars-Cat2023
closed
5 months ago
0
Add random dataset
#145
klei22
closed
5 months ago
1
Add scripts compatible with the MNIST dataset
#144
klei22
closed
5 months ago
1
Add script to mix bin files for combined training
#143
klei22
closed
5 months ago
1
Add scripts compatible with cosmo100k dataset
#142
klei22
closed
5 months ago
0
Add FIRE Positional Encoding
#141
gkielian
closed
5 months ago
1
Quest: Add a Rough draft for Mixture of Experts
#140
gkielian
opened
5 months ago
0
Quest: Create a training efficient version of windowed attention for experimenting with long contexts
#139
gkielian
opened
5 months ago
0
Add linear variations
#138
gkielian
closed
5 months ago
1
Add parallel mlp attn option
#137
gkielian
closed
5 months ago
1
Fix for run_experiments timestamp
#136
klei22
closed
5 months ago
1
Add means for beta, gamma, in and out logging
#135
gkielian
closed
4 months ago
0
Add fire positional encoding
#134
gkielian
closed
5 months ago
1
A Github action workflow to test GQA combination with gating
#133
Hrancheng
opened
5 months ago
0
Create Same Timestamp For Tensorboard Logs and Output Checkpoints
#132
mmoffatt2
closed
5 months ago
0
Add support for experimental layernorm variations
#131
gkielian
opened
5 months ago
0
Add mixtral8x7b example
#130
klei22
closed
5 months ago
1
Quest: Curriculum and Preprocessing for Translation Targets
#129
gkielian
opened
5 months ago
0
Quest: Create same timestamp and filename for tensorboard logs as output checkpoint
#128
gkielian
closed
5 months ago
1
Add softmax variation configuration sweep
#127
gkielian
closed
5 months ago
0
Quest: Have run experiments print a number of combinations and optional printing of all options to file
#126
gkielian
opened
6 months ago
0
Quest: Add Quantization Aware Training and Inference
#125
gkielian
opened
6 months ago
0
Add scripts compatible with additional datasets
#124
klei22
closed
5 months ago
0
Template Config for Swapping Out Different Datasets
#123
mmoffatt2
closed
5 months ago
0
OpenFASoC PPA Analyzer
#122
harshkhandeparkar
opened
6 months ago
0
Quests: Add Weather Dataset
#121
gkielian
opened
6 months ago
0
Add mlp and attn groups
#120
gkielian
closed
5 months ago
0
Add GQA and new rope variant
#119
gkielian
closed
5 months ago
0
Quest: Explore training stability of network with bf16 and fp16
#118
gkielian
opened
6 months ago
0
Quest: Add support for loss masking
#117
gkielian
opened
6 months ago
0
Quests: Rotary Positional Embedding
#116
gkielian
opened
6 months ago
2
add rough draft ROPE class
#115
alibillalhammoud
opened
6 months ago
3
Argparse Argument for "Patience" -- and Early Exit without Updates for "patience" number of steps
#114
mmoffatt2
closed
6 months ago
1
Documentation updates
#113
gkielian
closed
6 months ago
0
Add GQA prototype
#112
klei22
closed
6 months ago
0
Previous
Next