obriensystems
commented
4 months ago i9-13900KS running dual RTX-A4500 (20+20G) Ampere and i9-14900K running dual RTX-4090 (24+24G) Ada
CPU first
C:/wse_github/llama.cpp $ ./main.exe -m g:/models/gemma-7b.gguf -p "what partion of gold is made in exploding stars" -n 2000 -e --color -t 24
Log start
main: build = 2234 (973053d8)
main: built with cc (GCC) 13.2.0 for x86_64-w64-mingw32
main: seed = 1708573388
llama_model_loader: loaded meta data with 19 key-value pairs and 254 tensors from g:/models/gemma-7b.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = gemma
llama_model_loader: - kv 1: general.name str = gemma-7b
llama_model_loader: - kv 2: gemma.context_length u32 = 8192
llama_model_loader: - kv 3: gemma.block_count u32 = 28
llama_model_loader: - kv 4: gemma.embedding_length u32 = 3072
llama_model_loader: - kv 5: gemma.feed_forward_length u32 = 24576
llama_model_loader: - kv 6: gemma.attention.head_count u32 = 16
llama_model_loader: - kv 7: gemma.attention.head_count_kv u32 = 16
llama_model_loader: - kv 8: gemma.attention.key_length u32 = 256
llama_model_loader: - kv 9: gemma.attention.value_length u32 = 256
llama_model_loader: - kv 10: gemma.attention.layer_norm_rms_epsilon f32 = 0.000001
llama_model_loader: - kv 11: tokenizer.ggml.model str = llama
llama_model_loader: - kv 12: tokenizer.ggml.bos_token_id u32 = 2
llama_model_loader: - kv 13: tokenizer.ggml.eos_token_id u32 = 1
llama_model_loader: - kv 14: tokenizer.ggml.padding_token_id u32 = 0
llama_model_loader: - kv 15: tokenizer.ggml.unknown_token_id u32 = 3
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,256128] = ["<pad>", "<eos>", "<bos>", "<unk>", ...
llama_model_loader: - kv 17: tokenizer.ggml.scores arr[f32,256128] = [0.000000, 0.000000, 0.000000, 0.0000...
llama_model_loader: - kv 18: tokenizer.ggml.token_type arr[i32,256128] = [3, 3, 3, 2, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - type f32: 254 tensors
llm_load_vocab: mismatch in special tokens definition ( 544/256128 vs 388/256128 ).
llm_load_print_meta: format = GGUF V3 (latest)
llm_load_print_meta: arch = gemma
llm_load_print_meta: vocab type = SPM
llm_load_print_meta: n_vocab = 256128
llm_load_print_meta: n_merges = 0
llm_load_print_meta: n_ctx_train = 8192
llm_load_print_meta: n_embd = 3072
llm_load_print_meta: n_head = 16
llm_load_print_meta: n_head_kv = 16
llm_load_print_meta: n_layer = 28
llm_load_print_meta: n_rot = 192
llm_load_print_meta: n_embd_head_k = 256
llm_load_print_meta: n_embd_head_v = 256
llm_load_print_meta: n_gqa = 1
llm_load_print_meta: n_embd_k_gqa = 4096
llm_load_print_meta: n_embd_v_gqa = 4096
llm_load_print_meta: f_norm_eps = 0.0e+00
llm_load_print_meta: f_norm_rms_eps = 1.0e-06
llm_load_print_meta: f_clamp_kqv = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: n_ff = 24576
llm_load_print_meta: n_expert = 0
llm_load_print_meta: n_expert_used = 0
llm_load_print_meta: rope scaling = linear
llm_load_print_meta: freq_base_train = 10000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_yarn_orig_ctx = 8192
llm_load_print_meta: rope_finetuned = unknown
llm_load_print_meta: model type = 7B
llm_load_print_meta: model ftype = all F32 (guessed)
llm_load_print_meta: model params = 8.54 B
llm_load_print_meta: model size = 31.81 GiB (32.00 BPW)
llm_load_print_meta: general.name = gemma-7b
llm_load_print_meta: BOS token = 2 '<bos>'
llm_load_print_meta: EOS token = 1 '<eos>'
llm_load_print_meta: UNK token = 3 '<unk>'
llm_load_print_meta: PAD token = 0 '<pad>'
llm_load_print_meta: LF token = 227 '<0x0A>'
llm_load_tensors: ggml ctx size = 0.10 MiB
llm_load_tensors: CPU buffer size = 32570.17 MiB
......................................................................................
llama_new_context_with_model: n_ctx = 512
llama_new_context_with_model: freq_base = 10000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init: CPU KV buffer size = 224.00 MiB
llama_new_context_with_model: KV self size = 224.00 MiB, K (f16): 112.00 MiB, V (f16): 112.00 MiB
llama_new_context_with_model: CPU input buffer size = 8.01 MiB
llama_new_context_with_model: CPU compute buffer size = 506.25 MiB
llama_new_context_with_model: graph splits (measure): 1
system_info: n_threads = 24 / 32 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 |
sampling:
repeat_last_n = 64, repeat_penalty = 1.100, frequency_penalty = 0.000, presence_penalty = 0.000
top_k = 40, tfs_z = 1.000, top_p = 0.950, min_p = 0.050, typical_p = 1.000, temp = 0.800
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000
sampling order:
CFG -> Penalties -> top_k -> tfs_z -> typical_p -> top_p -> min_p -> temperature
generate: n_ctx = 512, n_batch = 512, n_predict = 2000, n_keep = 1
what partion of gold is made in exploding stars.
We have seen that there are very strong indications, both from meteoritic data and terrestrial rocks (see Chapter 8), that the Earth has experienced a bombardment by large bodies (planetesimals) during the first few million years after its formation $460 \times$ $\left(1-\delta_{\mathrm{E}}\right)$ ago. These planetensimals were largely made of silicates and iron, but they contained less metal than is found in meteorites today: their meteoritic equivalent was the enstatite chondrite with a low amount ( $9 \%$ by mass) metallic FeNi alloy; this type has been identified from both lunar rock fragments and martian samples. Thus these projectiles were largely made of silicates, but they contained 15\% metal compared to $\sim 30-42$ vol.\% for the present terrestrial core: therefore their impact on a young Earth must have produced an important amount (at least $~ \frac{8}{6}$ ) metallic FeNi alloy.
The total mass of these projectiles, which is estimated from both lunar rocks and martian samples to be $\sim 05$ M $_{E}$, implies that the fraction ejected into space by their impact was between a half $(1 /:)$ or two thirds (2/3) depending on the initial amount in FeNi alloy. Thus our Moon represents $7 \%$ of these projectiles, while Mars may have formed as much as one third $(\sim 4 \%)$. This last figure is to be compared with that found for planetary formation by dynamical processes which indicates a mass ratio between Earth and its moon at most equal
<h1>CHAPTER $\mathrm{X</h1>
to unity. The fraction ejected into space in FeNi alloy was therefore of the order $02(1 / 3)$, i e., it represents about half $(\sim \delta)$ the total amount of metallic iron, or an average abundance for meteoritic chondrites (see Table X-5). It is interesting to note that this fraction is very similar both to what we have estimated in Chapter VIII from present terrestrial core data and also from meteorites. Thus it appears more probable that a large part of the Earth's metallic FeNi alloy was formed by impact, rather than due solely or mainly because of differentiation during planetary formation (see Fig 10-2).
The other half $(\sim \delta)$ is probably made in stellar interiors where we have seen above the Moon. at as: to to to to to to to once more than $45 on both by by over Earth's surface,,,,,,,,,,,,,,,,, but but but when it came be expected from during its formation is is may indicate that purely theoretical reason
once. discretely and stimated to to to to to to to to in in in in in in in in in in in once more than 554 on on over Earth's a fine dust, mass the size of once they are small at any amount about $234 and as formed from above one- purely theoretical estimate is forこの日 twice its possible that hereto form $\quad$ formation. strictly speaking which was first very much larger in under an extremely large estimated fraction
on over Earth's slightly greater than a mass may have been the average once or at at any more of just barely bigger $0 within 1 to to amongst it seems highly about one another small amount formed during its possible origin for each and (a great deal less is not yet further our very much larger. Thus far above all $\quad$ strictly speaking as we still greater than most probably in the average while a mes quite big. once a mere theory ofRDONLY more... an an example, thereunder weight half that hereto form 1mipmap or on favourably huge new super- once andANNES ( over its structure size thatpelier one extremely heavy duty earth large beyond where it is almost double how very much larger than life itself the outside world above all but at (()one forividuated several times more slightly heavier.この日, thereunder $\left(((( ( ( ( ( ( ( ( ( ( ( ( ( once again highly probably a just half- purely because ofcreateServer size one and hereto form earth large beyond which had it is mesquite the almost double duty above all mass that within its structure weight very much larger than life itself outside world over twice more heavy slightly heavier. thereunder, (()one was formed somewhat smaller at $0 on once again highly probably greater in forming just about a half- once more heavily inside out whose outer size one greatly beyond which might be the almost double duty earth quite large above all mass formation weight within its structure very much larger than life itself outside world would have (somehow it is slightly heavier. The amount of discretely under(and somewhat formed on had twice more heavy that was in from $\quad$ some amount greater below average (that it is just occurred halfway through and once but not, meselfromore the release . [end of text]
llama_print_timings: load time = 4247.23 ms
llama_print_timings: sample time = 859.49 ms / 1030 runs ( 0.83 ms per token, 1198.38 tokens per second)
llama_print_timings: prompt eval time = 693.44 ms / 11 tokens ( 63.04 ms per token, 15.86 tokens per second)
llama_print_timings: eval time = 412311.19 ms / 1029 runs ( 400.69 ms per token, 2.50 tokens per second)
llama_print_timings: total time = 415226.57 ms / 1040 tokens
Log end
fmichaelobrien
see #27 https://ai.google.dev/gemma/docs?hl=en https://www.kaggle.com/models/google/gemma
Gemma on Vertex AI Model garden https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/335?_ga=2.34476193.-1036776313.1707424880&hl=en
https://obrienlabs.medium.com/google-gemma-7b-and-2b-llm-models-are-now-available-to-developers-as-oss-on-hugging-face-737f65688f0d
https://storage.googleapis.com/deepmind-media/gemma/gemma-report.pdf https://blog.google/technology/developers/gemma-open-models/
https://huggingface.co/google/gemma-7b https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/l4/PB-11316-001_v01.pdf
pull and remake the latest llama.cpp (see previous article running llama 70b - https://github.com/ObrienlabsDev/machine-learning/issues/7
https://github.com/abetlen/llama-cpp-python/issues/1207 https://github.com/ggerganov/llama.cpp/commit/580111d42b3b6ad0a390bfb267d6e3077506eb31
7B (32G model needs 64G on a CPU or a RTX-A6000/RTX-5000 Ada) and 2B (on a macbook M1Max:32G unified ram - working perfectly
https://cloud.google.com/blog/products/ai-machine-learning/performance-deepdive-of-gemma-on-google-cloud