-
Crawl data
1. Collect any data that suitable for our LLM
- Check quality of data // Should collect or not with the amount, available for collection, quality of text; paragraph, content, etc.
- C…
-
Do a literature review on the Multilingual Autoregressive model pretraining
Area to focus
1. Pretraining techniques that make multilingual training different from monolingual
2. Autoregressive models…
-
Explore pre-training configurations or other literatures
Example: Optimizer, dataset sampling, and vocab size of tokenizer
tokenizer type: SPM vs BPE
clean data thresold
learning rate (MUP)
!…
-
First, amazing theme, finally having something that works properly on my mobile device is so wonderful, thanks so much for developing this!
That said, and I hate to be that guy, but there are a cou…
-
**Description:**
Add MPT with Gradient Checkpointing and LoRa support into OpenThaiGPT pertaining code. We will use MPT with Lora for continue pertaining to task #179
**To Do:**
1. MPT Weight + MP…
-
Similar issue has been raised before, but this can not be worked around by running v6 before v4, as for instance with target MARK. The target is JOOL_SIIT
OS: Ubuntu 20.04 with kernel 5.4.0-104-gen…
-
The doc says that for the tree:
```
root
|
+----A
| |
| +---B
| |
| +---C
|
+----D
|
+---E
|
+---F
```
The resulting order in which node…
-
https://www.bcn.cl/siit/actualidad-territorial/nueva-region-de-nuble
-
Hi,
Just started to use Jool_siit, great SW.
I wish there was vrf support, so Jool_siit would read out device from vrf instead of global routing table.
And it would also be nice if one could sp…
-
Please consider adding Jool package for AsusWRT-Merlin. There is Tayga package but it is not maintained anymore.
- Jool does NAT64/SIIT and is currently active and maintained.
- Website -- www.joo…