Develop

[philpax]

The pending PRs were interrelated, but I didn't want to leave main in a half-working state, so I've merged all the PRs into a new develop branch. The plan is to work on this branch and leave main in maintenance mode until this is ready.

Closes #365, closes #403, closes #439, closes #77.

This integrates:

• a GGML version upgrade
• GGUF support
• BERT support
• APIs for context-shuffling

This is the to-do list:

• [ ] Update to the latest GGML
• [ ] Fix CUDA inference
• [ ] Fix OpenCL inference
• [ ] Fix Metal inference
• [ ] Fix the embedded tokenizer
• [ ] Readd quantisation
• [ ] Modularize the model definitions (i.e. move block inference to the block struct)
• [ ] Fix models (ensure they're uncommented in llm):
  • [ ] Fix BLOOM
  • [ ] Fix GPT-NeoX
  • [ ] Fix Falcon
  • [ ] Fix GPT-2
  • [ ] Fix GPT-J
  • [ ] Fix MPT
  • [ ] Fix BERT
• [ ] Remove the expects
• [ ] Fix the TODOs