Closed dan-homebrew closed 1 week ago
Here is the current Jan Data Folder structure:
jan 📁
assistants 📁: Contains Jan-supported assistants (currently only jan
). New copied assistants do not take effect. This folder will be created if it does not exist when the assistant-extension
is loaded.
jan
, but this is not used anywhere in the codebase. Currently, jan
is the only active assistant and is accessed via index 0 in the assistants array.Jan
.*
, meaning all models are available.Save instructions for new threads
feature implementation).retrieval
, has been created and supported so far.retrieval
, not referenced in the codebase.timeWeightedRetrieval
feature is activated. This feature was implemented by an external contributor (naming convention issue).retrieval
tool:
extensions 📁: Created on app launch (App level) if not already present. All extensions (compressed tgz) will be extracted here if they are not installed.
@janhq/monitoring-extension
). If no organization is defined, it becomes an extension folder (e.g., example-extension
in package.json
).package.json
. This file helps avoid looping through all extension folders, which is costly. Considered as deprecated soon, as manual extension copying may cause the file to become outdated (Changes to the extension core code are needed).logs 📁: Created when the monitoring-extension
is loaded.
models 📁: Created when the models-extension
is loaded.
model.json
may be located in a nested folder such as models/anthropic/claude-3.5-sonnet
. The app uses the model's ID to trace the original model folder path during loading. There is a known issue where the model's ID does not always match the folder path.model.json 📄:
sources: Contains model files, used during downloading:
filename: The filename persisted after download, which should match llama_model_path
if defined. Otherwise, the filename is used as llama_model_path
for GGUF models (llama.cpp engine).
url: The download URL for the model file.
id: The model ID described above, used for inference requests and model folder lookups.
object: OpenAI-compatible model object field.
name: Display name for the model (in model hub, model selection dropdown, and "my model" section).
version: Used during migrations. Newer versions of model.json
from extensions will overwrite older user data folder versions.
description: Displayed in the model hub.
format: GGUF or API, indicating whether it is a GGUF model or a remote model (API is mandatory and referenced in the codebase, especially in the model hub to filter remote models).
engine: Determines the engine/provider used for model load/unload or inference.
settings: Model load parameters:
llama_model_path: GGUF model filename.
prompt_template: Template used for parsing prompts.
ctx_len: Context length setting.
ngl: Number of GPU layers. This value is declared in the GGUF model file and varies by model.
embedding: Indicates if the model can be used as a text embedding model.
cpu_threads: Number of CPU threads for loading the model.
mmproj: MMPROJ file name for CLIP (multimodal model, vision).
cont_batching: Enables continuous batching.
vision_model: Indicates if the model is a vision model (supports multimodal vision).
parameters: Inference parameters:
temperature: Controls the model's risk-taking (usually between 0 and 1, but can be higher).
token_limit: Maximum possible token output.
stream: Determines whether the response uses event streams.
stop: Specifies stop words for halting output.
frequency_penalty: Applies a penalty based on token repetition.
presence_penalty: Applies a penalty for repeated tokens, similar to frequency_penalty but applies equally to all repeated tokens.
top_p: A sampling technique to control model determinism.
top_k: Limits the model's output to the top-k most probable tokens.
settings 📁: Created when the monitoring-extension
is loaded.
[extension-id] 📁: Stores extension settings with a similar structure to extensions.
settings.json 📄: Stores the extension setting values:
settings.json 📄: Stores app settings related to GPU acceleration. Generated automatically by the monitoring-extension
and cannot be edited, as it is overwritten based on hardware detection logic. This includes NVIDIA-SMI queries to check for drivers and .dll/.so
lookups to determine the CUDA version.
themes 📁: Created on app launch (App level).
threads 📁: Created when the conversational-extension
is loaded.
ulid()
.model.json
. Also contains the ID and engine of the selected model for quick querying by extensions, as extensions do not have access to all available models (due to isolation).lastMessage
, which provides GUI information but does not use OpenAI-compatible fields. {"id":"01J6Y6FH8PFTHQB5PNJTHEN27C","thread_id":"jan_1725437954","type":"Thread","role":"assistant","content":
[{"type":"text","text":{"value":"Hello! Is there something I can help you with or would you like to chat?","annotations":
[]}}],"status":"ready","created":1725442802966,"updated":1725442802966,"object":"thread.message"}
computed
fields (e.g. ID, object, format, extensions.json) that could be stripped out.save instructions
setting provided.There should be different
extensions.json
but if we scan extensions folder instead of reading metadata file we can considerpackage.json
engine field where it could work with Jan or Cortex.
WIP
What's the end goal for Jan application state?
User data, e.g. threads
, assistants
.
Cortex level data, e.g. models
, engine
configs?
~/.cortexcpp
into ~/.jan
Will Jan have separate folders for stable vs nightly, similar to cortex/
hi @0xSage ,
threads, assistants, messages
files structure in Cortex Platform to avoid the migration process
Overview
We need to document what current Jan "data structures" are:
We also need to define what our future state looks like:
Decisions to make
inference-cortex-extension
)Linked Issues
cortex.cpp
's data structures1654