Cloud-Harvest / CloudHarvestApi

interface between clients, the server cache, and other apis
Other
0 stars 0 forks source link

Harvest Data Silo Naming Conventions #44

Open fiona-june-leathers opened 3 weeks ago

fiona-june-leathers commented 3 weeks ago

We need to establish a standard naming convention for data components in Harvest. The current model establishes two names: ephemeral and persistent, but this is ultimately inflexible. Instead, we should establish different silo names for different components which (theoretically) could live in different places, even if they are all colocated in the same physical database instance.

For harvest-based systems, we should define it like: harvest-(function|(plugin)-(plugin-name))

Examples:

Name Engine Purpose
harvest-agents Redis Where task executors are listed, monitored, and used.
harvest-core Mongo Defines the location of the database used to administer the application and its metadata.
harvest-jobs Redis This shows active and queued tasks that executors process.
harvest-plugin-aws Mongo Where AWS data will live.
harvest-plugin-azure Mongo Where Azure data will live.
harvest-tokens Redis Ephemeral user tokens.
harvest-users Mongo Defines the location of the Harvest user accounts and their associated privileges.

This means expanding CloudHarvestApi setup routine with these components.

fiona-june-leathers commented 10 hours ago

We need a Redis harvest-agent-results so Task results are stored where users can retrieve them later.