redis / redis-om-python

Object mapping, and more, for Redis and Python
MIT License
1.12k stars 112 forks source link

Best Practices for Multi-Language Schema Source of Truth with Redis OM? #263

Closed ryanrussell closed 2 years ago

ryanrussell commented 2 years ago

Hi,

Love the idea of a First Class ORM around Redis. You guys are doing exciting things!

Goal

Find a best-practice mechanism to support Redis OM schema with a single source of truth between languages(Say Python and Node)

Question

In Python if we define an Embedded JSON Model object, how would we consume/interact in Node without a secondary schema definition to maintain in JS/TS?

Example

Consider an Embedded JSON Model structure in python like:

from redis_om import EmbeddedJsonModel, Field, JsonModel, Migrator

class Address(EmbeddedJsonModel):
    address1: str
    city: str = Field(index=True)
    state: str = Field(index=True)
    postal_code: str = Field(index=True)

class Ticker(JsonModel):
    ticker: str = Field(index=True)
    name: str = Field(index=True)
    market: str = Field(index=True)
    locale: str = Field(index=True)
    primary_exchange: str = Field(index=True)
    type: str = Field(index=True)
    active: bool = Field(index=True)
    currency_name: str = Field(index=True)
    cik: str = Field(index=True)
    composite_figi: str
    share_class_figi: str
    market_cap: float = Field(index=True)
    phone_number: str
    address: Address

It would be ideal to perform CRUD operations using Redis Om Node in addition to python without maintaining a separate schema definition.

The concern is without a single schema definition shared across languages there will be nasty days around migrations; especially if/when these are in a distributed consumer architecture with multiple languages accessing the objects.

Solutions?

It feels like mapping a TS file to other languages could answer the bell? Maybe there is a TS -> Python(.NET, Rust, etc) schema mapping utility for Redis OM already or in the roadmap?

Greatly appreciate any pointers if there is a best practice, pattern or tutorial that covers how to do this well.

Thank you in Advance, Ryan

marianhlavac commented 2 years ago

One of the more standard solutions might be using JSON Schema to define the schemas and use datamodel-code-generator to generate Pydantic models, and do the same on the TypeScript side. Check out the Pydantic docs on this topic.

ryanrussell commented 2 years ago

Thanks @marianhlavac