leapfrogtechnology / datamegh

Datamegh - Data Engineering for the cloud.
MIT License
2 stars 1 forks source link

Setup framework API packages #7

Closed kabirbaidhya closed 4 years ago

kabirbaidhya commented 4 years ago

Need to setup a basic structure for standard APIs exposed by datamegh.

Possible Structure

# Base Package for all the APIs.
datamegh.api

# Storage
datamegh.api.storage # Package for all the storage providers.

from datamegh.api.storage import s3

# Database
datamegh.api.db 

from datamegh.api.db import db, Model

# DB
db.insert(connection, table, data)
db.find_all(connection, table, [fields])
db.delete(connection, table, params)

# Model

class Employee extends Model:
    pass

list = Employee.find_all()

# Traansaction

trx = db.transaction()

db.insert(trx, 'employee', {...})
db.insert(trx, 'employee_roles', {...})

trx.commit()

# Queue

datamegh.api.queue

from datamegh.api.queue import sqs

# IPC / RPC

datamegh.api.http

from datamegh.api import http

http.request(method, config)

http.get(url)
http.post(url, payload, params)
http.put(url, params)
http.delete(url, params)

# Secret Managers

datamegh.api.scerets

from datamegh.api.secrets import vault, secret_manager

image