search

tazama-lf / frms-coe-lib

FRMS Center of Excellence package library
Apache License 2.0
3 stars 3 forks source link

coe-lib documentation update #98

Closed Sandy-at-Tazama closed 2 months ago

Sandy-at-Tazama commented 5 months ago

User Story Statement

As a Tazama system developer I want a template for updating the frms-coe-lib readme documentation; so that I know what to document; and so that the documentation is consistent and complete; and so that the system is as simple as possible to maintain and configure for an inexperienced user

As a Tazama product owner I want the coe-lib readme.md updated to align to the template and guide provided below so that the documentation is consistent and complete; and so that the system is as simple as possible to maintain and configure for an inexperienced user

Justus-at-Tazama commented 2 months ago

Table of Contents

Overview

frms-coe-lib is a foundational library designed to provide common functionalities and interfaces for the FRMS (Fraud Risk Management System) ecosystem. It includes utilities, data structures, and interfaces that support various components of the system. The library offers a range of features, including database management, logging, configuration management, rule evaluation, and message handling. It serves as a core dependency for other FRMS components, providing essential building blocks and standardized approaches for handling data and interactions.

Key features:

Installation

To install the frms-coe-lib package, you can use npm. This library requires Node.js and npm to be installed on your system.

  1. Install via npm:

    npm install @frmscoe/frms-coe-lib --registry=https://npm.pkg.github.com

The npm package is hosted on GitHub. Make sure you're authenticated with GitHub and have the necessary permissions to access the package (read:packages).

  1. Importing in your project:

Once installed, you can import the library in your project:

  import { LoggerService, CreateDatabaseManager } from '@frmscoe/frms-coe-lib';

  1. Dependencies:

    Ensure that you have all required dependencies installed, including any specific versions of third-party packages mentioned in the package's peer dependencies.

  2. Environment Configuration:

    Set up your environment configuration using a .env file or environment variables. Refer to the library's configuration requirements for details on necessary environment variables.

Usage

The frms-coe-lib library provides various functionalities for transaction monitoring, logging, and database management. Below is an overview of how to use the key features of the library.

1. Database Management

The CreateDatabaseManager function initializes and manages connections to multiple databases, including ArangoDB and Redis. This function returns an instance of DatabaseManagerInstance which includes methods to interact with the databases.

Example:

import { CreateDatabaseManager, DatabaseManagerInstance } from '@frmscoe/frms-coe-lib';

const dbConfig = {
  configuration: {
    url: 'database-url',
    databaseName: 'database-name',
    user: 'username',
    password: 'password',
    certPath: 'path-to-cert',
  },
  redisConfig: {
    db: 0,
    servers: [{ host: 'localhost', port: 6379 }],
    password: 'redis-password',
    isCluster: false,
  },
};

let databaseManager: DatabaseManagerInstance<typeof dbConfig>;

async function initDB() {
  databaseManager = await CreateDatabaseManager(dbConfig);
}

initDB();

2. Logger Service

The LoggerService class provides logging functionality, supporting different log levels like info, debug, warn, and error. It can also log messages to a GRPC service.

Example:

import { LoggerService } from '@frmscoe/frms-coe-lib';

const logger = new LoggerService('localhost:50051');

logger.log('This is an info message');
logger.warn('This is a warning');
logger.error(new Error('This is an error message'));

3. Apm Integration

The Apm class integrates with Elastic APM to track performance and errors. It provides methods to start transactions and spans.

Example:

import { Apm } from '@frmscoe/frms-coe-lib';

const apm = new Apm({
  serviceName: 'my-service',
  secretToken: 'apm-secret-token',
  serverUrl: 'apm-server-url',
  active: true,
});

const transaction = apm.startTransaction('transaction-name');
const span = apm.startSpan('span-name');

// Do something

span.end();
transaction.end();

4. Redis Service

The RedisService class provides methods to interact with Redis, including setting and getting JSON data, managing sets, and handling binary data.

Example:

import { RedisService } from '@frmscoe/frms-coe-lib';

async function useRedis() {
  const redisConfig = {
    db: 0,
    servers: [{ host: 'localhost', port: 6379 }],
    password: 'redis-password',
    isCluster: false,
  };

  const redisService = await RedisService.create(redisConfig);
  await redisService.setJson('key', JSON.stringify({ field: 'value' }), 300);
  const value = await redisService.getJson('key');
  console.log(value);
}

useRedis();

Modules and Classes

  1. ProtoBuf Module

    • Class: ProtoGrpcType
    • Description: Contains definitions related to Google Protocol Buffers for message types.
    • Methods:
      • google.protobuf.Empty: Represents an empty message.
      • lumberjack.LogLevel: Enum representing log levels.
      • lumberjack.LogMessage: Represents a log message.
      • lumberjack.Lumberjack: Represents the Lumberjack service with methods like SendLog.

  2. Logger Service

    • Class: LoggerService
    • Description: Provides logging capabilities, including sending logs to Lumberjack via gRPC or using Pino for ElasticSearch.
    • Methods:
      • log(message: string, serviceOperation?: string, id?: string, callback?: LogCallback): void: Logs a message.
      • debug(message: string, serviceOperation?: string, id?: string, callback?: LogCallback): void: Logs a debug message.
      • trace(message: string, serviceOperation?: string, id?: string, callback?: LogCallback): void: Logs a trace message.
      • warn(message: string, serviceOperation?: string, id?: string, callback?: LogCallback): void: Logs a warning message.
      • error(message: string | Error, innerError?: unknown, serviceOperation?: string, id?: string, callback?: LogCallback): void: Logs an error message.
      • fatal(message: string | Error, innerError?: unknown, serviceOperation?: string, id?: string, callback?: LogCallback): void: Logs a fatal error message.

  3. Database Manager

    • Class: DatabaseManager
    • Description: Manages database connections and interactions, including configuration and pseudonyms databases.
    • Methods:
      • CreateDatabaseManager<T>(config: T): Promise<DatabaseManagerInstance<T>>: Creates a database manager instance.
      • isReadyCheck(): any: Checks if the database services are ready.
      • quit(): void: Closes all database connections.

  4. Apm Service

    • Class: Apm
    • Description: Provides APM (Application Performance Management) integration using Elastic APM.
    • Methods:
      • startTransaction(name: string, options?: TransactionOptions): apm.Transaction | null: Starts a new transaction.
      • startSpan(name: string): apm.Span | null: Starts a new span.
      • getCurrentTraceparent(): string | null: Retrieves the current traceparent.

  5. Redis Service

    • Class: RedisService
    • Description: Provides methods for interacting with Redis, including setting and getting data.
    • Methods:
      • getJson(key: string): Promise<string>: Retrieves a JSON value from Redis.
      • getBuffer(key: string): Promise<Record<string, unknown>>: Retrieves a buffer value from Redis.
      • getMemberValues(key: string): Promise<Array<Record<string, unknown>>>: Retrieves members of a Redis set.
      • deleteKey(key: string): Promise<void>: Deletes a key from Redis.
      • setJson(key: string, value: string, expire: number): Promise<void>: Sets a JSON value in Redis with an expiration time.
      • set(key: string, value: RedisData, expire: number): Promise<void>: Sets a value in Redis with an expiration time.
      • setAdd(key: string, value: Record<string, unknown>): Promise<void>: Adds a value to a Redis set.
      • addOneGetAll(key: string, value: Record<string, unknown>): Promise<Array<Record<string, unknown>>>: Adds a value to a Redis set and retrieves all members.
      • addOneGetCount(key: string, value: Record<string, unknown>): Promise<number>: Adds a value to a Redis set and retrieves the count of members.
      • quit(): void: Closes the Redis connection.

  6. Protobuf Utilities

    • Functions:
    • createMessageBuffer(data: Record<string, unknown>): Buffer | undefined: Creates a message buffer from a data object.
    • createLogBuffer(data: Record<string, unknown>): Buffer | undefined: Creates a log buffer from a data object.
    • decodeLogBuffer(buffer: Buffer): LogMessageType | undefined: Decodes a log buffer into a LogMessageType.

  7. Unwrap Utility

    • Functions:
    • unwrap<T>(type: T[][]): T | undefined: Unwraps a 2D array and returns the item at [0][0].

Configuration

Environment Variables

The frms-coe-lib library uses environment variables to configure various components. Here are the key environment variables:

Configuration Files

In addition to environment variables, the library can be configured using configuration files. These files can be used to set up database connections, logging configurations, and other settings.

Logging Configuration

The logging configuration can be set through environment variables or configuration files. The following options are available:

Database Configuration

Database connections are configured using the DATABASE_URL, DATABASE_USER, DATABASE_PASSWORD, and other related environment variables. The database can be an SQL or NoSQL database, depending on the application's requirements.

Redis Configuration

Redis is used for caching and other purposes. The REDIS_DB, REDIS_SERVERS, REDIS_AUTH, and REDIS_IS_CLUSTER environment variables are used to configure the Redis connection.

Advanced Configuration

For advanced users, the library provides options to customize the behavior of certain components. These options can be set through additional configuration files or environment variables.

External Dependencies

1. arangojs

2. node-cache

3. protobufjs

4. elastic-apm-node

5. ioredis

6. pino

7. pino-elasticsearch

8. @elastic/ecs-pino-format

9. @grpc/grpc-js

10. @grpc/proto-loader

11. uuid

12. dotenv

13. node:path

14. fs

Contributing

If you want to contribute to the frms-coe-lib, please clone the repository and submit a pull request to the dev branch.

License

This library is a component of the Tazama project. The Tazama project is licensed under the Apache 2.0 License.