Closed Sandy-at-Tazama closed 2 months ago
frms-coe-lib
is a foundational library designed to provide common functionalities and interfaces for the FRMS (Fraud Risk Management System) ecosystem. It includes utilities, data structures, and interfaces that support various components of the system. The library offers a range of features, including database management, logging, configuration management, rule evaluation, and message handling. It serves as a core dependency for other FRMS components, providing essential building blocks and standardized approaches for handling data and interactions.
Key features:
To install the frms-coe-lib
package, you can use npm. This library requires Node.js and npm to be installed on your system.
Install via npm:
npm install @frmscoe/frms-coe-lib --registry=https://npm.pkg.github.com
The npm package is hosted on GitHub. Make sure you're authenticated with GitHub and have the necessary permissions to access the package (read:packages
).
Once installed, you can import the library in your project:
import { LoggerService, CreateDatabaseManager } from '@frmscoe/frms-coe-lib';
Dependencies:
Ensure that you have all required dependencies installed, including any specific versions of third-party packages mentioned in the package's peer dependencies.
Environment Configuration:
Set up your environment configuration using a .env
file or environment variables. Refer to the library's configuration requirements for details on necessary environment variables.
The frms-coe-lib
library provides various functionalities for transaction monitoring, logging, and database management. Below is an overview of how to use the key features of the library.
The CreateDatabaseManager
function initializes and manages connections to multiple databases, including ArangoDB and Redis. This function returns an instance of DatabaseManagerInstance
which includes methods to interact with the databases.
Example:
import { CreateDatabaseManager, DatabaseManagerInstance } from '@frmscoe/frms-coe-lib';
const dbConfig = {
configuration: {
url: 'database-url',
databaseName: 'database-name',
user: 'username',
password: 'password',
certPath: 'path-to-cert',
},
redisConfig: {
db: 0,
servers: [{ host: 'localhost', port: 6379 }],
password: 'redis-password',
isCluster: false,
},
};
let databaseManager: DatabaseManagerInstance<typeof dbConfig>;
async function initDB() {
databaseManager = await CreateDatabaseManager(dbConfig);
}
initDB();
The LoggerService
class provides logging functionality, supporting different log levels like info
, debug
, warn
, and error
. It can also log messages to a GRPC service.
Example:
import { LoggerService } from '@frmscoe/frms-coe-lib';
const logger = new LoggerService('localhost:50051');
logger.log('This is an info message');
logger.warn('This is a warning');
logger.error(new Error('This is an error message'));
The Apm
class integrates with Elastic APM to track performance and errors. It provides methods to start transactions and spans.
Example:
import { Apm } from '@frmscoe/frms-coe-lib';
const apm = new Apm({
serviceName: 'my-service',
secretToken: 'apm-secret-token',
serverUrl: 'apm-server-url',
active: true,
});
const transaction = apm.startTransaction('transaction-name');
const span = apm.startSpan('span-name');
// Do something
span.end();
transaction.end();
The RedisService
class provides methods to interact with Redis, including setting and getting JSON data, managing sets, and handling binary data.
Example:
import { RedisService } from '@frmscoe/frms-coe-lib';
async function useRedis() {
const redisConfig = {
db: 0,
servers: [{ host: 'localhost', port: 6379 }],
password: 'redis-password',
isCluster: false,
};
const redisService = await RedisService.create(redisConfig);
await redisService.setJson('key', JSON.stringify({ field: 'value' }), 300);
const value = await redisService.getJson('key');
console.log(value);
}
useRedis();
ProtoBuf Module
ProtoGrpcType
google.protobuf.Empty
: Represents an empty message.lumberjack.LogLevel
: Enum representing log levels.lumberjack.LogMessage
: Represents a log message.lumberjack.Lumberjack
: Represents the Lumberjack service with methods like SendLog
.Logger Service
LoggerService
log(message: string, serviceOperation?: string, id?: string, callback?: LogCallback): void
: Logs a message.debug(message: string, serviceOperation?: string, id?: string, callback?: LogCallback): void
: Logs a debug message.trace(message: string, serviceOperation?: string, id?: string, callback?: LogCallback): void
: Logs a trace message.warn(message: string, serviceOperation?: string, id?: string, callback?: LogCallback): void
: Logs a warning message.error(message: string | Error, innerError?: unknown, serviceOperation?: string, id?: string, callback?: LogCallback): void
: Logs an error message.fatal(message: string | Error, innerError?: unknown, serviceOperation?: string, id?: string, callback?: LogCallback): void
: Logs a fatal error message.Database Manager
DatabaseManager
CreateDatabaseManager<T>(config: T): Promise<DatabaseManagerInstance<T>>
: Creates a database manager instance.isReadyCheck(): any
: Checks if the database services are ready.quit(): void
: Closes all database connections.Apm Service
startTransaction(name: string, options?: TransactionOptions): apm.Transaction | null
: Starts a new transaction.startSpan(name: string): apm.Span | null
: Starts a new span.getCurrentTraceparent(): string | null
: Retrieves the current traceparent.Redis Service
getJson(key: string): Promise<string>
: Retrieves a JSON value from Redis.getBuffer(key: string): Promise<Record<string, unknown>>
: Retrieves a buffer value from Redis.getMemberValues(key: string): Promise<Array<Record<string, unknown>>>
: Retrieves members of a Redis set.deleteKey(key: string): Promise<void>
: Deletes a key from Redis.setJson(key: string, value: string, expire: number): Promise<void>
: Sets a JSON value in Redis with an expiration time.set(key: string, value: RedisData, expire: number): Promise<void>
: Sets a value in Redis with an expiration time.setAdd(key: string, value: Record<string, unknown>): Promise<void>
: Adds a value to a Redis set.addOneGetAll(key: string, value: Record<string, unknown>): Promise<Array<Record<string, unknown>>>
: Adds a value to a Redis set and retrieves all members.addOneGetCount(key: string, value: Record<string, unknown>): Promise<number>
: Adds a value to a Redis set and retrieves the count of members.quit(): void
: Closes the Redis connection.Protobuf Utilities
createMessageBuffer(data: Record<string, unknown>): Buffer | undefined
: Creates a message buffer from a data object.createLogBuffer(data: Record<string, unknown>): Buffer | undefined
: Creates a log buffer from a data object.decodeLogBuffer(buffer: Buffer): LogMessageType | undefined
: Decodes a log buffer into a LogMessageType
.Unwrap Utility
unwrap<T>(type: T[][]): T | undefined
: Unwraps a 2D array and returns the item at [0][0]
.The frms-coe-lib
library uses environment variables to configure various components. Here are the key environment variables:
DATABASE_URL
: The URL for the database.DATABASE_USER
: The username for the database.DATABASE_PASSWORD
: The password for the database.DATABASE_NAME
: The name of the database.DATABASE_CERT_PATH
: The path to the database certificate.REDIS_DB
: The Redis database number.REDIS_SERVERS
: The Redis servers, specified as a JSON string.REDIS_AUTH
: The Redis password.REDIS_IS_CLUSTER
: Specifies whether Redis is in cluster mode (true
or false
).LOGSTASH_HOST
: The Logstash host.LOGSTASH_PORT
: The Logstash port.LOGSTASH_LEVEL
: The log level for Logstash.NODE_ENV
: The node environment (e.g., development
, production
).MAX_CPU
: The maximum number of CPUs to use.In addition to environment variables, the library can be configured using configuration files. These files can be used to set up database connections, logging configurations, and other settings.
The logging configuration can be set through environment variables or configuration files. The following options are available:
info
, debug
, error
) using the LOGSTASH_LEVEL
environment variable.Database connections are configured using the DATABASE_URL
, DATABASE_USER
, DATABASE_PASSWORD
, and other related environment variables. The database can be an SQL or NoSQL database, depending on the application's requirements.
Redis is used for caching and other purposes. The REDIS_DB
, REDIS_SERVERS
, REDIS_AUTH
, and REDIS_IS_CLUSTER
environment variables are used to configure the Redis connection.
For advanced users, the library provides options to customize the behavior of certain components. These options can be set through additional configuration files or environment variables.
import { Database } from 'arangojs';
import NodeCache from 'node-cache';
import protobuf from 'protobufjs';
import apm, { type AgentConfigOptions, type TransactionOptions } from 'elastic-apm-node';
import Redis, { type Cluster } from 'ioredis';
import { type DestinationStream, pino } from 'pino';
import pinoElastic, { type DestinationStream } from 'pino-elasticsearch';
import { ecsFormat } from '@elastic/ecs-pino-format';
import * as grpc from '@grpc/grpc-js';
import * as protoLoader from '@grpc/proto-loader';
import { v4 } from 'uuid';
.env
file into process.env
.import { config as dotenv } from 'dotenv';
import path from 'node:path';
import fs from 'fs';
If you want to contribute to the frms-coe-lib
, please clone the repository and submit a pull request to the dev
branch.
This library is a component of the Tazama project. The Tazama project is licensed under the Apache 2.0 License.
User Story Statement
As a Tazama system developer I want a template for updating the frms-coe-lib readme documentation; so that I know what to document; and so that the documentation is consistent and complete; and so that the system is as simple as possible to maintain and configure for an inexperienced user
As a Tazama product owner I want the coe-lib readme.md updated to align to the template and guide provided below so that the documentation is consistent and complete; and so that the system is as simple as possible to maintain and configure for an inexperienced user