Exe Engine

[amolkittur]

Equity Module Testing

Description

We need to perform comprehensive testing of the Equity module, focusing on the short-term, mid-term, and long-term modules. The primary objectives are to ensure that orders are placed correctly, logged accurately, and properly written to the database under holdings.

Tasks

1. Test the short-term module

   • Verify order placement
   • Check logging functionality
   • Confirm database entries in holdings

2. Test the mid-term module

   • Verify order placement
   • Check logging functionality
   • Confirm database entries in holdings

3. Test the long-term module

   • Verify order placement
   • Check logging functionality
   • Confirm database entries in holdings

Acceptance Criteria

• [ ] All the files/db are auto generated in the location mentioned in the env file
• [ ] All three modules (short-term, mid-term, long-term) have been thoroughly tested
• [ ] Orders are placed correctly for each module
• [ ] All order details are logged accurately in the firebase db
• [ ] Holdings are correctly written to the sqlite database
• [ ] Any discrepancies or bugs are documented and reported

Additional Notes

• Please include screenshots or logs of any issues encountered during testing
• Document any unexpected behavior, even if it doesn't appear to be a bug
• Consider edge cases and stress testing scenarios

[Serendipity2-0]

@amolkittur Its possible to get a better documentation like this: https://docs.docker.com/engine/api/v1.25/#tag/Image refer to: https://fastapi.tiangolo.com/#alternative-api-docs-upgrade to how to do this for our project. Include this in your story.

[amolkittur]

Sections Involved:

1. Onboarding of Clients
2. Execution (Exe) Engine
3. Database Logging

---

1. Onboarding of Clients

• Description: The onboarding process for new clients is successfully implemented. This involves capturing and storing client details, ensuring compliance with regulatory requirements, and setting up initial configurations.

---

2. Execution (Exe) Engine

The Exe Engine is divided into three phases:

A. Before Market

1. Login to Broker
   • Description: Successfully implemented login functionality for respective brokers.

2. Funds Validator

   • Description:
     • Fetch free cash from the broker.
     • Update the portfolio section in the accounts section of the Firebase DB.
     • Split the free cash based on the capital allocation of each segment.
     • Update the free cash with today's date as a prefix.
     • Example Portfolio Dict:

       {
       "18Jul24_Portfolio_AccountValue": 141559,
       "18Jul24_Portfolio_FreeCash": 141558.6,
       "18Jul24_Portfolio_Holdings": 81028
       }

     • Example Segment Dict:

       {
       "Equity": {
       "18Jul24_Equity_AccountValue": 141559,
       "18Jul24_Equity_FreeCash": 141558.6,
       "18Jul24_Equity_Holdings": 81028
       }
       }

3. Market Info Endpoint

   • Description:
     • Create an endpoint for the admin page to update and log market information values.
     • Example Market Info Dict:

       {
       "EquityQtyAmplifier": 1,
       "OBQtyAmplifier": 2,
       "OSQtyAmplifier": 1,
       "TradeView": "Bullish"
       }

4. Daily Instrument Aggregator

   • Status: Completed
   • Description: Successfully implemented the daily instrument aggregator to consolidate and update daily trading instruments.

B. During Market

1. Running Strategy Scripts
   • Description:
     • Execute strategy scripts during market hours.
     • Log parameters such as entry points and strategy parameters in the Firebase DB.
     • We can use this to do the testing of the strategy(https://github.com/Serendipity2-0/TradeManV1/issues/42#issue-2407899935)

2. Calculate Quantity

   • Description:
     • Calculate the quantity based on risk, capital allocation, and other factors.
     • Fetch free cash and calculate the allocation for each strategy.
     • Update the quantity in the strategy dictionary.
     • Example Strategy Dict:

       {
       "Strategy1": {
       "Qty": 28,
       "CapitalAllocation": 50,
       "StrategyName": "PyStocks",
       "TradeState": {
        "orders": [
          {
            "avg_prc": 83.2,
            "exchange_token": 9881,
            "message": "PASS",
            "order_id": "240212501608640",
            "qty": 321,
            "time_stamp": "2024-02-12 10:12",
            "trade_id": "PS2_LG_MO_EN"
          }
        ]
       }
       }
       }

3. Place Orders

   • Description:
     • Use the order details dictionary provided by the strategy script to place orders.

4. Trade State Verification

   • Description:
     • Ensure the trade state is updated correctly with all relevant values.

C. After Market

1. Database Logging Verification

   • Description:
     • Verify the logged values in the DB, such as trade ID, entry, exit, time, price, PnL, tax, etc.

2. Holdings Collection Verification

   • Description:
     • Check the values in the Holdings collection in the database for accuracy and completeness.

---

3. Database Logging

• Description: Ensure all trading activities and associated data are logged accurately in the database for compliance, analysis, and auditing purposes.

---

Task Division

---

1. Admin

• [ ] Make sure all the values in accounts, broker, profile and strategies are filled in the user json in firebase
• [ ] Make sure all the parameters required to run the strategy are updated in the Firebase strategy collection
• [ ] Update the env file for the system in which we will run the main code

2. Backend

• [ ] Login into respective broker
• [ ] Funds validator
• [ ] Market info params
• [ ] Instrument aggregator
• [ ] Executing the strategies at the correct time
• [ ] Placing the orders
• [ ] Write the TradeState at the correct path
• [ ] Log the tradestate with the correct values in the sql db
• [ ] Log the holdings to the db

Fast API

• [ ] admin/login
• [ ] admin/tradestate
• [ ] admin/error-monitor
• [ ] admin/signal-log
• [ ] admin/modify-strategy-param
• [ ] admin/modify-market-info
• [ ] admin/modify-user-risk
• [ ] admin/send-notification
• [ ] admin/logout
• [ ] user/login
• [ ] user/profile
• [ ] user/portfolio-view
• [ ] user/strategy-data
• [ ] user/holdings
• [ ] user/transactions
• [ ] user/logout

3. Frontend

• [ ] Admin page with all the tabs
• [ ] User page to display all the data

[amolkittur]

@amolkittur Its possible to get a better documentation like this: https://docs.docker.com/engine/api/v1.25/#tag/Image refer to: https://fastapi.tiangolo.com/#alternative-api-docs-upgrade to how to do this for our project. Include this in your story.

I will check and update the swagger/docs into a new look

[amolkittur]

Endpoints for admin pages

• [ ] Endpoints for admin pages(list of users, json of the user and update it.

[amolkittur]

Funds Validator

• Description:
  • [ ] Fetch free cash from the broker.
  • [ ] Update the portfolio section in the accounts section of the Firebase DB.
  • [ ] Update the free cash with today's date as a prefix.
  • Example Portfolio Dict:

    {
     "18Jul24_Portfolio_AccountValue": 141559,
     "18Jul24_Portfolio_FreeCash": 141558.6,
     "18Jul24_Portfolio_Holdings": 81028
    }

[amolkittur]

As discussed with @omkarh25 , we will update the base capital field in the accounts section and split the capital based on the Capital Allocation during the onboarding process. After the freecash values have been updated in the segment, we start trading from the freecash available in the segment and maintain the account separately for each segment.

For quantity calculation, we can directly fetch the freecash from the segment, calculate the quantity, and trade with the calculated quantity. At the end of the day (EOD), we calculate the P&L, holdings, and update the values in the segment.

[amolkittur]

Today we celebrate a key milestone: our system's first successful order. This achievement marks the beginning of an exciting phase of enhancements and growth for our project.

Major error

• Funds allocation for segment and terms
• qty calculation based on the allocated capital

[amolkittur]

Phase 2 Enhancement Docs

This document is for planned enhancements for our software across Admin, Backend, and Frontend components to improve functionality and user experience over the next 2 weeks.

1. Admin

1. Setup Env files for prod, staging and dev

   • Create environment-specific configuration files to manage settings across different deployment stages.

2. Update the admin section with the primary accounts

   • Zerodha and Firstock primary account broker details should be uploaded in the admin section and use it for daily login

3. Update the primary accounts with the live brokers (Zerodha and Firstock)

   • Check for the brokers which are used for day to day trading (AliceBlue can be removed)

4. Logic for Funds Validation at the portfolio level

   • Implement a system to verify and validate funds across the entire portfolio.

5. Qty calculation (Done)

   • Quantity calculation feature has been completed and implemented.

6. Instrument Aggregator

   • Logic to replace aliceblue feature which were used for developing the instrument db and replace it with the latest brokers(Firstock)

2. Backend

1. Separate script for equity calculation which needs to be executed before the market hours

   • Develop a standalone script to perform equity calculations prior to market opening.

2. Schedule the tasks (short, mid and long term) i.e daily, weekly and monthly

   • Set up automated task scheduling for various timeframes to ensure regular system updates and maintenance.

3. Error handling logic for new user while placing stoploss

   • Implement robust error management for new users when they set stop-loss orders because the strategy collection wont be present in the db when running the script for the first time.

4. Eod report should include all the opted segments

   • Expand end-of-day reports to cover all segments that users have opted into(Equity and Derivatives).

5. Refactoring the fast api endpoints used for onboarding process

   • Include Tr_No in the firebase
   • Update the Accounts section with the latest format(without the dates)
   • Add BaseCapital to the accounts section
   • Validation to check wether all the fields are present in the dict which is being uploaded

6. Add indicators/strategies to mid and long term

   • As of now only 2 strategies are present in mid and long term

3. Frontend

1. Order executor endpoints

   • Develop API endpoints for executing trading orders from the frontend.

2. Endpoint to modify the orders/stoploss

   • Create an API endpoint allowing users to adjust existing orders and stop-loss settings.

3. Send notice to all users

   • Implement a system-wide notification feature to broadcast important messages to all users.

4. Admin dashboard page (Should include AUM/ graphs)

   • Design and implement an admin dashboard displaying key metrics like AUM and relevant graphs.

5. Live pnl and live positions for users

   • Develop real-time display of profit/loss and current positions for each user.

[Serendipity2-0]

@amolkittur Closing this issue. Hope you have transferred all the remaining tasks to phase 2. Well done!