finos-labs / emir-specific-rag

Apache License 2.0
7 stars 5 forks source link

Generate & Gather CDM Model Training Data #17

Open karlmoll opened 3 months ago

karlmoll commented 3 months ago

Collaboration Call to Action

Description of Problem:

The AI4Finance team has the domain expertise and resources to train, fine-tune, and benchmark LLMs on CDM, but they do not have domain expertise in order to gather existing data/documentation/faqs/etc or to generate a list of "reference meaningful fact based questions & answers" in order to provide a source of truth in their training efforts.

Potential Solutions:

  1. Need to continue to provide additional CDM documentation
  2. Provide a list of 100-200 reference questions and answers
  3. Host workshop with domain experts to generate additional Q&A's & training data
  4. Synthetic Data

Note: Need to generate V1 ideally by end of third week of August in order to review and get it to the AI4Finance team so that they can begin training with time for OSFF.

lucaborella89 commented 3 months ago

@manel-martos

manel-martos commented 3 months ago

CDM example java project - navigate through rosetta-source/src/main/resources/result-json-files to find test pack samples for anonymized CDM trade representations coming from different contributions.

Initially I would suggest you use the fpml-5-XX/products samples since they have a good coverage across different asset classes and products. These are TradeState CDM instances. Please find docs link that you might find helpful to get further details. FINOS CDM documentation

lucaborella89 commented 3 months ago

Questions Examples:

manel-martos commented 1 month ago

I'm adding a collection of question samples we got during CDM/DRR trainings and modeling. I know they may not address completely your request but at least they show the main interests we could observe for participants to extend the model or create their own implementations. We didn't add the answers since we had no capacity but all questions should be found in the CDM Documentation portal and other resources that will get published by FINOS in the near future. I also asked some organizations for public FAQ around CDM without success.

  1. How does product qualification (payout) work, and what attributes influence it?
  2. How is the lifecycle of a financial product, such as an interest rate swap, represented?
  3. What is a WorkflowStep, and how is it related to a business event?
  4. How does the model handle complex trade structures like baskets or portfolios?
  5. What is a Business event, and how is it used in the context of trade processing?
  6. What is a Trade State, and how does it evolve through the lifecycle of a transaction?
  7. How does the model manage the workflow of corporate actions, such as dividends or stock splits?
  8. How does the qualification of events work, and how can it be related to reporting action types?
  9. How does the qualification of the product work and what are the main ones?
  10. What are the key challenges in implementing a standardized model across an organization?
  11. How does the system ensure data consistency across different lifecycle events of a trade?
  12. What role does the model play in standardizing trade representation across different systems?
  13. How does the model handle the differences between cleared and uncleared trades?
  14. What is the process for novation, and what are the differences between partial novation and full novation?
  15. How does the model facilitate cross-border trading and reporting?
  16. How is collateral management represented, and what are the key processes involved?
  17. What role does technology, such as distributed ledger or smart contracts, play in automating processes?
  18. How does the model manage regulatory requirements across different jurisdictions?
  19. How does the model improve the accuracy and timeliness of data submissions?
  20. How does the model integrate with existing market infrastructure, such as clearing houses or trade repositories?
  21. What are the benefits of using a common model for post-trade processing?
  22. How does the model handle reconciliation processes between trading parties?
  23. What role does the model play in automating corporate actions and other lifecycle events?
  24. How does the model ensure transparency and traceability of financial transactions?
  25. What are the key differences in handling various asset classes, such as equities, fixed income, and derivatives?
  26. What are the key steps involved in the clearing process, and how is it represented in the lifecycle of a trade?
  27. How does the allocation process work, and what factors determine how trades are allocated across different accounts?
  28. What triggers the early termination of a contract, and how is this process managed?
  29. How is novation processed, and what are the roles of the original and new counterparties?
  30. How does the model manage the workflow for trade termination events, such as early termination or exercise of options?
  31. How are settlement instructions generated and processed after a clearing event?
  32. What are the key considerations when managing the allocation of trades in a portfolio?
  33. How is the termination of a contract due to default handled, and what are the consequences for the involved parties?
  34. What are the key differences in how Exchange-Traded Derivatives (ETD) and Over-The-Counter (OTC) derivatives are represented and processed?
  35. How does the model manage Positions, including adjustments and settlements?
  36. How does the model handle the reallocation of Positions after a novation event?
  37. What are the common causes for early termination of a contract, and how are these recorded?
  38. How does the system ensure that all parties are informed and updated during a novation event?
  39. What is the role of a central counterparty (CCP) in the clearing process, and how does it affect the trade's lifecycle?
  40. How are collateral requirements adjusted following a clearing or novation event?
  41. What is the process for executing a partial termination, and how does it differ from a full termination?
  42. How does the early termination of a contract impact the associated collateral and margin requirements?