add symbol/market metadata

[drew2323]

add various macro economic data (bond yields, interest rates, inflation including comparing with estimates) and symbol relevant data (earning, dividends etc.)

Allow adding new various data on the fly (global, specific economy/industry related, symbol related) - namodelovat strukturu (časová/statická etc.)

Zvážit využití finGPT, market sentiment, news sentiment....

[drew2323]

Global or market relevant or symbol relevant data will be stored as metadata represented by timeseries keyed by its name located in the specific section.

Updating workflow should be designed to update data - initial vs. update) - plugin system.

symbol_metadata - keyed by symbol, includes earnings, revenue, dividend dates etc. market_metadata - keyed by market, includes metrics like cpi, fed rates

# Example data for AAPL Earnings
aapl_earnings = {
    'date': ['2024-07-27', '2024-04-27', '2024-01-27'],
    'actual': [1.25, 1.15, 1.20],
    'estimate': [1.20, 1.18, 1.22]
}
# Example data for AAPL Dividend Ex-Dates
aapl_dividends = {
    'date': ['2024-08-10', '2024-05-10', '2024-02-10'],
    'dividend': [0.22, 0.21, 0.20]
}

# Combine into a dictionary for symbols
symbol_metadata = {
    'AAPL': {
        'Earnings': aapl_earnings,
        'Dividend_Ex_Date': aapl_dividends
    }
}

# Example data for CPI as a DataFrame
cpi_df = {
    'date': ['2024-08-01', '2024-07-01', '2024-06-01'],
    'actual': [3.2, 3.0, 3.1],
    'estimate': [3.1, 2.9, 3.0]
}

# Example data for Fed Interest Rate
fed_rate = {
    'date': ['2024-08-01', '2024-07-01', '2024-06-01'],
    'actual': [5.25, 5.00, 5.10],
    'estimate': [5.3, 5.1, 5.15]
}

# Combine into a dictionary
us_market_metadata = {
    'CPI': cpi_df,
    'Fed_Interest_Rate': fed_rate_df
}

markets_metadata = {
   'US': us_market_metadata
}

[drew2323]

Assumption: metadata are standalone data entity and not directly linked to source data (historical/rt aggregated data)

Suggested data structure:

• sources - keyed by source ID, contains aggregated data and indicators (solved in #223)
• symbol_metadata - keyed by symbol, contains symbol metadata
• market_metadata - keyed by market, contains market metadata

Maybe covered by metadata variable, keyed by symbol/market.

Thus can be solved separately in this issue.

Domyslet jak se budou plnit symbol a market metadata, konifgurace ve stratvars a životní cyklus (jak je používat v indikátorech a podmínkách, např. suspendovat v earnigns dnech nebo hodinu před a po CPI) stratvars override vs default (X nejpoužívanějších), odkuď se buou brát, např. automatický mapovací systém(stejný název jak v get_asset_info …)

[drew2323]

According to new decision the symbol and market metadata will be realized as type of source, whose source_id will be defined in add_data. That way it will be possible to have same approach for all data sources, including possibility to create indicators on them and use in signal conditions. Data Loader will be updated to accommodate intiail load and update of those metadata sources (maybe new component). fyi @pvlasak more info on req doc