Question - Indicators and Historical Data

[sebasira]

This is not a bug, just a question because I do not understand how I can achieve it.

I'd like to get information about some basic indicators as RSI, EMA and SMA... If I understand correctly I need to calculate them myself, right? The exchanges or any API do not provide that information, right? (I mean free/open-source API)

So if I need to calculate it my self, I need to import historical data, let's say for the last 30days. There's an issue about that (#736) but I got lost with it and it's not clear to me how should I import historical data. Both, for dry run and for live (production) environments.

Could you please provide a guide about it or an example project? Thank you very much!

(Then I will need to know how to calculate those indicators, but that will surely be another issue)

[xblaatx]

To get information about indicators u can make use of the ta4j library. They have created a bunch of indicators where u can make use of. So no need to calculate things your own. Please take a look at https://ta4j.github.io/ta4j-wiki/Trading-strategies.html or look at the example provided here https://trading-bot.cassandre.tech/learn/technical-analysis.html#create-your-strategy-2.

For importing historical data you have 2 ways to do it. 1) Make use of the buildin feature. see https://trading-bot.cassandre.tech/learn/import-historical-data.html#overview.

2) If xchange provides a method for fetching historical data for your exchange. You can override the initialize() method. and implement some code to import them into your barSeries. Setting the setHistoricalImport provided by the BasicTa4jCassandreStrategy class to true, new bars are build further regarding the last imported tickers timestamp. Something like this.

` @Override public void initialize() {

    try {
        Exchange krakenExchange = ExchangeFactory.INSTANCE.createExchange(KrakenExchange.class);
        KrakenMarketDataService marketDataService = (KrakenMarketDataService) krakenExchange.getMarketDataService();
        LocalDateTime time = LocalDateTime.now().minusMinutes(DURATION * (MAX_BAR_COUNT + 1));
        ZoneId zoneId = ZoneId.systemDefault();
        long epoch = time.atZone(zoneId).toEpochSecond();
        KrakenOHLCs krakenOHLCs = marketDataService.getKrakenOHLC(new CurrencyPair(Currency.ETH, Currency.EUR), DURATION, epoch);
        for (KrakenOHLC krakenOHLC : krakenOHLCs.getOHLCs()) {
            if (krakenOHLC.getTime() <= krakenOHLCs.getLast()) {
                BaseBar bar = new BaseBar(
                        Duration.ofMinutes(DURATION),
                        ZonedDateTime.ofInstant(Instant.ofEpochSecond(krakenOHLC.getTime()), ZoneId.systemDefault()).plusMinutes(DURATION),
                        krakenOHLC.getOpen().doubleValue(),
                        krakenOHLC.getHigh().doubleValue(),
                        krakenOHLC.getLow().doubleValue(),
                        krakenOHLC.getClose().doubleValue(),
                        krakenOHLC.getVolume().doubleValue(),
                        krakenOHLC.getCount()
                );
                getSeries().addBar(bar);
            }
        }
        System.out.println("historical tickers for ETH imported");
        setHistoricalImport(true);
    } catch (IOException e) {
        e.printStackTrace();
    }
}`

[sebasira]

Thank you! I'll try that tonight and let you know!

[straumat]

@sebasira the reply from @xblaatx is excellent! I have nothing more to add

[sebasira]

@xblaatx thank you very much, you have opened my mind a lot! I've been working from your previous reply, trying to learn a little bit more.

About the historical import, It's very clear to how to do it. This article in the Cassandre web page was helpful Backtest your strategy. But still have a little question about this that I will ask in the end.

I've also found this repo crypto-trading-bot-tutorial that helped me a lot to understand one of your links https://trading-bot.cassandre.tech/learn/technical-analysis.html#create-your-strategy-2.

Before, I only knew about BasicCassandreStrategy, now with your post I've learn that there's another type of Strategy named BasicTa4jCassandreStrategy. This brings up more questions.

Please, don't get me wrong, I don't want to make noise on the issues, I just want to learn (I know maybe an issue is not the best channel, but once closed, if you think it would be helpful for other I can submit a PR adding what you teach me to a wiki article so it can serve as reference for the community)

So we have BasicCassandreStrategy and BasicTa4jCassandreStrategy. If I understood correctly, the first one only get the ticker updates and it's up to me what I want to do with that data. The BasicTa4jCassandreStrategy introduce the BarSeries, and I'm not sure if I understand what they are. Bar, Tickers, Candles... are all the same?

Looking at the source code of BasicTa4jCassandreStrategy I can see:

@Override
public final void tickersUpdates(final Set<TickerDTO> tickers) {
    // We only retrieve the ticker requested by the strategy (only one because it's a ta4j strategy).
    final Map<CurrencyPairDTO, TickerDTO> tickersUpdates = tickers.stream()
            .filter(ticker -> getRequestedCurrencyPair().equals(ticker.getCurrencyPair()))
            .collect(Collectors.toMap(TickerDTO::getCurrencyPair, Function.identity()));

    tickersUpdates.values().forEach(ticker -> {
        getLastTickers().put(ticker.getCurrencyPair(), ticker);
        if (series.getEndIndex() > 0 && isHistoricalImport) {
            barAggregator.update(series.getLastBar().getEndTime(), ticker.getLast());
            isHistoricalImport = false;
        } else {
            barAggregator.update(ticker.getTimestamp(), ticker.getLast());
        }
    });

    // We update the positions with tickers.
    updatePositionsWithTickersUpdates(tickersUpdates);

    onTickersUpdates(tickersUpdates);
}

Please correct me if I'm wrong, the BarSeries is created from the tickerUpdates. So a Bar has the same information as the Ticker. Or maybe the ticker updates build a Bar over time? In this case will a Bar be a Candle? Because I was looking at Tickers as Candles. This messes with me a lot!

We define the number of Bar we want the BarSeries to have with getMaximumBarCount(), and we can define the time that separate two bars with getDelayBetweenTwoBars. This is not clear to me, (maybe is because the above assumption that the Bar has the same information as the Ticker is wrong). Because if every Ticker creates a Bar, then the distance between two bar is the rate of the Ticker.

I'm looking at SimpleTa4jStrategy.java from crypto-trading-bot-tutorial repo. There the Strategy is defined as:

@Override
public Strategy getStrategy() {
    ClosePriceIndicator closePrice = new ClosePriceIndicator(getSeries());
    SMAIndicator sma = new SMAIndicator(closePrice, 3);
    return new BaseStrategy(new UnderIndicatorRule(sma, closePrice), new OverIndicatorRule(sma, closePrice));
}

Here's how I understand it. We use the BarSeries to get a ClosePriceIndicator (this would be a collection of cloisng prices from the BarSeries?), then use it to calculate a SimpleMovingAverage using only the last 3 closing prices of the indicator, right? And finally the Strategy would be to enter the market (send a buy signal shouldEnter()) when the price closes above the calculated SMA, and exit the market (send a sell signal shouldSell()) when the price drops below the SMA.

Am I following?

And finally about the Historical Data and the Indicators. If I set the delayBetweenTwoBar to be a single day (Duration.ofDays(1)), and I also set the maximumBarCount to 100, and I want to get a 21-days SMA, I will do the following

ClosePriceIndicator closePrice = new ClosePriceIndicator(getSeries());
SMAIndicator sma = new SMAIndicator(closePrice, 21);

But I will have to wait for at least 21 days to collect the data. That's were the import enters the party. But what I don't get is that it would be a void in the data from the latest imported value until the bot gets online. Wouldn't that affect the analysis? And also... suppose it will run fine for a year and then I need to shut it down and then back online again a day later. I would need to restart the import process right?

Thank you very much! I appreciate it a lot!

[xblaatx]

So we have BasicCassandreStrategy and BasicTa4jCassandreStrategy. If I understood correctly, the first one only get the ticker updates and it's up to me what I want to do with that data. The BasicTa4jCassandreStrategy introduce the BarSeries, and I'm not sure if I understand what they are. Bar, Tickers, Candles... are all the same?

Indeed the BasicCassandreStrategy only gets ticker updates. And nothing else happens with that data. It is up to you to handle it. While the BasicTa4jCassandreStrategy provides a way to make use of a strategy. It puts ticker updates in a bar. And when the given duration is exceeded it pushes the bar in a barserie. then it runs the strategy to look for entry and exit points. So you can see a barserie as a candle chart and a bar as a candle.

I think you are a bit confused about the ticker thing. While importing historical data. we call a ticker the OHLC data from a given startpoint and with a given duration. Lets say OHLC data started at 14:00 and with a duration of 1 hour. This can be seen as a candle (bar) on a 1 hour chart. If the bot is running it also receives periodic tickers at a rate of lets say 10 seconds. Cassandre uses these tickers to build a next candle( bar ). It keeps track of the open, highest, lowest , close prices. and when a ticker timestamp exceeds the specified duration between 2 bars. It pushes this data into a bar.

Here's how I understand it. We use the BarSeries to get a ClosePriceIndicator (this would be a collection of cloisng prices from the BarSeries?), then use it to calculate a SimpleMovingAverage using only the last 3 closing prices of the indicator, right? And finally the Strategy would be to enter the market (send a buy signal shouldEnter()) when the price closes above the calculated SMA, and exit the market (send a sell signal shouldSell()) when the price drops below the SMA.

I understand what you are trying to do. But this won't work. In fact i think the documentation is wrong also. If you run your bot like this. It wil shouldEnter() every time the 3 period EMA is under the close price. and it will shouldExit() every time the the 3 period EMA is over the close price. this will give signals on every bar added. A beter solution would be something like this.

@Override public Strategy getStrategy() { ClosePriceIndicator closePrice = new ClosePriceIndicator(getSeries()); SMAIndicator sma = new SMAIndicator(closePrice, 3); return new BaseStrategy(new CrossedUpIndicatorRule(closePrice, sma), new CrossedDownIndicatorRule(closePrice, sma)); }

Now only when the close price crosses up the sma a buying signal is made. and an exit signal is made when the closePrice crosses down the sma.

But I will have to wait for at least 21 days to collect the data. That's were the import enters the party. But what I don't get is that it would be a void in the data from the latest imported value until the bot gets online. Wouldn't that affect the analysis? And also... suppose it will run fine for a year and then I need to shut it down and then back online again a day later. I would need to restart the import process right?

As long you import the latest historical ticker from your exchange there will be no void or gap. Cassandre will just build the next bar and pushes it into the serie when the duration is exceeded. When using setHistoricalImport(true); the bot will take the end of the last imported ticker in account.

example: exchange ticker rates are at fixed times. like 14:00, 15:00 ... If U start the bot at 14:30. It will build a new bar and add it to the barSeries when the last update ticker received exceeds 15:00. Otherwise if u don't use setHistoricalImport(true) it will build the bar according the specified duration and pushes the bar in the serie at 15:30 if a 1 hour duration is specified. this wil result in 1 candle( bar) with a duration of 1,5 hours. And is affecting the strategy accuracy.

So just start the import process every time you start the bot. That is why I use my exchange provided method. I don't have to do things manually before every restart

[sebasira]

@xblaatx water-clear response! Thank you very much!

Now I've got a better understanding about Ticker and Bar. Also thank you for the CrossedUpIndicatorRule.

About the historical data I'll try to implement the exchange method so the process could be automated, if I succeed in doing so, I'll try to refactor it in a way that it could be generic for any exchange so when they all provide that endpoint, it would be easier to use across them.

I think we can close this issue for now

[straumat]

Thx @xblaatx !