Age filter seems not working

[xmflsct]

I encounter 2 situations:

1. All pairs get the same listed day - 4
2. Seems like listed day is not returned or something, but at least age filter does not work (https://www.freqtrade.io/en/stable/plugins/#agefilter)

[mikekonan]

Provide your config, proxy version and bot version, will investigate it

[xmflsct]

Proxy version 1.0.8. Because I am running it on ARM64, I clone your repo and build whenever you publish an update. Bot version 2021.10 with Kucoin temp fix https://github.com/freqtrade/freqtrade/commit/c70fdea886cf138ca09c14d70ce15e09dd37278b, also locally built because of this modification.

My config:

{
  "fiat_display_currency": "EUR",
  "stake_currency": "USDT",
  "stake_amount": "unlimited",
  "tradable_balance_ratio": 0.99,
  "dry_run": true,
  "dry_run_wallet": 10000,
  "db_url": "sqlite:////freqtrade/user_data/dryrun.sqlite",
  "unfilledtimeout": {
    "buy": 10,
    "sell": 30,
    "unit": "minutes"
  },
  "cancel_open_orders_on_exit": true,
  "bid_strategy": {
    "price_side": "ask",
    "ask_last_balance": 0.0,
    "use_order_book": false,
    "order_book_top": 1,
    "check_depth_of_market": {
      "enabled": false,
      "bids_to_ask_delta": 1
    }
  },
  "ask_strategy": {
    "price_side": "bid",
    "use_order_book": false,
    "order_book_top": 1,
    "order_book_max": 1
  },
  "pairlists":[
    {
      "method": "VolumePairList",
      "number_assets": 60,
      "sort_key": "quoteVolume",
      "refresh_period": 900
    },
    {
      "method": "AgeFilter",
      "min_days_listed": 14
    },
    {
      "method": "SpreadFilter",
      "max_spread_ratio": 0.006
    },
    {
      "method": "RangeStabilityFilter",
      "lookback_days": 3,
      "min_rate_of_change": 0.05,
      "refresh_period": 1800
    },
    {
      "method": "VolatilityFilter",
      "lookback_days": 3,
      "min_volatility": 0.02,
      "max_volatility": 0.75,
      "refresh_period": 43200
    },
    {
      "method": "ShuffleFilter"
    },
    {
      "method": "StaticPairList"
    }
  ],
  "exchange": {
    "name": "kucoin",
    "pair_whitelist": [
      "BTC/USDT"
    ],
    "pair_blacklist": [
      "KCS/.*",
      ".*(3|3L|3S)/.*",
      "(AUD|EUR|GBP|CHF|CAD|JPY)/.*",
      "(BUSD|USDT|TUSD|USDC|CUSDT|DAI|USDN|CUSD)/.*",
      "(ACM|AFA|ALA|ALL|APL|ASR|ATM|BAR|CAI|CITY|FOR|GAL|GOZ|IBFK|JUV|LEG|LOCK-1|NAVI|NMR|NOV|OG|PFL|PORTO|PSG|ROUSH|STV|TH|TRA|UCH|UFC|YBO)/.*",
      "(CHZ|SLP|XVS|MEM|AMPL|XYM|POLX|CARR|SKEY|MASK|KLV|TLOS)/.*",
      "(BASIC|LPT)/USDT"
    ],
    "ccxt_config": {
      "enableRateLimit": false,
      "urls": {
        "api": {
          "public": "http://proxy:8080/kucoin",
          "private": "http://proxy:8080/kucoin"
        }
      }
    },
    "ccxt_async_config": {
      "enableRateLimit": false
    }
  },
  "order_types": {
    "buy": "market",
    "sell": "market",
    "emergencysell": "market",
    "forcebuy": "market",
    "forcesell": "market",
    "stoploss": "market",
    "stoploss_on_exchange": false,
    "stoploss_on_exchange_interval": 60
  },
  "edge": {
    "enabled": false,
    "process_throttle_secs": 3600,
    "allowed_risk": 0.01
  },
  "initial_state": "running",
  "forcebuy_enable": false,
  "internals": {
    "heartbeat_interval": 450
  },
  "telegram": {
    "enabled": false,
    "token": "",
    "chat_id": ""
  },
  "api_server": {
    "enabled": false,
    "listen_ip_address": "0.0.0.0",
    "listen_port": 8880,
    "verbosity": "error",
    "enable_openapi": false,
    "jwt_secret_key": "",
    "username": "",
    "password": ""
  }
}

[xmflsct]

Some logs

2021-11-18 16:24:16,727 - VolumePairList - INFO - Searching 60 pairs: ['BTC/USDT', 'ETH/USDT', 'CRO/USDT', 'LUNA/USDT', 'SAND/USDT', 'BLOK/USDT', 'KDA/USDT', 'MANA/USDT', 'ALGO/USDT', 'XRP/USDT', 'GALAX/USDT', 'AVAX/USDT', 'SHIB/USDT', 'DOT/USDT', 'VRA/USDT', 'BNB/USDT', 'FTM/USDT', 'SOL/USDT', 'LTC/USDT', 'QRDO/USDT', 'ADA/USDT', 'RNDR/USDT', 'VET/USDT', 'ELON/USDT', 'LRC/USDT', 'DOGE/USDT', 'JASMY/USDT', 'ROSE/USDT', 'FLUX/USDT', 'NUM/USDT', 'ATOM/USDT', 'HTR/USDT', 'TRX/USDT', 'ONE/USDT', 'SHILL/USDT', 'RMRK/USDT', 'MATIC/USDT', 'TEL/USDT', 'XTM/USDT', 'NAKA/USDT', 'MOVR/USDT', 'ENJ/USDT', 'TRIAS/USDT', 'XLM/USDT', 'SCLP/USDT', 'CHR/USDT', 'LINK/USDT', 'PYR/USDT', 'WAXP/USDT', 'CRV/USDT', 'AUDIO/USDT', 'EOS/USDT', 'GRT/USDT', 'QNT/USDT', 'WILD/USDT', 'UNO/USDT', 'CAKE/USDT', 'KOK/USDT', 'FKX/USDT', 'XPR/USDT']

2021-11-18 16:24:17,594 - AgeFilter - INFO - Validated 60 pairs.

2021-11-18 16:24:17,595 - SpreadFilter - INFO - Removed SHILL/USDT from whitelist, because spread 0.776% > 0.6%
2021-11-18 16:24:17,595 - SpreadFilter - INFO - Removed TRIAS/USDT from whitelist, because spread 0.611% > 0.6%
2021-11-18 16:24:17,595 - SpreadFilter - INFO - Removed UNO/USDT from whitelist, because spread 0.718% > 0.6%
2021-11-18 16:24:17,595 - SpreadFilter - INFO - Removed FKX/USDT from whitelist, because spread 0.682% > 0.6%

2021-11-18 16:24:19,006 - VolatilityFilter - INFO - Removed NUM/USDT from whitelist, because volatility over 3 days is: 0.000 which is not in the configured range of 0.02-0.75.

2021-11-18 16:24:19,042 - freqtrade.rpc.rpc_manager - INFO - Sending rpc message: {'type': status, 'status': 'running'}
2021-11-18 16:24:19,092 - freqtrade.worker - INFO - Changing state to: RUNNING

I would expect NUM/USDT to be removed already by the age filter as it is listed just today, but it is not.

[mikekonan]

Can you check it with the latest version?

[xmflsct]

Nope it still doesn't work.

Also, the first time strategy that runs will get age 0, and that filters out all pairs. The consecutive run does not return age 0, but still no age data returned.

[xmflsct]

I just tried again v1.0.8. The first strategy that connects to the proxy shows correct age data, and correct age filter being applied. But subsequent strategy connecting to the proxy does not get the age filter - even no filter at all it seems.

[mikekonan]

1.0.9

[xmflsct]

https://github.com/mikekonan/freqtrade-proxy/issues/10#issuecomment-973241752 this comment refers to 1.0.9. https://github.com/mikekonan/freqtrade-proxy/issues/10#issuecomment-973244694 this comment refers to 1.0.8.

Sorry if I wasn't clear enough.

[mikekonan]

Try with https://github.com/mikekonan/freqtrade-proxy/releases/tag/v1.0.10 Also, take care of the timeout parameter in the configs example

[xmflsct]

Thanks for the update! At least now the filters are working for the first run/connection.

But subsequent connection still does not get AgeFilter data. Meaning when I start a second bot connecting to the proxy, the second bot does not show AgeFilter being applied. Both bots share the same config so they should behave the same.

[liquidmind313]

How do u recognise that AgeFilter data does not go trough ?

[xmflsct]

@liquidmind313 well, the log does not say that a pair such as NUM/USDT which is 1 day old that should be removed. The first strat that connects to the proxy says it has been removed, but not any subsequent connections.

[mikekonan]

have you restarted freqtrades?

[xmflsct]

Yes of course. Closed down everything, and start from new.

[mikekonan]

try 1.0.11

[xmflsct]

Thank you very much! It works now. I will review the data again after 1 day to see if it gets updated. Closing this issue for now.

[xmflsct]

I can confirm it works over day, meaning age data do get refreshed.

[kennylbj]

@mikekonan Is v1.0.11 released on the docker repo?

[mikekonan]

https://hub.docker.com/r/mikekonan/freqtrade-proxy/tags