EA MtApi5 get locked/frozen under mutli-symbol optimization mode during backtesting

joecklau commented 3 years ago

Situation:

When running backtesting on MtApi5 with multiple symbols optimization mode, once the last tick of a symbol within the date-range is proceeded, MtApi5 will stay frozen. No connection/command will be able to sent to MtApi5 at this status.

Expected behaviour:

When trying backtesting multiple symbols optimization mode on other default sample EA, once all ticks of a symbol are proceeded, the ticks of next symbols will be started proceeded until the last symbol is also proceeded.

How to reproduce the situation:

Ensure the market watch has several/multiple symbols selected.
Optimization = "All symbols selected in market watch"
Set Date to custom period, pick a few days for fewest data to reach "the last tick" status.
For input, any BacktestingLockTicks mode can get the same result, but I usually picked "LOCK EVERY TICK"
For Agents, I disabled all other cores but leaving only one core enabled for the testing as I use 1 port only. (I guess only 1 agent/core can be supported for backtesting on MtApi5 consider the port connection setting?)

Then start the backtesting, wait a for a moment until the last tick of 1st symbol is reached. Then it will stay in frozen status.

What's also tested:

I tried re-connecting to the MtApi5 by the dotnet dll via the port, but unlike normal situation which was be ok to reconnect, it will be null response when it's under the frozen status.

joecklau commented 3 years ago

More testings: For first iteration on a symbol, Inside MQL, OnInit() are OnDeinit(const int reason) are invoked successfully. After the first iteration, the EA starts OnInit() but stays inside "wait for command (BacktestingReady) from remote side to be ready for work" waiting loop.

At this moment, inside C# side, unlike first iteration, it's very difficult for MtApi5Client.BeginConnect(port) to connect to the MtApi5 EA. Even the MtApi5Client.ConnectionState sometimes changed to CONNECTED after some retrials, commands from MtApi5Client dll towards MtApi5 EA always get "Response from MetaTrader is null" at this stage.

I doubt maybe some resource was not completely released/closed inside MT5Connector.dll -> deinitExpert(int expertHandle, string& err) So that the MtApi5Client cannot re-connect via the same port for next iteration?

joecklau commented 3 years ago

Root cause is found, but I think my handling is not suitable to create pull request, better just put my findings and dirty handling here as reference to @vdemydiuk . It's about _stopTimer and the handling in _stopTimer_Elapsed(object sender, System.Timers.ElapsedEventArgs e) inside /MTApiService/Mt5Expert.cs

In MT5 multiple symbol optimization backtesting mode, when one symbol's processing end and MT5 start another symbol's backtesting, it looks very likely violating the situation "Mt5Expert has received new tick during 2 sec in testing mode", thus invoking Deinit() unexpectedly to me.

As it's just local machine backtesting, my dirty handling is only comment-out this lines, then it works:

            //Log.Warn("Mt5Expert has received new tick during 2 sec in testing mode. The possible cause: user has stopped the tester manually in MetaTrader 5.");
            //Deinit();

Here's the log before my handling as reference:

2021-08-17 00:20:32,688 [1] INFO  MTApiService.MtAdapter - AddExpert: begin. expert = ExpertHandle = 0, Quote = Instrument = MSFT.NAS; Bid = 246.41; Ask = 246.42; ExpertHandle = 0
2021-08-17 00:20:32,801 [1] ERROR MTApiService.MtServer - CreateServiceHost: Error! The ChannelDispatcher at 'net.tcp://221.168.1.40:18228/MtApiService' with contract(s) '"IMtApi"' is unable to open its IChannelListener.
2021-08-17 00:20:32,803 [1] ERROR MTApiService.MtServer - CreateServiceHost: Error! The ChannelDispatcher at 'net.tcp://172.21.0.204:18228/MtApiService' with contract(s) '"IMtApi"' is unable to open its IChannelListener.
2021-08-17 00:20:32,805 [1] ERROR MTApiService.MtServer - CreateServiceHost: Error! The ChannelDispatcher at 'net.tcp://221.168.1.41:18228/MtApiService' with contract(s) '"IMtApi"' is unable to open its IChannelListener.
2021-08-17 00:20:32,807 [1] ERROR MTApiService.MtServer - CreateServiceHost: Error! The ChannelDispatcher at 'net.tcp://172.21.144.1:18228/MtApiService' with contract(s) '"IMtApi"' is unable to open its IChannelListener.
2021-08-17 00:20:32,813 [1] INFO  MTApiService.MtAdapter - AddExpert: end
2021-08-17 00:20:52,411 [1] INFO  MTApiService.MtAdapter - RemoveExpert: begin. expertHandle = 0
2021-08-17 00:20:52,421 [1] INFO  MTApiService.MtAdapter - RemoveExpert: end
2021-08-17 00:20:52,498 [1] INFO  MTApiService.MtAdapter - AddExpert: begin. expert = ExpertHandle = 0, Quote = Instrument = TSLA.NAS; Bid = 670.75; Ask = 670.83; ExpertHandle = 0
2021-08-17 00:20:52,499 [1] INFO  MTApiService.MtAdapter - AddExpert: end
2021-08-17 00:20:54,418 [9] WARN  MTApiService.Mt5Expert - Mt5Expert has received new tick during 2 sec in testing mode. The possible cause: user has stopped the tester manually in MetaTrader 5.
2021-08-17 00:20:54,461 [1] ERROR MTApiService.MtAdapter - The given key was not present in the dictionary.

KptKuck commented 3 years ago

Situation:

When running backtesting on MtApi5 with multiple symbols optimization mode, once the last tick of a symbol within the date-range is proceeded, MtApi5 will stay frozen. No connection/command will be able to sent to MtApi5 at this status.

Expected behaviour:

When trying backtesting multiple symbols optimization mode on other default sample EA, once all ticks of a symbol are proceeded, the ticks of next symbols will be started proceeded until the last symbol is also proceeded.

How to reproduce the situation:
* Ensure the market watch has several/multiple symbols selected.

* Optimization = "All symbols selected in market watch"

* Set Date to custom period, pick a few days for fewest data to reach "the last tick" status.

* For input, any BacktestingLockTicks mode can get the same result, but I usually picked "LOCK EVERY TICK"

* For Agents, I disabled all other cores but leaving only one core enabled for the testing as I use 1 port only. (I guess only 1 agent/core can be supported for backtesting on MtApi5 consider the port connection setting?)
Then start the backtesting, wait a for a moment until the last tick of 1st symbol is reached. Then it will stay in frozen status.

What's also tested:

I tried re-connecting to the MtApi5 by the dotnet dll via the port, but unlike normal situation which was be ok to reconnect, it will be null response when it's under the frozen status.

This API does not work with the optimization. When a new run starts the connection is disconnected. This can not work like this.

joecklau commented 3 years ago

Situation:

When running backtesting on MtApi5 with multiple symbols optimization mode, once the last tick of a symbol within the date-range is proceeded, MtApi5 will stay frozen. No connection/command will be able to sent to MtApi5 at this status.

Expected behaviour:

When trying backtesting multiple symbols optimization mode on other default sample EA, once all ticks of a symbol are proceeded, the ticks of next symbols will be started proceeded until the last symbol is also proceeded.

How to reproduce the situation:
* Ensure the market watch has several/multiple symbols selected.

* Optimization = "All symbols selected in market watch"

* Set Date to custom period, pick a few days for fewest data to reach "the last tick" status.

* For input, any BacktestingLockTicks mode can get the same result, but I usually picked "LOCK EVERY TICK"

* For Agents, I disabled all other cores but leaving only one core enabled for the testing as I use 1 port only. (I guess only 1 agent/core can be supported for backtesting on MtApi5 consider the port connection setting?)
Then start the backtesting, wait a for a moment until the last tick of 1st symbol is reached. Then it will stay in frozen status.

What's also tested:

I tried re-connecting to the MtApi5 by the dotnet dll via the port, but unlike normal situation which was be ok to reconnect, it will be null response when it's under the frozen status.
This API does not work with the optimization. When a new run starts the connection is disconnected. This can not work like this.

MetaTrader Terminal has auto-update mechanism, which will auto-close-and-restart the terminals periodically. So disconnection handling is as-expected. My handling practice is re-connection. So the key for my usage is to ensure the disconnected EA/dll don't hold the port/resource unexpectedly.

So far my dirty modification somehow allows me to do multi-symbol optimization, with random pauses every a number of symbols, which requires a few stop-and-start clicks to resume the progress... I'm sure there should be better modification on /MTApiService/Mt5Expert.cs rather than my dirty one above.

KptKuck commented 2 years ago

Maybe this will help you.

Do you know that you can start the strategy tester individually via the CLI ? In my project a MatLab program controls the strategy tester. there I am variable in the symbol,timeframe or start date and end date. Up to 4 instances run in parallel. Look in the documentation under:

Metatrader advanced start.

https://www.metatrader5.com/en/terminal/help/start_advanced/start

joecklau commented 2 years ago

Maybe this will help you.

Do you know that you can start the strategy tester individually via the CLI ? In my project a MatLab program controls the strategy tester. there I am variable in the symbol,timeframe or start date and end date. Up to 4 instances run in parallel. Look in the documentation under:

Metatrader advanced start.

https://www.metatrader5.com/en/terminal/help/start_advanced/start

Interesting! Combining with some CLI and dynamic .ini file generation coding work, it seems possible to do many things instead of relying of optimisation option in UI! Not limited to MtApi, but any EA in MT5. Thanks for such an inspiring input!

vdemydiuk / mtapi