Backtesting on Linux VM's Issue: "Main pumping: timeout on stop"

[groschovskiy]

Hi. I run a simple backtest with different inputs but received an issue. Could you please specify dependency for OS before running backtesting?

0027:fixme:mountmgr:harddisk_ioctl returning zero-filled buffer for IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 74080 (device=7 access=1 func=20 method=0)
0027:fixme:mountmgr:query_property Faking StorageDeviceProperty data
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 2d0c10 (device=2d access=0 func=304 method=0)
0027:fixme:mountmgr:harddisk_ioctl returning zero-filled buffer for IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS
0027:fixme:mountmgr:query_property Faking StorageDeviceProperty data
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 74080 (device=7 access=1 func=20 method=0)
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 2d0c10 (device=2d access=0 func=304 method=0)
0027:fixme:mountmgr:harddisk_ioctl returning zero-filled buffer for IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 74080 (device=7 access=1 func=20 method=0)
0027:fixme:mountmgr:query_property Faking StorageDeviceProperty data
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 2d0c10 (device=2d access=0 func=304 method=0)
0027:fixme:mountmgr:harddisk_ioctl returning zero-filled buffer for IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS
0027:fixme:mountmgr:query_property Faking StorageDeviceProperty data
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 74080 (device=7 access=1 func=20 method=0)
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 2d0c10 (device=2d access=0 func=304 method=0)
0027:fixme:mountmgr:harddisk_ioctl returning zero-filled buffer for IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 74080 (device=7 access=1 func=20 method=0)
0027:fixme:mountmgr:query_property Faking StorageDeviceProperty data
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 2d0c10 (device=2d access=0 func=304 method=0)
0027:fixme:mountmgr:harddisk_ioctl returning zero-filled buffer for IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS
0027:fixme:mountmgr:query_property Faking StorageDeviceProperty data
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 74080 (device=7 access=1 func=20 method=0)
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 2d0c10 (device=2d access=0 func=304 method=0)
0   18:10:46.907    Migration: data to 'C:\Program Files\MetaTrader 4' migration started
0   18:10:46.909    Migration: data migration finished, no virtual folder
0   18:10:46.910    LiveUpdate: update C:\Program Files\MetaTrader 4\MQL4 folder started
0   18:10:48.128    LiveUpdate: update C:\Program Files\MetaTrader 4\MQL4 folder finished
0027:fixme:mountmgr:harddisk_ioctl returning zero-filled buffer for IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 74080 (device=7 access=1 func=20 method=0)
0027:fixme:mountmgr:query_property Faking StorageDeviceProperty data
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 2d0c10 (device=2d access=0 func=304 method=0)
0027:fixme:mountmgr:harddisk_ioctl returning zero-filled buffer for IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS
0027:fixme:mountmgr:query_property Faking StorageDeviceProperty data
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 74080 (device=7 access=1 func=20 method=0)
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 2d0c10 (device=2d access=0 func=304 method=0)
0027:fixme:mountmgr:harddisk_ioctl returning zero-filled buffer for IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 74080 (device=7 access=1 func=20 method=0)
0027:fixme:mountmgr:query_property Faking StorageDeviceProperty data
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 2d0c10 (device=2d access=0 func=304 method=0)
0027:fixme:mountmgr:harddisk_ioctl returning zero-filled buffer for IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS
0027:fixme:mountmgr:query_property Faking StorageDeviceProperty data
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 74080 (device=7 access=1 func=20 method=0)
0027:fixme:mountmgr:harddisk_ioctl Unsupported ioctl 2d0c10 (device=2d access=0 func=304 method=0)
0097:fixme:thread:create_user_shared_data_thread Creating user shared data update thread.
0   18:10:48.197    MetaTrader 4 build 1220 started (MetaQuotes Software Corp.)
0   18:10:48.198    Microsoft Windows 7 x64, IE 11, 2 x Intel Xeon Processor (Skylake, IBRS), Memory: 7021 / 7792 Mb, Disk: 66 / 75 Gb, GMT+0
0   18:10:48.198    Data Folder: C:\Program Files\MetaTrader 4
0   18:10:48.506    Sources recompilation started
0   18:10:57.932    Sources recompilation finished (29 compiled in 9425 msec)
0   18:10:57.933    Started with configuration file 'config/mt4-tester.ini'
0   18:10:50.301    Compiling 'RSI'
0   18:10:50.301    Compiling 'Stochastic'
0   18:10:50.301    Compiling 'ZigZag'
0   18:10:50.302    Compiling 'SimplePanel'
0   18:10:54.121    Compiling 'PeriodConverter'
0   18:10:54.121    Compiling 'PrintPaths'
0   18:10:54.122    Compiling 'DLLSampleTester'
0   18:10:54.123    Compiling 'PipeClient'
0   18:10:54.123    Compiling 'PipeClientPure'
0   18:10:56.214    Compiling 'stdlib'
3   18:11:04.989    Main pumping: timeout on stop
Checking logs...
tester/logs/20191106.log:3  18:11:00.870    Tester: cannot start with configuration file because expert MACD Sample not loaded
ERROR: RUN failed.
Exiting /opt/scripts/run_backtest.sh with 1

[groschovskiy]

I see this issue on different VPS in different hosting providers. Could you please consult me about it?

Main pumping: timeout on stop Checking logs... tester/logs/20191106.log:3 18:11:00.870 Tester: cannot start with configuration file because expert MACD Sample not loaded ERROR: RUN failed. Exiting /opt/scripts/run_backtest.sh with 1

[kenorb]

Main pumping: timeout on stop

This is some weird error which started happening recently out of the blue, we're trying to find the cause. Not sure why this happens. Are you running this in Docker on your local machine?

Related: EA31337/EA31337-classes/issues/69

[kenorb]

The other issue was fixed by changing Ubuntu distribution from xenial to trusty. The Docker image has been tested under Docker 17.09.0-ce and 19.03.4.

If you still have the problem, can you share your version of Docker?

[groschovskiy]

I use the latest macOS Catalina with Docker. In this configuration, everything works well. But when I run tester on cloud instances in GCP or AWS, issue reproduced.

[root@google-cloud-backtest-vm ~]# docker --version Docker version 1.13.1, build 7f2769b/1.13.1

[kenorb]

It seems you're not using Docker CE.

Can you uninstall Docker Engine and install Docker CE instead? So it reports 18.x or 19.x?

See: https://docs.docker.com/install/linux/docker-ce/ubuntu/

So:

$ docker --version
Docker version 18.03.1-ce, build 9ee9f40

or

$ docker --version
Docker version 19.03.4, build 9013bf5

[groschovskiy]

@kenorb I tested few Linux distros and all decided to use Mac Mini and Windows machine. They work fine. Maybe you have a docker container with the pre-installed terminal? Use Windows for running algos very expensive for a lot of beginner traders.

[kenorb]

@groschovskiy Are you sure you've installed Docker CE version on Linux, not Engine (docker-engine/docker.io) and still doesn't work?

Btw. you can try Hetzner VPSes, they've cheap servers. Last time when I've tried, the testing was working fine under Docker.