search

c9s / bbgo

The modern cryptocurrency trading bot framework written in Go.
https://bbgo.finance
GNU Affero General Public License v3.0
1.22k stars 293 forks source link

Grid strategy not working #282

Closed arthurwolf closed 2 years ago

arthurwolf commented 2 years ago

No matter what I do, I can't get the grid strategy to generate trades (in backtesting).

I filled/synced the data correctly using:

./bbgo backtest --exchange binance -v --sync --sync-only --sync-from 2021-11-20 --config grid.yaml

My grid.yaml is like this:


arthur@crypto:~/bbgo$ cat grid.yaml
---
sessions:
  binance:
    exchange: binance
    envVarPrefix: binance
backtest:
  startTime: "2021-11-25"
  endTime: "2021-11-28"
  symbols:
  - ATAUSDT
  account:
    #makerCommission: 15
    #takerCommission: 15
    balances:
      ATA: 0.0
      USDT: 100000
exchangeStrategies:
- on: binance
  grid:
    symbol: ATAUSDT
    quantity: 100
    gridNumber: 100
    profitSpread: 0.05
    upperPrice: 1.5
    lowerPrice: 1.0
    long: true

And I run it with the following command:

./bbgo backtest --exchange binance --sync-from 2021-11-01 --config grid.yaml --base-asset-baseline

And I get the following result:


arthur@crypto:~/bbgo$ ./bbgo backtest --exchange binance --sync-from 2021-11-01 --config grid.yaml --base-asset-baseline
[0000]  INFO starting backtest with startTime Thu Nov 25 00:00:00 2021
[0010]  INFO ATAUSDT PROFIT AND LOSS REPORT
[0010]  INFO ===============================================
[0010]  INFO TRADES SINCE: 0001-01-01 00:00:00 +0000 UTC
[0010]  INFO NUMBER OF TRADES: 0
[0010]  INFO AVERAGE COST: $ 0.00
[0010]  INFO TOTAL BUY VOLUME: 0.000000
[0010]  INFO TOTAL SELL VOLUME: 0.000000
[0010]  INFO STOCK: 0.000000
[0010]  INFO FEE (USD): 0.000000
[0010]  INFO CURRENT PRICE: $ 1.21
[0010]  INFO CURRENCY FEES:
[0010]  INFO PROFIT: $ 0.00
[0010]  INFO UNREALIZED PROFIT: $ 0.00
[0010]  INFO INITIAL BALANCES:
[0010]  INFO  USDT: 100000.000000
[0010]  INFO FINAL BALANCES:
[0010]  INFO  USDT: 100000.000000
[0010]  INFO INITIAL ASSET ~= 96665 ATA (1 ATA = 1.034500)
[0010]  INFO FINAL ASSET ~= 82987 ATA (1 ATA = 1.205000)
[0010]  INFO ATA BASE ASSET PERFORMANCE: -14.15% (= (82987.55 - 96665.06) / 96665.06)
[0010]  INFO ATA PERFORMANCE: 16.48% (= (1.21 - 1.03) / 1.03)

Adding -v to the command, here are the initial lines (before the screen is filled with "k-line closed" lines):

time="2021-11-29T00:50:20Z" level=info msg="starting backtest with startTime Thu Nov 25 00:00:00 2021"
time="2021-11-29T00:50:20Z" level=info msg="setting risk controls: &{SessionBasedRiskControl:map[]}"
time="2021-11-29T00:50:20Z" level=info msg="attaching strategy *grid.Strategy on binance instead of [binance]"
time="2021-11-29T00:50:20Z" level=info msg="querying market info..." exchange=binance
time="2021-11-29T00:50:20Z" level=info msg="querying balances from session binance..." session=binance
time="2021-11-29T00:50:20Z" level=info msg="binance account" session=binance
time="2021-11-29T00:50:20Z" level=info msg=" USDT: 100000.000000"
time="2021-11-29T00:50:20Z" level=info msg="SELECT * FROM trades WHERE exchange = :exchange AND symbol = :symbol ORDER BY gid ASC"
time="2021-11-29T00:50:20Z" level=info msg="symbol ATAUSDT: 0 trades loaded"
time="2021-11-29T00:50:25Z" level=info msg="ATAUSDT last price: 1.034500"
time="2021-11-29T00:50:25Z" level=info msg="found symbol based strategy from grid.Strategy"
time="2021-11-29T00:50:25Z" level=info msg="using group id 1813379259 from fnv(grid-ATAUSDT-100-150000000-100000000)" strategy=grid
time="2021-11-29T00:50:25Z" level=info msg="subscribing ATAUSDT kline 1m" session=binance
time="2021-11-29T00:50:25Z" level=info msg="connecting session binance..." session=binance
time="2021-11-29T00:50:25Z" level=info msg="collecting backtest configurations..."
time="2021-11-29T00:50:25Z" level=info msg="used symbols: [ATAUSDT] and intervals: [1m 1d]"
time="2021-11-29T00:50:25Z" level=info msg="kline closed: binance 2021-11-25 00:00 ATAUSDT 1m Open: 1.03450000 Close: 1.03700000 High: 1.03880000 Low: 1.03300000 Volume: 11707.00000000 Change: 0.0025 Max Change: 0.0058" marketData=kline session=binance
time="2021-11-29T00:50:25Z" level=info msg="kline closed: binance 2021-11-25 00:01 ATAUSDT 1m Open: 1.03640000 Close: 1.03710000 High: 1.03800000 Low: 1.03520000 Volume: 5464.00000000 Change: 0.0007 Max Change: 0.0028" marketData=kline session=binance
time="2021-11-29T00:50:25Z" level=info msg="kline closed: binance 2021-11-25 00:02 ATAUSDT 1m Open: 1.03790000 Close: 1.03530000 High: 1.03790000 Low: 1.03520000 Volume: 7278.00000000 Change: -0.0026 Max Change: 0.0027" marketData=kline session=binance
time="2021-11-29T00:50:25Z" level=info msg="kline closed: binance 2021-11-25 00:03 ATAUSDT 1m Open: 1.03430000 Close: 1.03710000 High: 1.03720000 Low: 1.03420000 Volume: 9097.00000000 Change: 0.0028 Max Change: 0.0030" marketData=kline session=binance
time="2021-11-29T00:50:25Z" level=info msg="kline closed: binance 2021-11-25 00:04 ATAUSDT 1m Open: 1.03520000 Close: 1.03360000 High: 1.03530000 Low: 1.03270000 Volume: 6041.00000000 Change: -0.0016 Max Change: 0.0026" marketData=kline session=binance
time="2021-11-29T00:50:25Z" level=info msg="kline closed: binance 2021-11-25 00:05 ATAUSDT 1m Open: 1.03360000 Close: 1.03430000 High: 1.03430000 Low: 1.03270000 Volume: 2206.00000000 Change: 0.0007 Max Change: 0.0016" marketData=kline session=binance
time="2021-11-29T00:50:25Z" level=info msg="kline closed: binance 2021-11-25 00:06 ATAUSDT 1m Open: 1.03310000 Close: 1.03290000 High: 1.03450000 Low: 1.03290000 Volume: 816.00000000 Change: -0.0002 Max Change: 0.0016" marketData=kline session=binance
time="2021-11-29T00:50:25Z" level=info msg="kline closed: binance 2021-11-25 00:07 ATAUSDT 1m Open: 1.03250000 Close: 1.02470000 High: 1.03280000 Low: 1.02290000 Volume: 46101.00000000 Change: -0.0078 Max Change: 0.0099" marketData=kline session=binance
time="2021-11-29T00:50:25Z" level=info msg="kline closed: binance 2021-11-25 00:08 ATAUSDT 1m Open: 1.02480000 Close: 1.02280000 High: 1.02500000 Low: 1.02010000 Volume: 13110.00000000 Change: -0.0020 Max Change: 0.0049" marketData=kline session=binance
time="2021-11-29T00:50:25Z" level=info msg="kline closed: binance 2021-11-25 00:09 ATAUSDT 1m Open: 1.02270000 Close: 1.02040000 High: 1.02360000 Low: 1.02020000 Volume: 13339.00000000 Change: -0.0023 Max Change: 0.0034" marketData=kline session=binance
time="2021-11-29T00:50:25Z" level=info msg="kline closed: binance 2021-11-25 00:10 ATAUSDT 1m Open: 1.02150000 Close: 1.02160000 High: 1.02340000 Low: 1.02150000 Volume: 2375.00000000 Change: 0.0001 Max Change: 0.0019" marketData=kline session=binance
time="2021-11-29T00:50:25Z" level=info msg="kline closed: binance 2021-11-25 00:11 ATAUSDT 1m Open: 1.02180000 Close: 1.02020000 High: 1.02180000 Low: 1.02020000 Volume: 8237.00000000 Change: -0.0016 Max Change: 0.0016" marketData=kline session=binance

I should get trades, with these same parameters, other bots (Kucoin's UI, and a modified zenbot) generate trades, but for some reason I can't get any trades to happen here.

Am I doing something wrong with the parameters or the config somehow ?

I'm using MySQL for the backend.

Any help would be extremely welcome, and whatever the solution is, I'll create a PR with the answer to add it to the documentation (if adequate).

Thanks a lot in advance!

c9s commented 2 years ago

yes, you should get trades, I am checking it

c9s commented 2 years ago

I downloaded your config and run backtest it and here is my output:

[0003]  INFO ATAUSDT PROFIT AND LOSS REPORT
[0003]  INFO ===============================================
[0003]  INFO TRADES SINCE: 2021-11-25 00:07:59.999 +0000 UTC
[0003]  INFO NUMBER OF TRADES: 26
[0003]  INFO AVERAGE COST: $ 1.02
[0003]  INFO TOTAL BUY VOLUME: 1299.805000
[0003]  INFO TOTAL SELL VOLUME: 1239.248858
[0003]  INFO STOCK: 60.556142
[0003]  INFO FEE (USD): 0.397800
[0003]  INFO CURRENT PRICE: $ 1.21
[0003]  INFO CURRENCY FEES:
[0003]  INFO  - ATA: 0.195000
[0003]  INFO  - USDT: 0.198900
[0003]  INFO PROFIT: $ 61.19
[0003]  INFO UNREALIZED PROFIT: $ 72.26
[0003]  INFO INITIAL BALANCES:
[0003]  INFO  USDT: 100000.000000
[0003]  INFO FINAL BALANCES:
[0003]  INFO  ATA: 60.751142
[0003]  INFO  USDT: 100000.000000
[0003]  INFO INITIAL ASSET ~= 96665 ATA (1 ATA = 1.034500)
[0003]  INFO FINAL ASSET ~= 83048 ATA (1 ATA = 1.205000)
[0003]  INFO ATA BASE ASSET PERFORMANCE: -14.09% (= (83048.30 - 96665.06) / 96665.06)
[0003]  INFO ATA PERFORMANCE: 16.48% (= (1.21 - 1.03) / 1.03)
c9s commented 2 years ago

I can see the arbitrages were triggered:

[0002]  INFO submitting arbitrage order: SubmitOrder ATAUSDT LIMIT BUY 100.000000 @ 1.010000 against filled order ORDER backtest 22 ATAUSDT SELL 95.283019/95.283019 @ 1.060000 -> FILLED strategy=grid
[0002]  INFO submitting order: SubmitOrder ATAUSDT LIMIT BUY 100.000000 @ 1.010000
[0002]  INFO kline closed: binance 2021-11-26 08:31 ATAUSDT 1m O: 1.0394 H: 1.0627 L: 1.0374 C: 1.0624 CHG: 0.0230 MAXCHG: 0.0253 V: 97058.0000 QV: 102015.78 TBBV: 66667.00 marketData=kline session=binance
[0002]  INFO submitting arbitrage order: SubmitOrder ATAUSDT LIMIT BUY 100.000000 @ 1.030000 against filled order ORDER backtest 19 ATAUSDT SELL 95.370370/95.370370 @ 1.080000 -> FILLED strategy=grid
[0002]  INFO submitting order: SubmitOrder ATAUSDT LIMIT BUY 100.000000 @ 1.030000
[0002]  INFO submitting arbitrage order: SubmitOrder ATAUSDT LIMIT BUY 100.000000 @ 1.020000 against filled order ORDER backtest 20 ATAUSDT SELL 95.327103/95.327103 @ 1.070000 -> FILLED strategy=grid
[0002]  INFO submitting order: SubmitOrder ATAUSDT LIMIT BUY 100.000000 @ 1.020000
[0002]  INFO submitting arbitrage order: SubmitOrder ATAUSDT LIMIT BUY 100.000000 @ 1.025000 against filled order ORDER backtest 21 ATAUSDT SELL 95.348837/95.348837 @ 1.075000 -> FILLED strategy=grid
[0002]  INFO submitting order: SubmitOrder ATAUSDT LIMIT BUY 100.000000 @ 1.025000
[0002]  INFO submitting arbitrage order: SubmitOrder ATAUSDT LIMIT BUY 100.000000 @ 1.015000 against filled order ORDER backtest 23 ATAUSDT SELL 95.305164/95.305164 @ 1.065000 -> FILLED strategy=grid
[0002]  INFO submitting order: SubmitOrder ATAUSDT LIMIT BUY 100.000000 @ 1.015000
[0002]  INFO kline closed: binance 2021-11-26 08:32 ATAUSDT 1m O: 1.0622 H: 1.0888 L: 1.0562 C: 1.0562 CHG: -0.0060 MAXCHG: 0.0326 V: 121304.0000 QV: 130464.57 TBBV: 66897.00 marketData=kline session=binance
[0002]  INFO kline closed: binance 2021-11-26 08:33 ATAUSDT 1m O: 1.0600 H: 1.0634 L: 1.0330 C: 1.0377 CHG: -0.0223 MAXCHG: 0.0304 V: 103736.0000 QV: 109045.91 TBBV: 28404.00 marketData=kline session=binance
[0002]  INFO kline closed: binance 2021-11-26 08:34 ATAUSDT 1m O: 1.0343 H: 1.0350 L: 1.0100 C: 1.0322 CHG: -0.0021 MAXCHG: 0.0250 V: 210330.0000 QV: 215068.77 TBBV: 69980.00 marketData=kline session=binance
[0002]  INFO submitting arbitrage order: SubmitOrder ATAUSDT LIMIT SELL 95.370370 @ 1.080000 against filled order ORDER backtest 27 ATAUSDT BUY 100.000000/100.000000 @ 1.030000 -> FILLED strategy=grid
[0002]  INFO submitting order: SubmitOrder ATAUSDT LIMIT SELL 95.370370 @ 1.080000
[0002]  INFO kline closed: binance 2021-11-26 08:35 ATAUSDT 1m O: 1.0348 H: 1.0403 L: 1.0210 C: 1.0259 CHG: -0.0089 MAXCHG: 0.0193 V: 101499.0000 QV: 104590.71 TBBV: 52742.00 marketData=kline session=binance
[0002]  INFO submitting arbitrage order: SubmitOrder ATAUSDT LIMIT SELL 95.348837 @ 1.075000 against filled order ORDER backtest 29 ATAUSDT BUY 100.000000/100.000000 @ 1.025000 -> FILLED strategy=grid
[0002]  INFO submitting order: SubmitOrder ATAUSDT LIMIT SELL 95.348837 @ 1.075000
[0002]  INFO kline closed: binance 2021-11-26 08:36 ATAUSDT 1m O: 1.0263 H: 1.0361 L: 1.0210 C: 1.0327 CHG: 0.0064 MAXCHG: 0.0151 V: 76469.0000 QV: 78723.98 TBBV: 31817.00 marketData=kline session=binance
[0002]  INFO kline closed: binance 2021-11-26 08:37 ATAUSDT 1m O: 1.0302 H: 1.0350 L: 1.0300 C: 1.0330 CHG: 0.0028 MAXCHG: 0.0050 V: 112206.0000 QV: 115733.10 TBBV: 48191.00 marketData=kline session=binance
[0002]  INFO submitting arbitrage order: SubmitOrder ATAUSDT LIMIT BUY 100.000000 @ 1.025000 against filled order ORDER backtest 32 ATAUSDT SELL 95.348837/95.348837 @ 1.075000 -> FILLED strategy=grid
[0002]  INFO submitting order: SubmitOrder ATAUSDT LIMIT BUY 100.000000 @ 1.025000
c9s commented 2 years ago

@arthurwolf just to confirm: are you using the latest main branch version?

c9s commented 2 years ago

I checked the logs, it looks like you're using older version?

arthurwolf commented 2 years ago

I used this command as per the documentation: bash <(curl -s https://raw.githubusercontent.com/c9s/bbgo/main/scripts/setup-grid.sh)

That does not install the latest version?

(thanks a lot for answering/helping, so appreciated! any help you provide, I'll strive to transform into PRs improving documentation as I can)

arthurwolf commented 2 years ago

I checked, and https://raw.githubusercontent.com/c9s/bbgo/main/scripts/setup-grid.sh has a line:

version=v1.15.4

but the github page says the latest version is 1.18.0

So I'm presuming that's the issue here. I'll try changing the script, running it again, and see if it solves the issue.

arthurwolf commented 2 years ago

After changing the version as said above, it's now working!

See:

Screenshot from 2021-11-29 13-30-50

Thank you so much!

arthurwolf commented 2 years ago

It is now saying this:

Screenshot from 2021-11-29 13-32-52

I do not understand what it is saying/asking, and requiring to press "y" means I can not automate execution, is there any way to fix this?

Does this mean that for backtesting, I should use a separate database (and therefore .env file)? I know how to do that, but how will it know I'm using a separate one/not display the message?

arthurwolf commented 2 years ago

I have been running the command some more, and 1/2 to 1/3 of executions produce this error:

(edit: it seems to happen exactly half of the time in a switch: happens, doesn't, happens, doesn't, happens etc...)

time="2021-11-29T12:42:49Z" level=info msg="starting backtest with startTime Thu Nov 25 00:00:00 2021"
time="2021-11-29T12:42:49Z" level=warning msg="!!! To run backtest, you should use an isolated database for storing backtest trades !!!"
time="2021-11-29T12:42:49Z" level=warning msg="!!! The trade record in the current database WILL ALL BE DELETE !!!"
Are you sure to continue? [y/N]: time="2021-11-29T12:42:52Z" level=info msg="querying market info..." exchange=binance
time="2021-11-29T12:42:53Z" level=info msg="querying balances from session binance..." session=binance
time="2021-11-29T12:42:53Z" level=info msg="binance account" session=binance
time="2021-11-29T12:42:53Z" level=info msg=" USDT: 100000.000000"
fatal error: concurrent map read and map write

goroutine 62 [running]:
runtime.throw(0x1251c68, 0x21)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/runtime/panic.go:1117 +0x72 fp=0xc001b13b20 sp=0xc001b13af0 pc=0x436bd2
runtime.mapaccess2_faststr(0x10c4880, 0xc000220a80, 0xc0004c21e0, 0x7, 0x0, 0xed934bd80)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/runtime/map_faststr.go:116 +0x4a5 fp=0xc001b13b90 sp=0xc001b13b20 pc=0x414845
github.com/c9s/bbgo/pkg/backtest.(*Stream).Connect.func2(0xc000a0a3c0, 0xc000b0e0e0, 0x1, 0x1, 0xc00014c2a0, 0x2, 0x2)
        /Users/runner/work/bbgo/bbgo/pkg/backtest/stream.go:90 +0x2ce fp=0xc001b13fa8 sp=0xc001b13b90 pc=0xd9090e
runtime.goexit()
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc001b13fb0 sp=0xc001b13fa8 pc=0x46eb01
created by github.com/c9s/bbgo/pkg/backtest.(*Stream).Connect
        /Users/runner/work/bbgo/bbgo/pkg/backtest/stream.go:84 +0xde8

goroutine 1 [runnable]:
github.com/c9s/bbgo/pkg/backtest.(*Stream).Connect(0xc000a0a280, 0x13ba478, 0xc0000dfc40, 0x21, 0xc001290880)
        /Users/runner/work/bbgo/bbgo/pkg/backtest/stream.go:73 +0xa85
github.com/c9s/bbgo/pkg/bbgo.(*Environment).Connect(0xc0000180c0, 0x13ba478, 0xc0000dfc40, 0x7, 0xc0004d2088)
        /Users/runner/work/bbgo/bbgo/pkg/bbgo/environment.go:452 +0x58f
github.com/c9s/bbgo/pkg/bbgo.(*Trader).Run(0xc0007fdf80, 0x13ba478, 0xc0000dfc40, 0x0, 0x0)
        /Users/runner/work/bbgo/bbgo/pkg/bbgo/trader.go:333 +0x4aa
github.com/c9s/bbgo/pkg/cmd.glob..func2(0x1c987a0, 0xc00057e2a0, 0x0, 0x7, 0x0, 0x0)
        /Users/runner/work/bbgo/bbgo/pkg/cmd/backtest.go:242 +0x1979
github.com/spf13/cobra.(*Command).execute(0x1c987a0, 0xc00057e070, 0x7, 0x7, 0x1c987a0, 0xc00057e070)
        /Users/runner/go/pkg/mod/github.com/spf13/cobra@v1.1.1/command.go:850 +0x472
github.com/spf13/cobra.(*Command).ExecuteC(0x1c994c0, 0x8, 0x0, 0x0)
        /Users/runner/go/pkg/mod/github.com/spf13/cobra@v1.1.1/command.go:958 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
        /Users/runner/go/pkg/mod/github.com/spf13/cobra@v1.1.1/command.go:895
github.com/c9s/bbgo/pkg/cmd.Execute()
        /Users/runner/work/bbgo/bbgo/pkg/cmd/root.go:178 +0x7f9
main.main()
        /Users/runner/work/bbgo/bbgo/cmd/bbgo/main.go:8 +0x25

goroutine 60 [select]:
database/sql.(*DB).connectionOpener(0xc000125ee0, 0x13ba478, 0xc0000dff40)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/database/sql/sql.go:1133 +0xb5
created by database/sql.OpenDB
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/database/sql/sql.go:740 +0x12a

[...]

        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000b37fb0 sp=0xc000b37fa8 pc=0x46eb01
created by github.com/c9s/bbgo/pkg/backtest.(*Stream).Connect
        /Users/runner/work/bbgo/bbgo/pkg/backtest/stream.go:84 +0xde8

goroutine 1 [runnable]:
github.com/c9s/bbgo/pkg/backtest.(*Stream).Connect(0xc0004f4500, 0x13ba478, 0xc000531c40, 0x21, 0xc00111e880)
        /Users/runner/work/bbgo/bbgo/pkg/backtest/stream.go:73 +0xa85
github.com/c9s/bbgo/pkg/bbgo.(*Environment).Connect(0xc0000e20c0, 0x13ba478, 0xc000531c40, 0x7, 0xc000a3c1a8)
        /Users/runner/work/bbgo/bbgo/pkg/bbgo/environment.go:452 +0x58f
github.com/c9s/bbgo/pkg/bbgo.(*Trader).Run(0xc00135d500, 0x13ba478, 0xc000531c40, 0x0, 0x0)
        /Users/runner/work/bbgo/bbgo/pkg/bbgo/trader.go:333 +0x4aa
github.com/c9s/bbgo/pkg/cmd.glob..func2(0x1c987a0, 0xc0001a8690, 0x0, 0x7, 0x0, 0x0)
        /Users/runner/work/bbgo/bbgo/pkg/cmd/backtest.go:242 +0x1979
github.com/spf13/cobra.(*Command).execute(0x1c987a0, 0xc0001a8620, 0x7, 0x7, 0x1c987a0, 0xc0001a8620)
        /Users/runner/go/pkg/mod/github.com/spf13/cobra@v1.1.1/command.go:850 +0x472
github.com/spf13/cobra.(*Command).ExecuteC(0x1c994c0, 0x8, 0x0, 0x0)
        /Users/runner/go/pkg/mod/github.com/spf13/cobra@v1.1.1/command.go:958 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
        /Users/runner/go/pkg/mod/github.com/spf13/cobra@v1.1.1/command.go:895
github.com/c9s/bbgo/pkg/cmd.Execute()
        /Users/runner/work/bbgo/bbgo/pkg/cmd/root.go:178 +0x7f9
main.main()
        /Users/runner/work/bbgo/bbgo/cmd/bbgo/main.go:8 +0x25

goroutine 70 [select]:
database/sql.(*DB).connectionOpener(0xc0000f5d40, 0x13ba478, 0xc000531f40)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/database/sql/sql.go:1133 +0xb5
created by database/sql.OpenDB
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/database/sql/sql.go:740 +0x12a

goroutine 71 [select]:
github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1(0xc000595da0, 0xc000495560, 0xc000a67d40)
        /Users/runner/go/pkg/mod/github.com/go-sql-driver/mysql@v1.5.0/connection.go:621 +0xa5
created by github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher
        /Users/runner/go/pkg/mod/github.com/go-sql-driver/mysql@v1.5.0/connection.go:618 +0xbe

goroutine 68 [IO wait]:
internal/poll.runtime_pollWait(0x7fc0cde7a258, 0x72, 0xffffffffffffffff)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc00018c298, 0x72, 0xa600, 0xa646, 0xffffffffffffffff)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc00018c280, 0xc000b00000, 0xa646, 0xa646, 0x0, 0x0, 0x0)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/internal/poll/fd_unix.go:166 +0x1d5
net.(*netFD).Read(0xc00018c280, 0xc000b00000, 0xa646, 0xa646, 0x807f, 0xc000b025c2, 0x5)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/net/fd_posix.go:55 +0x4f
net.(*conn).Read(0xc00000e150, 0xc000b00000, 0xa646, 0xa646, 0x0, 0x0, 0x0)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/net/net.go:183 +0x91
crypto/tls.(*atLeastReader).Read(0xc000795d10, 0xc000b00000, 0xa646, 0xa646, 0x807f, 0xc0000b0400, 0x0)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/crypto/tls/conn.go:776 +0x63
bytes.(*Buffer).ReadFrom(0xc00012c978, 0x13a3c40, 0xc000795d10, 0x40bd05, 0x10e4bc0, 0x11d27a0)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/bytes/buffer.go:204 +0xbe
crypto/tls.(*Conn).readFromUntil(0xc00012c700, 0x13a4d80, 0xc00000e150, 0x5, 0xc00000e150, 0x9)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/crypto/tls/conn.go:798 +0xf3
crypto/tls.(*Conn).readRecordOrCCS(0xc00012c700, 0x0, 0x0, 0xc000046800)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/crypto/tls/conn.go:605 +0x115
crypto/tls.(*Conn).readRecord(...)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/crypto/tls/conn.go:573
crypto/tls.(*Conn).Read(0xc00012c700, 0xc0004ba000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/crypto/tls/conn.go:1276 +0x165
bufio.(*Reader).Read(0xc0005959e0, 0xc0001acc78, 0x9, 0x9, 0x6d664b, 0xc0007d3c78, 0x406fa5)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/bufio/bufio.go:227 +0x222
io.ReadAtLeast(0x13a3ac0, 0xc0005959e0, 0xc0001acc78, 0x9, 0x9, 0x9, 0xc00062f4a0, 0x14a13a0972c500, 0xc00062f4a0)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/io/io.go:328 +0x87
io.ReadFull(...)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/io/io.go:347
net/http.http2readFrameHeader(0xc0001acc78, 0x9, 0x9, 0x13a3ac0, 0xc0005959e0, 0x0, 0x0, 0xc0006a4b40, 0x0)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/net/http/h2_bundle.go:1477 +0x89
net/http.(*http2Framer).ReadFrame(0xc0001acc40, 0xc0006a4b40, 0x0, 0x0, 0x0)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/net/http/h2_bundle.go:1735 +0xa5
net/http.(*http2clientConnReadLoop).run(0xc0007d3fa8, 0x0, 0x0)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/net/http/h2_bundle.go:8331 +0xd8
net/http.(*http2ClientConn).readLoop(0xc000430c00)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/net/http/h2_bundle.go:8253 +0x6f
created by net/http.(*http2Transport).newClientConn
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/net/http/h2_bundle.go:7217 +0x6c5

goroutine 97 [runnable]:
reflect.Value.Addr(0x105f9e0, 0xc0025b8148, 0x18e, 0xc000c32230, 0x1, 0x1)
        /Users/runner/hostedtoolcache/go/1.16.9/x64/src/reflect/value.go:269 +0x8b
github.com/jmoiron/sqlx.fieldsByTraversal(0x12053a0, 0xc0025b80c0, 0x199, 0xc000556820, 0x11, 0x11, 0xc000a3c5a0, 0x11, 0x11, 0xc00061a201, ...)
        /Users/runner/go/pkg/mod/github.com/jmoiron/sqlx@v1.2.0/sqlx.go:1030 +0x1a9
github.com/jmoiron/sqlx.(*Rows).StructScan(0xc0004f00a0, 0x11d5a60, 0xc0025b80c0, 0x0, 0x0)
        /Users/runner/go/pkg/mod/github.com/jmoiron/sqlx@v1.2.0/sqlx.go:620 +0x15b
github.com/c9s/bbgo/pkg/service.(*BacktestService).scanRowsCh.func1(0xc001b26060, 0xc001b26000, 0xc0004f00a0)
        /Users/runner/work/bbgo/bbgo/pkg/service/backtest.go:173 +0x171
created by github.com/c9s/bbgo/pkg/service.(*BacktestService).scanRowsCh
        /Users/runner/work/bbgo/bbgo/pkg/service/backtest.go:166 +0x93
c9s commented 2 years ago

the mutex issue is now fixed in 47f1eaf1

c9s commented 2 years ago

The script is updated to v1.18.0 in commit d2ceb9919b2e93755a4a9b444138a5c9d3a7fdeb

arthurwolf commented 2 years ago

@c9s thank you very much! I'm not yet able to compile myself, so my only way to get this change is through a release, do you know when is the next time you plan on making a release?