Can not connect to TWS on Win32

[uwe]

I'm on a Mac (OS 10.9). I run TWS on a virtualbox WinXP image (32bit), because TWS sucks on Mac (I can not cycle between windows, I have to use the dock and the right click and so on).

When I try gofinance/ib under mac it works fine (e. g. with the IB Gateway), but when I connect from the Mac to the Windows VM it prints:

Could not connect to IB: EOF panic: runtime error: invalid memory address or nil pointer dereference [signal 0xb code=0x1 addr=0x0 pc=0x22f1]

goroutine 1 [running]: main.main() /Users/uwe.voelker/go/bb.go:21 +0x2f1

goroutine 5 [chan send]: github.com/gofinance/ib.func·001() /Users/uwe.voelker/go/src/github.com/gofinance/ib/engine.go:88 +0x76 created by github.com/gofinance/ib.uniqueID /Users/uwe.voelker/go/src/github.com/gofinance/ib/engine.go:91 +0xfb

goroutine 6 [chan send]: github.com/gofinance/ib.func·001() /Users/uwe.voelker/go/src/github.com/gofinance/ib/engine.go:88 +0x76 created by github.com/gofinance/ib.uniqueID /Users/uwe.voelker/go/src/github.com/gofinance/ib/engine.go:91 +0xfb exit status 2

I get the same (similar) output directly on Windows (I installed Go there as well).

A few years ago I wrote an IB API in Perl. I just tried to connect from Mac to my VM (via port forwarding in VirtualBox) and it worked.

So the connection to IB is possible! The problem must be somewhere in the Go library.

I would like to use Go for this task. How can I debug that further?

Are other users connection to TWS an Windows?

Thanks for your help.

[dsouzae]

Hi Uwe,

Can you show your code how you are connecting to TWS running on the Windows VM? Specifically looking for the NewEngineOptions passed to the NewEngine function.

And the code on how you are connecting using your Perl script might help on diagnosing connection problems.

[uwe]

This is my Go code:

package main

import ( "fmt" "github.com/gofinance/ib" "github.com/kr/pretty" "time" )

func main() { gw := "127.0.0.1:7496" opts := ib.NewEngineOptions{Gateway: gw}

engine, err := ib.NewEngine(opts)
if err != nil {
    fmt.Println("Could not connect to IB:", err)
}

id := engine.NextRequestID()
fmt.Println(id)

contract := ib.Contract{
    Symbol:       "GOOG",
    SecurityType: "STK",
    Exchange:     "SMART",
    Currency:     "USD",
}

m, err := ib.NewChainManager(engine, contract)
if err != nil {
    fmt.Println(err)
}

time.Sleep(5 * time.Second)

fmt.Printf("%# v", pretty.Formatter(m.Chains()))

defer m.Close()
defer engine.Stop()

}

I assume the code for the Contract search is wrong, but it fails way earlier.

The same code runs on Mac OS (at least the connection and the NextRequestID() call).

I have port forwarding for my VM. This is working, as my perl script can connect to the Windows VM (from Mac) without any problems.

Here is my perl script (https://github.com/uwe/finance-tws-simple/blob/master/examples/contract.pl):

!/usr/bin/env perl

show contract details

use strict; use warnings;

use Data::Dumper; use Finance::TWS::Simple;

my $tws = Finance::TWS::Simple->new( host => $ENV{TWS_HOST} || '127.0.0.1', port => $ENV{TWS_PORT} || '7496', );

my $contract = $tws->struct( Contract => { symbol => 'EUR', secType => 'CASH', exchange => 'IDEALPRO', localSymbol => 'EUR.USD', }, );

my $details = $tws->call( ContractDetails => { contract => $contract, }, );

warn Dumper $details;

As the perl code can connect, I doubt there is a general connection problem.

[dsouzae]

From your first output, it looks like it connects but when it reads the buffer from the server, it looks like the server response is shorter than what was expected. I think the EOF is from a read on a short buffer.

What version of TWS are you running on your Windows? The golang version needs 943.2a or higher, see the testserver directory for the jar file.

Also you can can set "DumpConversation" to true in NewEngineOptions for extra debugging. You might be able to turn on client logging in TWS.

[uwe]

On Windows I use the latest (web) version. I will report the exact number this evening. This is slightly newer than my Mac version.

When I set DumpConversation, the output does not change. Maybe it's the first read with the problem. I once put ReadByte (or whatever it was called) directly after the connect and it could not even read one byte - the error was the same: EOF. So something is really strange here.

[uwe]

On Windows I use 948.3c (from 5th of January 2015).

[uwe]

Maybe I found something: In types.go the startApi (71, 1, cliend_id) packet is sent immediately after the client_version (63). When I look at the Cpp reference implementation, startApi is just sent after the server_version and server_timestamp is received.

[dsouzae]

not sure where that is. but if you look at engine.go, function handshake, it sends client version and reads server version first.

I think the EOF is returned from the call to serverShake.read...

[uwe]

I looked at types.go:

func (c clientHandshake) write(b bytes.Buffer) error { if err := writeInt(b, c.version); err != nil { return err } if err := writeInt(b, mStartAPI); err != nil { return err } if err := writeInt(b, 1); err != nil { return err } return writeInt(b, c.id) }

[dsouzae]

you are right. I need to check against the IB Java reference implementation version of the latest version against the version this project is built against. I think if you used an old version of TWS it probably will work, but might of changed behaviour in a later version of TWS.

[dsouzae]

I checked the reference implement that gofinance/ib is based off. You are right the handshake is not done correctly.

I will see if I can reproduce the issue and see if I can supply you with a patch after the weekend.

[dsouzae]

Hi Uwe,

See my pull request #9.

I was able to reproduce the EOF issue, and my pull request fixes the issue. Still need to test if it works on older clients, but it matches the Java reference implementation for client version 63.

[uwe]

Thanks. I will check this with my setup tomorrow (or maybe this evening). But so far it looks right.

Regarding older servers, I'm quite sure it won't work with very old servers because client version 63 changed the handshake. Before it was client version and then directly the client id. With 63 this startApi was introduced. But I think that is fine, because even the latest stable API is quite old (and the beta doesn't change that much).

[dsouzae]

The code limits minimum server version to be 70. I tested against server versions 70 and 76, connection works okay for me.

[uwe]

Thanks a lot. Your patch solves my connection problem.