Socat error connection refused

tomer-ds commented 11 months ago

Is your feature request related to a problem? Please describe. I am writing a helm chart to deploy this image. I was previously using https://github.com/rylorin/ib-gateway-docker but would prefer this one as it seems more fully maintained... Very many thanks for that by the way.

Of course I would be happy to share the helm chart when done
I seem to having an issue configuring the deployment. I keep getting "connection refused" error

socat[212] E connect(5, AF=2 127.0.0.1:4000, 16): Connection refused

Describe the solution you'd like Just some help really figuring out what I migth be doing wrong. Maybe there is some nuance which needs special attention when running in Kubernetes

Describe alternatives you've considered As stated I already had everything up and running with rylorin/ib-gateway-docker. But long-term it seems better to make the move

Additional context Add any other context or screenshots about the feature request here.

        # Note that in the comments in this file, TWS refers to both the Trader
        # Workstation and the IB Gateway, unless explicitly stated otherwise.
        #
        # When referred to below, the default value for a setting is the value
        # assumed if either the setting is included but no value is specified, or
        # the setting is not included at all.
        #
        # IBC may also be used to start the FIX CTCI Gateway. All settings
        # relating to this have names prefixed with FIX.
        #
        # The IB API Gateway and the FIX CTCI Gateway share the same code. Which
        # gateway actually runs is governed by an option on the initial gateway
        # login screen. The FIX setting described under IBC Startup
        # Settings below controls this.

        # =============================================================================
        # 1.   IBC Startup Settings
        # =============================================================================

        # IBC may be used to start the IB Gateway for the FIX CTCI. This
        # setting must be set to 'yes' if you want to run the FIX CTCI gateway. The
        # default is 'no'.

        FIX=false

        # =============================================================================
        # 2.   Authentication Settings
        # =============================================================================

        # TWS and the IB API gateway require a single username and password.
        # You may specify the username and password using the following settings:
        #
        #   IbLoginId
        #   IbPassword
        #
        # Alternatively, you can specify the username and password in the command
        # files used to start TWS or the Gateway, but this is not recommended for
        # security reasons.
        #
        # If you don't specify them, you will be prompted for them in the usual
        # login dialog when TWS starts (but whatever you have specified will be
        # included in the dialog automatically: for example you may specify the
        # username but not the password, and then you will be prompted for the
        # password via the login dialog). Note that if you specify either
        # the username or the password (or both) in the command file, then
        # IbLoginId and IbPassword settings defined in this file are ignored.
        #
        #
        # The FIX CTCI gateway requires one username and password for FIX order
        # routing, and optionally a separate username and password for market
        # data connections. You may specify the usernames and passwords using
        # the following settings:
        #
        #   FIXLoginId
        #   FIXPassword
        #   IbLoginId   (optional - for market data connections)
        #   IbPassword  (optional - for market data connections)
        #
        # Alternatively you can specify the FIX username and password in the
        # command file used to start the FIX CTCI Gateway, but this is not
        # recommended for security reasons.
        #
        # If you don't specify them, you will be prompted for them in the usual
        # login dialog when FIX CTCI gateway starts (but whatever you have
        # specified will be included in the dialog automatically: for example
        # you may specify the usernames but not the passwords, and then you will
        # be prompted for the passwords via the login dialog). Note that if you
        # specify either the FIX username or the FIX password (or both) on the
        # command line, then FIXLoginId and FIXPassword settings defined in this
        # file are ignored; he same applies to the market data username and
        # password.

        # IB API Authentication Settings
        # ------------------------------

        # Your TWS username:

        IbLoginId=*******

        # Your TWS password:

        IbPassword=*******

        # FIX CTCI Authentication Settings
        # --------------------------------

        # Your FIX CTCI username:

        FIXLoginId=

        # Your FIX CTCI password:

        FIXPassword=

        # Second Factor Authentication Settings
        # -------------------------------------

        # If you have enabled more than one second factor authentication
        # device, TWS presents a list from which you must select the device
        # you want to use for this login. You can use this setting to
        # instruct IBC to select a particular item in the list on your
        # behalf. Note that you must spell this value exactly as it appears
        # in the list. If no value is set, you must manually select the
        # relevant list entry.

        SecondFactorDevice=

        # If you use the IBKR Mobile app for second factor authentication,
        # and you fail to complete the process before the time limit imposed
        # by IBKR, this setting tells IBC whether to automatically restart
        # the login sequence, giving you another opportunity to complete
        # second factor authentication. 
        #
        # Permitted values are 'yes' and 'no'.
        #
        # If this setting is not present or has no value, then the value
        # of the deprecated ExitAfterSecondFactorAuthenticationTimeout is
        # used instead. If this also has no value, then this setting defaults
        # to 'no'.
        #
        # NB: you must be using IBC v3.14.0 or later to use this setting:
        # earlier versions ignore it.

        ReloginAfterSecondFactorAuthenticationTimeout=yes

        # This setting is only relevant if
        # ReloginAfterSecondFactorAuthenticationTimeout is set to 'yes',
        # or if ExitAfterSecondFactorAuthenticationTimeout is set to 'yes'.
        #
        # It controls how long (in seconds) IBC waits for login to complete
        # after the user acknowledges the second factor authentication
        # alert at the IBKR Mobile app. If login has not completed after
        # this time, IBC terminates.
        # The default value is 60.

        SecondFactorAuthenticationExitInterval=

        # This setting specifies the timeout for second factor authentication
        # imposed by IB. The value is in seconds. You should not change this
        # setting unless you have reason to believe that IB has changed the
        # timeout. The default value is 180.

        SecondFactorAuthenticationTimeout=180

        # Trading Mode
        # ------------
        #
        # This indicates whether the live account or the paper trading
        # account corresponding to the supplied credentials is to be used.
        # The allowed values are 'live' (the default) and 'paper'.
        #
        # If this is set to 'live', then the credentials for the live
        # account must be supplied. If it is set to 'paper', then either
        # the live or the paper-trading credentials may be supplied.

        TradingMode=paper

        # Paper-trading Account Warning
        # -----------------------------
        #
        # Logging in to a paper-trading account results in TWS displaying
        # a dialog asking the user to confirm that they are aware that this
        # is not a brokerage account. Until this dialog has been accepted,
        # TWS will not allow API connections to succeed. Setting this
        # to 'yes' (the default) will cause IBC to automatically
        # confirm acceptance. Setting it to 'no' will leave the dialog
        # on display, and the user will have to deal with it manually.

        AcceptNonBrokerageAccountWarning=yes

        # Login Dialog Display Timeout
        #-----------------------------
        #
        # In some circumstances, starting TWS may result in failure to display
        # the login dialog. Restarting TWS may help to resolve this situation,
        # and IBC does this automatically.
        #
        # This setting controls how long (in seconds) IBC waits for the login
        # dialog to appear before restarting TWS.
        #
        # Note that in normal circumstances with a reasonably specified 
        # computer the time to displaying the login dialog is typically less
        # than 20 seconds, and frequently much less. However many factors can
        # influence this, and it is unwise to set this value too low.
        #
        # The default value is 60.

        LoginDialogDisplayTimeout=60

        # =============================================================================
        # 3.   TWS Startup Settings
        # =============================================================================

        # Path to settings store
        # ----------------------
        #
        # Path to the directory where TWS should store its settings. This is
        # normally the folder in which TWS is installed. However you may set
        # it to some other location if you wish (for example if you want to
        # run multiple instances of TWS with different settings).
        #
        # It is recommended for clarity that you use an absolute path. The
        # effect of using a relative path is undefined.
        #
        # Linux and macOS users should use the appropriate path syntax.
        #
        # Note that, for Windows users, you MUST use double separator
        # characters to separate the elements of the folder path: for
        # example, IbDir=C:\\IBLiveSettings is valid, but
        # IbDir=C:\IBLiveSettings is NOT valid and will give unexpected
        # results. Linux and macOS users need not use double separators,
        # but they are acceptable.
        #
        # The default is the current working directory when IBC is
        # started, unless the TWS_SETTINGS_PATH setting in the relevant
        # start script is set.
        #
        # If both this setting and TWS_SETTINGS_PATH are set, then this
        # setting takes priority. Note that if they have different values,
        # auto-restart will not work.
        #
        # NB: this setting is now DEPRECATED. You should use the
        # TWS_SETTINGS_PATH setting in the relevant start script.

        IbDir=

        # Store settings on server
        # ------------------------
        #
        # If you wish to store a copy of your TWS settings on IB's
        # servers as well as locally on your computer, set this to
        # 'yes': this enables you to run TWS on different computers
        # with the same configuration, market data lines, etc. If set
        # to 'no', running TWS on different computers will not share the
        # same settings. If no value is specified, TWS will obtain its
        # settings from the same place as the last time this user logged
        # in (whether manually or using IBC).

        StoreSettingsOnServer=

        # Minimize TWS on startup
        # -----------------------
        #
        # Set to 'yes' to minimize TWS when it starts:

        MinimizeMainWindow=no

        # Existing Session Detected Action
        # --------------------------------
        #
        # When a user logs on to an IBKR account for trading purposes by any means, the
        # IBKR account server checks to see whether the account is already logged in
        # elsewhere. If so, a dialog is displayed to both the users that enables them
        # to determine what happens next. The 'ExistingSessionDetectedAction' setting
        # instructs TWS how to proceed when it displays this dialog:
        #
        #   * If the new TWS session is set to 'secondary', the existing session continues
        #     and the new session terminates. Thus a secondary TWS session can never
        #     override any other session.
        #
        #   * If the existing TWS session is set to 'primary', the existing session
        #     continues and the new session terminates (even if the new session is also
        #     set to primary). Thus a primary TWS session can never be overridden by
        #     any new session).
        #
        #   * If both the existing and the new TWS sessions are set to 'primaryoverride',
        #     the existing session terminates and the new session proceeds.
        #
        #   * If the existing TWS session is set to 'manual', the user must handle the
        #     dialog.
        #
        # The difference between 'primary' and 'primaryoverride' is that a
        # 'primaryoverride' session can be overriden over by a new 'primary' session,
        # but a 'primary' session cannot be overriden by any other session.
        #
        # When set to 'primary', if another TWS session is started and manually told to
        # end the 'primary' session, the 'primary' session is automatically reconnected.
        #
        # The default is 'manual'.

        ExistingSessionDetectedAction=primaryoverride

        # Override TWS API Port Number
        # ----------------------------
        #
        # If OverrideTwsApiPort is set to an integer, IBC changes the 
        # 'Socket port' in TWS's API configuration to that number shortly 
        # after startup (but note that for the FIX Gateway, this setting is
        # actually stored in jts.ini rather than the Gateway's settings
        # file). Leaving the setting blank will make no change to 
        # the current setting. This setting is only intended for use in 
        # certain specialized situations where the port number needs to 
        # be set dynamically at run-time, and for the FIX Gateway: most
        # non-FIX users will never need it, so don't use it unless you know
        # you need it.

        OverrideTwsApiPort=

        # Override TWS Master Client ID
        # -----------------------------
        #
        # If OverrideTwsMasterClientID is set to an integer, IBC changes the
        # 'Master Client ID' value in TWS's API configuration to that 
        # value shortly after startup. Leaving the setting blank will make
        # no change to the current setting. This setting is only intended 
        # for use in certain specialized situations where the value needs to
        # be set dynamically at run-time: most users will never need it,
        # so don't use it unless you know you need it.

        OverrideTwsMasterClientID=

        # Read-only Login
        # ---------------
        #
        # If ReadOnlyLogin is set to 'yes', and the user is enrolled in IB's
        # account security programme, the user will not be asked to perform
        # the second factor authentication action, and login to TWS will
        # occur automatically in read-only mode: in this mode, placing or
        # managing orders is not allowed. 
        #
        # If set to 'no', and the user is enrolled in IB's account security
        # programme, the second factor authentication process is handled
        # according to the Second Factor Authentication Settings described
        # elsewhere in this file.
        #
        # If the user is not enrolled in IB's account security programme,
        # this setting is ignored. The default is 'no'.

        ReadOnlyLogin=no

        # Read-only API
        # -------------
        #
        # If ReadOnlyApi is set to 'yes', API programs cannot submit, modify
        # or cancel orders. If set to 'no', API programs can do these things.
        # If not set, the existing TWS/Gateway configuration is unchanged.
        # NB: this setting is really only supplied for the benefit of new TWS
        # or Gateway instances that are being automatically installed and
        # started without user intervention (eg Docker containers). Where
        # a user is involved, they should use the Global Configuration to
        # set the relevant checkbox (this only needs to be done once) and
        # not provide a value for this setting.

        ReadOnlyApi=no

        # API Precautions
        # ---------------
        # 
        # These settings relate to the corresponding 'Precautions' checkboxes in the
        # API section of the Global Configuration dialog.
        #
        # For all of these, the accepted values are:
        # - 'yes' sets the checkbox
        # - 'no' clears the checkbox
        # - if not set, the existing TWS/Gateway configuration is unchanged
        #
        # NB: thess settings are really only supplied for the benefit of new TWS
        # or Gateway instances that are being automatically installed and
        # started without user intervention, or where user settings are not preserved
        # between sessions (eg some Docker containers). Where a user is involved, they
        # should use the Global Configuration to set the relevant checkboxes and not
        # provide values for these settings.

        BypassOrderPrecautions=

        BypassBondWarning=

        BypassNegativeYieldToWorstConfirmation=

        BypassCalledBondWarning=

        BypassSameActionPairTradeWarning=

        BypassPriceBasedVolatilityRiskWarning=

        BypassUSStocksMarketDataInSharesWarning=

        BypassRedirectOrderWarning=

        BypassNoOverfillProtectionPrecaution=

        # Market data size for US stocks - lots or shares
        # -----------------------------------------------
        #
        # Since IB introduced the option of market data for US stocks showing
        # bid, ask and last sizes in shares rather than lots, TWS and Gateway
        # display a dialog immediately after login notifying the user about
        # this and requiring user input before allowing market data to be
        # accessed. The user can request that the dialog not be shown again.
        #
        # It is recommended that the user should handle this dialog manually
        # rather than using these settings, which are provided for situations 
        # where the user interface is not easily accessible, or where user
        # settings are not preserved between sessions (eg some Docker images).
        #
        # - If this setting is set to 'accept', the dialog will be handled
        #   automatically and the option to not show it again will be
        #   selected.
        #
        #   Note that in this case, the only way to allow the dialog to be
        #   displayed again is to manually enable the 'Bid, Ask and Last
        #   Size Display Update' message in the 'Messages' section of the TWS
        #   configuration dialog. So you should only use 'Accept' if you are
        #   sure you really don't want the dialog to be displayed again, or
        #   you have easy access to the user interface.
        #
        # - If set to 'defer', the dialog will be handled automatically (so
        #   that market data will start), but the option to not show it again
        #   will not be selected, and it will be shown again after the next
        #   login.
        #
        # - If set to 'ignore', the user has to deal with the dialog manually.
        #
        # The default value is 'ignore'.
        #
        # Note if set to 'accept' or 'defer', TWS also automatically sets
        # the API settings checkbox labelled 'Send market data in lots for
        # US stocks for dual-mode API clients'. IBC cannot prevent this.
        # However you can change this immmediately by setting
        # SendMarketDataInLotsForUSstocks (see below) to 'no' .

        AcceptBidAskLastSizeDisplayUpdateNotification=accept

        # This setting determines whether the API settings checkbox labelled
        # 'Send market data in lots for US stocks for dual-mode API clients'
        # is set or cleared. If set to 'yes', the checkbox is set. If set to
        # 'no' the checkbox is cleared. If defaulted, the checkbox is
        # unchanged.

        SendMarketDataInLotsForUSstocks=

        # Trusted API Client IPs
        # ----------------------
        #
        # NB: THIS SETTING IS ONLY RELEVANT FOR THE GATEWAY, AND ONLY WHEN FIX=yes.
        # In all other cases it is ignored.
        #
        # This is a list of IP addresses separated by commas. API clients with IP
        # addresses in this list are able to connect to the API without Gateway
        # generating the 'Incoming connection' popup.
        #
        # Note that 127.0.0.1 is always permitted to connect, so do not include it
        # in this setting.

        TrustedTwsApiClientIPs=127.0.0.1

        # Reset Order ID Sequence
        # -----------------------
        #
        # The setting resets the order id sequence for orders submitted via the API, so
        # that the next invocation of the `NextValidId` API callback will return the
        # value 1. The reset occurs when TWS starts.
        #
        # Note that order ids are reset for all API clients, except those that have
        # outstanding (ie incomplete) orders: their order id sequence carries on as
        # before.
        #
        # Valid values are 'yes', 'true', 'false' and 'no'. The default is 'no'.

        ResetOrderIdsAtStart=

        # This setting specifies IBC's action when TWS displays the dialog asking for
        # confirmation of a request to reset the API order id sequence.
        #
        # Note that the Gateway never displays this dialog, so this setting is ignored
        # for a Gateway session.
        #
        # Valid values consist of two strings separated by a solidus '/'. The first
        # value specifies the action to take when the order id reset request resulted
        # from setting ResetOrderIdsAtStart=yes. The second specifies the action to
        # take when the order id reset request is a result of the user clicking the
        # 'Reset API order ID sequence' button in the API configuration. Each value
        # must be one of the following:
        #
        #    'confirm' 
        #        order ids will be reset
        #
        #    'reject' 
        #        order ids will not be reset
        #
        #    'ignore' 
        #        IBC will ignore the dialog. The user must take action.
        #
        #    The default setting is ignore/ignore

        # Examples:
        #
        #    'confirm/reject' - confirm order id reset only if ResetOrderIdsAtStart=yes
        #                       and reject any user-initiated requests
        #
        #    'ignore/confirm' - user must decide what to do if ResetOrderIdsAtStart=yes
        #                       and confirm user-initiated requests
        #
        #    'reject/ignore'  - reject order id reset if  ResetOrderIdsAtStart=yes but
        #                       allow user to handle user-initiated requests 

        ConfirmOrderIdReset=

        # =============================================================================
        # 4.   TWS Auto-Logoff and Auto-Restart
        # =============================================================================
        #
        # TWS and Gateway insist on being restarted every day. Two alternative
        # automatic options are offered: 
        #
        #    - Auto-Logoff: at a specified time, TWS shuts down tidily, without
        #      restarting.
        #
        #    - Auto-Restart: at a specified time, TWS shuts down and then restarts
        #      without the user having to re-autheticate.
        #
        # The normal way to configure the time at which this happens is via the Lock
        # and Exit section of the Configuration dialog. Once this time has been
        # configured in this way, the setting persists until the user changes it again.
        #
        # However, there are situations where there is no user available to do this
        # configuration, or where there is no persistent storage (for example some
        # Docker images). In such cases, the auto-restart or auto-logoff time can be
        # set whenever IBC starts with the settings below.
        #
        # The value, if specified, must be a time in HH:MM AM/PM format, for example
        # 08:00 AM or 10:00 PM. Note that there must be a single space between the
        # two parts of this value; also that midnight is "12:00 AM" and midday is
        # "12:00 PM".
        #
        # If no value is specified for either setting, the currently configured
        # settings will apply. If a value is supplied for one setting, the other
        # setting is cleared. If values are supplied for both settings, only the
        # auto-restart time is set, and the auto-logoff time is cleared.
        #
        # Note that for a normal TWS/Gateway installation with persistent storage
        # (for example on a desktop computer) the value will be persisted as if the
        # user had set it via the configuration dialog.
        #
        # If you choose to auto-restart, you should take note of the considerations
        # described at the link below. Note that where this information mentions
        # 'manual authentication', restarting IBC will do the job (IBKR does not
        # recognise the existence of IBC in its docuemntation).
        #
        #  https://www.interactivebrokers.com/en/software/tws/twsguide.htm#usersguidebook/configuretws/auto_restart_info.htm
        #
        # If you use the "RESTART" command via the IBC command server, and IBC is
        # running any version of the Gateway (or a version of TWS earlier than 1018),
        # note that this will set the Auto-Restart time in Gateway/TWS's configuration
        # dialog to the time at which the restart actually happens (which may be up to
        # a minute after the RESTART command is issued). To prevent future auto-
        # restarts at this time, you must make sure you have set AutoLogoffTime or
        # AutoRestartTime to your desired value before running IBC. NB: this does not
        # apply to TWS from version 1018 onwards.

        AutoLogoffTime=

        AutoRestartTime=

        # =============================================================================
        # 5.   TWS Tidy Closedown Time
        # =============================================================================
        #
        # Specifies a time at which TWS will close down tidily, with no restart.
        #
        # There is little reason to use this setting. It is similar to AutoLogoffTime,
        # but can include a day-of-the-week, whereas AutoLogoffTime and AutoRestartTime
        # apply every day. So for example you could use ClosedownAt in conjunction with
        # AutoRestartTime to shut down TWS on Friday evenings after the markets
        # close, without it running on Saturday as well.
        #
        # To tell IBC to tidily close TWS at a specified time every
        # day, set this value to <hh:mm>, for example:
        # ClosedownAt=22:00
        #
        # To tell IBC to tidily close TWS at a specified day and time
        # each week, set this value to <dayOfWeek hh:mm>, for example:
        # ClosedownAt=Friday 22:00
        #
        # Note that the day of the week must be specified using your
        # default locale. Also note that Java will only accept
        # characters encoded to ISO 8859-1 (Latin-1). This means that
        # if the day name in your default locale uses any non-Latin-1
        # characters you need to encode them using Unicode escapes
        # (see http://java.sun.com/docs/books/jls/third_edition/html/lexical.html#3.3
        # for details). For example, to tidily close TWS at 12:00 on
        # Saturday where the default locale is Simplified Chinese,
        # use the following:
        # #ClosedownAt=\u661F\u671F\u516D 12:00

        ClosedownAt=

        # =============================================================================
        # 6.   Other TWS Settings
        # =============================================================================

        # Accept Incoming Connection
        # --------------------------
        #
        # If set to 'accept', IBC automatically accepts incoming
        # API connection dialogs. If set to 'reject', IBC
        # automatically rejects incoming API connection dialogs. If
        # set to 'manual', the user must decide whether to accept or reject
        # incoming API connection dialogs. The default is 'manual'.
        # NB: it is recommended to set this to 'reject', and to explicitly
        # configure which IP addresses can connect to the API in TWS's API
        # configuration page, as this is much more secure (in this case, no
        # incoming API connection dialogs will occur for those IP addresses).

        AcceptIncomingConnectionAction=accept

        # Allow Blind Trading
        # -------------------
        #
        # If you attempt to place an order for a contract for which
        # you have no market data subscription, TWS displays a dialog
        # to warn you against such blind trading.
        #
        #   yes   means the dialog is dismissed as though the user had
        #     clicked the 'Ok' button: this means that you accept
        #     the risk and want the order to be submitted.
        #
        #   no    means the dialog remains on display and must be
        #         handled by the user.

        AllowBlindTrading=yes

        # Save Settings on a Schedule
        # ---------------------------
        #
        # You can tell TWS to automatically save its settings on a schedule
        # of your choosing. You can specify one or more specific times,
        # like this:
        #
        # SaveTwsSettingsAt=HH:MM [ HH:MM]...
        #
        # for example:
        # SaveTwsSettingsAt=08:00   12:30 17:30
        #
        # Or you can specify an interval at which settings are to be saved,
        # optionally starting at a specific time and continuing until another
        # time, like this:
        #
        #SaveTwsSettingsAt=Every n [{mins | hours}] [hh:mm] [hh:mm]
        #
        # where the first hh:mm is the start time and the second is the end
        # time. If you don't specify the end time, settings are saved regularly
        # from the start time till midnight. If you don't specify the start time.
        # settings are saved regularly all day, beginning at 00:00. Note that
        # settings will always be saved at the end time, even if that is not
        # exactly one interval later than the previous time. If neither 'mins'
        # nor 'hours' is specified, 'mins' is assumed. Examples:
        #
        # To save every 30 minutes all day starting at 00:00
        #SaveTwsSettingsAt=Every 30
        #SaveTwsSettingsAt=Every 30 mins
        #
        # To save every hour starting at 08:00 and ending at midnight
        #SaveTwsSettingsAt=Every 1 hours 08:00
        #SaveTwsSettingsAt=Every 1 hours 08:00 00:00
        #
        # To save every 90 minutes starting at 08:00 up to and including 17:43
        #SaveTwsSettingsAt=Every 90 08:00 17:43

        SaveTwsSettingsAt=

        # Confirm Crypto Currency Orders Automatically
        # --------------------------------------------
        #
        # When you place an order for a cryptocurrency contract, a dialog is displayed
        # asking you to confirm that you want to place the order, and notifying you
        # that you are placing an order to trade cryptocurrency with Paxos, a New York
        # limited trust company, and not at Interactive Brokers.
        #
        #   transmit    means that the order will be placed automatically, and the
        #               dialog will then be closed
        #
        #   cancel      means that the order will not be placed, and the dialog will
        #               then be closed
        #
        #   manual      means that IBC will take no action and the user must deal
        #               with the dialog

        ConfirmCryptoCurrencyOrders=transmit

        # =============================================================================
        # 7.   Settings Specific to Indian Versions of TWS
        # =============================================================================

        # Indian versions of TWS may display a password expiry
        # notification dialog and a NSE Compliance dialog. These can be
        # dismissed by setting the following to yes. By default the
        # password expiry notice is not dismissed, but the NSE Compliance
        # notice is dismissed.

        # Warning: setting DismissPasswordExpiryWarning=yes will mean
        # you will not be notified when your password is about to expire.
        # You must then take other measures to ensure that your password
        # is changed within the expiry period, otherwise IBC will
        # not be able to login successfully.

        DismissPasswordExpiryWarning=no
        DismissNSEComplianceNotice=yes

        # =============================================================================
        # 8.   IBC Command Server Settings
        # =============================================================================

        # Do NOT CHANGE THE FOLLOWING SETTINGS unless you
        # intend to issue commands to IBC (for example
        # using telnet). Note that these settings have nothing to
        # do with running programs that use the TWS API.

        # Command Server Port Number
        # --------------------------
        #
        # The port number that IBC listens on for commands
        # such as "STOP". DO NOT set this to the port number
        # used for TWS API connections.
        #
        # The convention is to use 7462 for this port,
        # but it must be set to a different value from any other
        # IBC instance that might run at the same time.
        #
        # The default value is 0, which tells IBC not to start
        # the command server

        #CommandServerPort=7462
        CommandServerPort=0

        # Permitted Command Sources
        # -------------------------
        #
        # A comma separated list of IP addresses, or host names,
        # which are allowed addresses for sending commands to
        # IBC.  Commands can always be sent from the
        # same host as IBC is running on.

        ControlFrom=<some CIDR ranges>

        # Address for Receiving Commands
        # ------------------------------
        #
        # Specifies the IP address on which the Command Server
        # is to listen. For a multi-homed host, this can be used
        # to specify that connection requests are only to be
        # accepted on the specified address. The default is to
        # accept connection requests on all local addresses.

        BindAddress=

        # Command Prompt
        # --------------
        #
        # The specified string is output by the server when
        # the connection is first opened and after the completion
        # of each command. This can be useful if sending commands
        # using an interactive program such as telnet. The default
        # is that no prompt is output.
        # For example:
        #
        # CommandPrompt=>

        CommandPrompt=ibcontroller>

        # Suppress Command Server Info Messages
        # -------------------------------------
        #
        # Some commands can return intermediate information about
        # their progress. This setting controls whether such
        # information is sent. The default is that such information
        # is not sent.

        SuppressInfoMessages=yes

        # =============================================================================
        # 9.   Diagnostic Settings
        # =============================================================================
        #
        # IBC can log information about the structure of windows
        # displayed by TWS. This information is useful when adding
        # new features to IBC or when behaviour is not as expected. 
        #
        # The logged information shows the hierarchical organisation
        # of all the components of the window, and includes the
        # current values of text boxes and labels.
        #
        # Note that this structure logging has a small performance
        # impact, and depending on the settings can cause the logfile
        # size to be significantly increased. It is therefore
        # recommended that the LogStructureWhen setting be set to
        # 'never' (the default) unless there is a specific reason
        # that this information is needed.

        # Scope of Structure Logging
        # --------------------------
        #
        # The LogStructureScope setting indicates which windows are
        # eligible for structure logging:
        #
        #    - (default value) if set to 'known', only windows that
        #      IBC recognizes are eligible - these are windows that
        #      IBC has some interest in monitoring, usually to take
        #      some action on the user's behalf;
        #
        #    - if set to 'unknown', only windows that IBC does not
        #      recognize are eligible. Most windows displayed by
        #      TWS fall into this category;
        #
        #    - if set to 'untitled', only windows that IBC does not
        #      recognize and that have no title are eligible. These
        #      are usually message boxes or similar small windows,
        #
        #    - if set to 'all', then every window displayed by TWS
        #      is eligible.
        #

        LogStructureScope=known

        # When to Log Window Structure
        # ----------------------------
        #
        # The LogStructureWhen setting specifies the circumstances
        # when eligible TWS windows have their structure logged:
        #
        #     - if set to 'open' or 'yes' or 'true', IBC logs the
        #       structure of an eligible window the first time it
        #       is encountered;
        #
        #     - if set to 'openclose', the structure is logged every
        #       time an eligible window is opened or closed;
        #
        #    - if set to 'activate', the structure is logged every
        #      time an eligible window is made active;
        #
        #    - (default value) if set to 'never' or 'no' or 'false',
        #      structure information is never logged.
        #

        LogStructureWhen=never

        # DEPRECATED SETTING
        # ------------------
        #
        # LogComponents - THIS SETTING WILL BE REMOVED IN A FUTURE
        # RELEASE
        #
        # If LogComponents is set to any value, this is equivalent
        # to setting LogStructureWhen to that same value and
        # LogStructureScope to 'all': the actual values of those
        # settings are ignored. The default is that the values
        # of LogStructureScope and LogStructureWhen are honoured.

        #LogComponents=

gnzsnz commented 11 months ago

Hi,

I'm glad to help. But i would need more information. based the error message that you reported it seems like an error with port 4000.

I'm not sure why you are sharing IBC's config file. it's not needed as it's by default part of the container. By default IBC will change IB gateways port to 4000. but on the file that you shared it's not using port 4000 --> OverrideTwsApiPort=. So i don't know why you are sharing it, or what is your expectation on the file that you are sharing. But something is telling me that might be cause of the problem.

So as i said I'm happy to help. but i would need more details on what you are doing.

if you copy the docker-compose.yml file and create an .env file with the content shown on the README.md you get a working container. as simple as that. that's actually my objective that this just works.

Questions

1) are you pulling the image from ghcr.io? 2) are you building it locally? in this case many things can go wrong 3) if (1) can you share the output of docker compose config. please exclude your user&pass 4) can you share the output of docker compose up

tomer-ds commented 11 months ago

Ok so I think I understood my disconnect. I was attempting to use my own configuration file without prompting the image with the env "CUSTOM_CONFIG" I guess I need also need to work on that configuration. Since I am running this in Kubernetes, my configMap will overwrite whatever file you have put in the image, but I guess this is what was breaking, because I configured the deployment with a selection of the EnvVars you require (analogous to .env file) and now the error has stopped

env:
  - name: TWS_USERID
    value: {{ .Values.ibcConfig.twsUserId }}
  - name: TWS_PASSWORD
    value: {{ .Values.ibcConfig.twsPassword }}
  - name: TRADING_MODE
    value: {{ .Values.ibcConfig.tradingMode }}
  - name: READ_ONLY_API
    value: "no"
  - name: VNC_SERVER_PASSWORD
    value: {{ .Values.ibcConfig.vncServerPassword }}
  - name: TWOFA_TIMEOUT_ACTION
    value: restart
  - name: AUTO_RESTART_TIME
    value: 8:00AM
  - name: RELOGIN_AFTER_2FA_TIMEOUT
    value: "yes"

Anyway, huge thanks again for the great image. If you would like to have a look at my helm chart just let me know, might be a nice addition to your project It would require some cooperation between us to ensure everything is exposed as per your requirements, but totally doable

tomer-ds commented 11 months ago

Apologies for closing and reopening... I litterally just checked back and I have started seeing new error

socat[272] E connect(5, AF=2 127.0.0.1:4000, 16): Connection timed out

Can you tell me how this port is connected? Does socat request somehow require 4000 to open on the container? Maybe I need to open it on the kubernetes pod?

tomer-ds commented 11 months ago

I realize I never answered all your questions:

are you pulling the image from ghcr.io? - yes
are you building it locally? in this case many things can go wrong - no, kubernetes is pulling from ghcr

if (1) can you share the output of docker compose config. please exclude your user&pass This is the kubernetes manifest as helm creates it... does it help?


---
# Source: ib-gateway/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: ib-gateway
labels:
helm.sh/chart: ib-gateway-0.1.0
app.kubernetes.io/name: ib-gateway
app.kubernetes.io/instance: ib-gateway
app.kubernetes.io/version: "1.16.0"
app.kubernetes.io/managed-by: Helm
spec:
replicas: 1
selector:
matchLabels:
  app.kubernetes.io/name: ib-gateway
  app.kubernetes.io/instance: ib-gateway
template:
metadata:
  labels:
    app.kubernetes.io/name: ib-gateway
    app.kubernetes.io/instance: ib-gateway
spec:
  serviceAccountName: ib-gateway
  securityContext:
    {}
  containers:
    - name: ib-gateway
      securityContext:
        {}
      env:
        - name: TWS_USERID
          value: *******
        - name: TWS_PASSWORD
          value: *******
        - name: TRADING_MODE
          value: paper
        - name: READ_ONLY_API
          value: "no"
        - name: VNC_SERVER_PASSWORD
          value: ********
        - name: TWOFA_TIMEOUT_ACTION
          value: restart
        - name: AUTO_RESTART_TIME
          value: 8:00AM
        - name: RELOGIN_AFTER_2FA_TIMEOUT
          value: "yes"
      image: "ghcr.io/gnzsnz/ib-gateway:stable"
      imagePullPolicy: IfNotPresent
      resources:
        {}
      ports:
        - name: ibgatewaylive
          containerPort: 4001
          protocol: TCP
        - name: ibgatewaypaper
          containerPort: 4002
          protocol: TCP
        - name: vnc
          containerPort: 5900
          protocol: TCP



4. can you share the output of docker compose up - How necessary is this? If crucial I will do so. I was hoping to avoid this process, so all I've been working with is your image from GHCR and not the repo itself

tomer-ds commented 11 months ago

Some context to the error

2023-10-30 09:57:52:945 IBC: detected frame entitled: Starting application...; event=Closed
2023-10-30 09:57:52:946 IBC: Login has completed
2023-10-30 09:57:52:948 IBC: detected dialog entitled: DU8017938 Trader Workstation Configuration (Simulated Trading); event=Focused
2023-10-30 09:57:53:016 IBC: Performing port configuration
Forking :::4000 onto 0.0.0.0:4002
2023/10/30 08:04:33 socat[244] E connect(5, AF=2 127.0.0.1:4000, 16): Connection timed out
2023/10/30 08:04:37 socat[245] E connect(5, AF=2 127.0.0.1:4000, 16): Connection timed out
2023/10/30 08:04:39 socat[246] E connect(5, AF=2 127.0.0.1:4000, 16): Connection timed out

Would yo prefer the entire log?

gnzsnz commented 11 months ago

hi,

you need to look the scripts used by the container.

run.sh script drives everything. your problem is related to fork_ports_delayed.sh.

if you don't use CUSTOM_CONFIG then the IBC config.ini file in the container will change IB gateways port from 4000 to 4001/4002, and socat will be free to use port 4000. this is something that i plan to change at some point because is confusing. but is how the original dev set it.

if you ARE using CUSTOM_CONFIG then you need to handle the port yourself. and this is what (I think) you are not doing. check what the container does, understand it and then put your changes on top. I won't work otherwise.

tomer-ds commented 11 months ago

How can I contribute? Would you prefer me to create a branch with a PR or maybe a fork?

gnzsnz commented 11 months ago

@tomer-ds there is nothing to commit into the repo. you just need to adjust your settings accordingly.

tomer-ds commented 11 months ago

So I tried both scenarios and even if I do not give custom config, i still get the error

tomer-ds commented 11 months ago

I was thinking maybe it was an issue with Socat running in Kubernetes POD... So I found a way to get Socat as sidecar container, but in order to support this I need to edit run.sh and add the option to NOT run socat via the container itself

gnzsnz commented 11 months ago

I'm afraid that I won't be able to help you.

the container works well, it's an issue your setup. a fresh container pulled from ghcr.io using the docker-compose.yml and .env file in the README.md file works just fine.

I can't help you with helm/kubernetes.

make sure that when you start the container you can see

ib-gateway-docker-ib-gateway-1  | 2023-11-01 12:26:01:908 IBC: Performing port configuration
ib-gateway-docker-ib-gateway-1  | 2023-11-01 12:26:01:908 IBC: TWS API socket port was set to 4002
ib-gateway-docker-ib-gateway-1  | 2023-11-01 12:26:01:909 IBC: TWS API socket port now set to 4000

and a few lines bellow

ib-gateway-docker-ib-gateway-1  | Forking :::4000 onto 0.0.0.0:4002

This will change default port from 4001/4002 to 4000 (the container does it so you don't have to do it) and then socat will pipe anything connecting in 0.0.0.0:4001/4002 to localhost:4000. if you overwrite what the container does then you need to manage it by yourself.

tomer-ds commented 11 months ago

Quite right, indeed docker compse works just fine. I guess it has something to do with what is going in the kubernetes pod. Anyway I guess I will go and find that out.

Thanks anyway for the great container! I will create a fork and see what magic I can do 🙏🏼

jcity commented 10 months ago

@gnzsnz thank you for putting this image together. Looking for a little guidance on using it - I think I'm close but might be doing something obviously wrong. Maybe this will help someone in the future as well.

I'm creating an image with this Dockerfile

FROM ghcr.io/gnzsnz/tws-rdesktop:10.26

# Prepare system
RUN apt-get update -y && \
  DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends --yes \
  telnet \
  vim \
  python3-pip \
  python3-setuptools \
  && if test "$(dpkg --print-architecture)" = "armhf" ; then python3 -m pip config set global.extra-index-url https://www.piwheels.org/simple ; fi \
  && pip install pipenv \
  && pip uninstall -y setuptools \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/* \

WORKDIR /home/ibgateway/

COPY Pipfile .
COPY Pipfile.lock .
COPY code ./code

COPY config/ibc/config.ini.tmpl /opt/ibc/config.ini.tmpl

# Install dependencies
RUN sudo pipenv install --system --deploy

This builds and starts successfully with the following docker-compose.yml file:

version: "3.4"

name: algo-trader
services:
  tws:
    restart: always
    shm_size: "2gb" #optional
    security_opt:
      - seccomp:unconfined #optional
    build:
      context: .
      dockerfile: Dockerfile
      tags:
        - "ib-gateway:latest"
    image: ib-gateway:latest
    platform: linux/amd64
    environment:
      PUID: 1000
      PGID: 1000
      PASSWD: ${PASSWD:-}
      TWS_USERID: ${TWS_USERID}
      TWS_PASSWORD: ${TWS_PASSWORD}
      TRADING_MODE: ${TRADING_MODE:-paper}
      TWS_SETTINGS_PATH: ${TWS_SETTINGS_PATH:-}
      READ_ONLY_API: ${READ_ONLY_API:-}
      TWOFA_TIMEOUT_ACTION: ${TWOFA_TIMEOUT_ACTION:-exit}
      BYPASS_WARNING: ${BYPASS_WARNING:-}
      AUTO_RESTART_TIME: ${AUTO_RESTART_TIME:-}
      AUTO_LOGOFF_TIME: ${AUTO_LOGOFF_TIME:-}
      SAVE_TWS_SETTINGS: ${SAVE_TWS_SETTINGS:-}
      RELOGIN_AFTER_TWOFA_TIMEOUT: ${RELOGIN_AFTER_TWOFA_TIMEOUT:-no}
      TWOFA_EXIT_INTERVAL: ${TWOFA_EXIT_INTERVAL:-60}
      TIME_ZONE: ${TIME_ZONE:-Etc/UTC}
      TZ: ${TIME_ZONE:-Etc/UTC}
      CUSTOM_CONFIG: ${CUSTOM_CONFIG:-NO}
      SSH_TUNNEL: ${SSH_TUNNEL:-}
      SSH_OPTIONS: ${SSH_OPTIONS:-}
      SSH_ALIVE_INTERVAL: ${SSH_ALIVE_INTERVAL:-}
      SSH_ALIVE_COUNT: ${SSH_ALIVE_COUNT:-}
      SSH_PASSPHRASE: ${SSH_PASSPHRASE:-}
      SSH_REMOTE_PORT: ${SSH_REMOTE_PORT:-}
      SSH_USER_TUNNEL: ${SSH_USER_TUNNEL:-}
      SSH_RESTART: ${SSH_RESTART:-}
      SSH_RDP_PORT: ${SSH_RDP_PORT:-}
    ports:
      - "127.0.0.1:7496:7498" # live
      - "127.0.0.1:7497:7499" # paper
      - "127.0.0.1:3370:3389" # xrdp

And this .env file

TWS_USERID=my-user-id
TWS_PASSWORD=my-pass
TWS_SETTINGS_PATH=
TRADING_MODE=paper
READ_ONLY_API=no
VNC_SERVER_PASSWORD=myVncPassword
TWOFA_TIMEOUT_ACTION=restart
BYPASS_WARNING=
AUTO_RESTART_TIME=11:59 PM
AUTO_LOGOFF_TIME=
SAVE_TWS_SETTINGS=
RELOGIN_AFTER_2FA_TIMEOUT=yes
TIME_ZONE=Etc/UTC
CUSTOM_CONFIG=
SSH_TUNNEL=
SSH_OPTIONS=
SSH_ALIVE_INTERVAL=
SSH_ALIVE_COUNT=
SSH_PASSPHRASE=
SSH_REMOTE_PORT=
SSH_USER_TUNNEL=
SSH_RESTART=
SSH_VNC_PORT=

The one change to config.ini.tmpl I made was to set AcceptIncomingConnectionAction=accept

The container starts successfully and I can see that port 7499 is open but every connection request is being refused:

algo-trader-tws-1  | Forking :::7497 onto 0.0.0.0:7499 > trading mode paper 
algo-trader-tws-1  | 2023/11/30 21:26:41 socat[1494] E connect(5, AF=2 127.0.0.1:7497, 16): Connection refused
algo-trader-tws-1  | 2023/11/30 21:28:47 socat[1699] E connect(5, AF=2 127.0.0.1:7497, 16): Connection refused
algo-trader-tws-1  | 2023/11/30 21:33:34 socat[2196] E connect(5, AF=2 127.0.0.1:7497, 16): Connection refused
algo-trader-tws-1  | 2023/11/30 21:36:38 socat[2494] E connect(5, AF=2 127.0.0.1:7497, 16): Connection refused
algo-trader-tws-1  | 2023/11/30 21:39:15 socat[2750] E connect(5, AF=2 127.0.0.1:7497, 16): Connection refused
algo-trader-tws-1  | 2023/11/30 21:40:57 socat[2926] E connect(5, AF=2 127.0.0.1:7497, 16): Connection refused

I'm exec'ing into the container and trying to connect via telnet and through a python script using ib_insync. This is the script

import asyncio
import logging
import time

from ib_insync import IB, util

util.patchAsyncio()

def onConnected():
    logging.info("CONNECTED TO IB")
    time.sleep(10)

ib = IB()
ib.connectedEvent += onConnected
completion_future = asyncio.Future()

ib.connect(
    "127.0.0.1",
    7499,
    clientId=1,
    timeout=4,
    account="my-account-number",
)

If I'm following along correctly I should be able to connect to 7499 through my script because the container is exposing that port locally. Or do I need to go the ssh bastion route?

Appreciate any tips you might have

jcity commented 10 months ago

Alright, I got something basic working. I am just going to drop this here in case anyone comes across it later:

Dockerfile:

# Use latest stable version
FROM ghcr.io/gnzsnz/ib-gateway:10.19

# Prepare system
USER root
RUN apt-get update -y && \
  DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends --yes \
  python3-pip \
  python3-setuptools \
  && if test "$(dpkg --print-architecture)" = "armhf" ; then python3 -m pip config set global.extra-index-url https://www.piwheels.org/simple ; fi \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/* \
  && pip install pipenv 

# Copy app files
COPY Pipfile /home/ibgateway
COPY Pipfile.lock /home/ibgateway
COPY code /home/ibgateway/code

# Install dependencies
RUN pipenv install --system --deploy

RUN chown -R ibgateway:ibgateway /home/ibgateway/code

USER ibgateway
WORKDIR /home/ibgateway

Docker Compose file:

version: "3.4"

name: ib-gateway
services:
  ib-gateway:
    restart: always
    build:
      context: .
      dockerfile: Dockerfile
      tags:
        - "ib-gateway:latest"
    image: ib-gateway:latest
    platform: linux/amd64
    environment:
      TWS_USERID: ${TWS_USERID}
      TWS_PASSWORD: ${TWS_PASSWORD}
      TRADING_MODE: ${TRADING_MODE:-paper}
      TWS_SETTINGS_PATH: ${TWS_SETTINGS_PATH:-}
      READ_ONLY_API: ${READ_ONLY_API:-}
      VNC_SERVER_PASSWORD: ${VNC_SERVER_PASSWORD:-}
      TWOFA_TIMEOUT_ACTION: ${TWOFA_TIMEOUT_ACTION:-exit}
      BYPASS_WARNING: ${BYPASS_WARNING:-}
      AUTO_RESTART_TIME: ${AUTO_RESTART_TIME:-}
      AUTO_LOGOFF_TIME: ${AUTO_LOGOFF_TIME:-}
      SAVE_TWS_SETTINGS: ${SAVE_TWS_SETTINGS:-}
      RELOGIN_AFTER_TWOFA_TIMEOUT: ${RELOGIN_AFTER_TWOFA_TIMEOUT:-no}
      TWOFA_EXIT_INTERVAL: ${TWOFA_EXIT_INTERVAL:-60}
      TIME_ZONE: ${TIME_ZONE:-Etc/UTC}
      TZ: ${TIME_ZONE:-Etc/UTC}
      CUSTOM_CONFIG: ${CUSTOM_CONFIG:-NO}
      SSH_TUNNEL: ${SSH_TUNNEL:-}
      SSH_OPTIONS: ${SSH_OPTIONS:-}
      SSH_ALIVE_INTERVAL: ${SSH_ALIVE_INTERVAL:-}
      SSH_ALIVE_COUNT: ${SSH_ALIVE_COUNT:-}
      SSH_PASSPHRASE: ${SSH_PASSPHRASE:-}
      SSH_REMOTE_PORT: ${SSH_REMOTE_PORT:-}
      SSH_USER_TUNNEL: ${SSH_USER_TUNNEL:-}
      SSH_RESTART: ${SSH_RESTART:-}
      SSH_VNC_PORT: ${SSH_VNC_PORT:-}
    ports:
      - "127.0.0.1:4001:4003"
      - "127.0.0.1:4002:4004"
      - "127.0.0.1:5900:5900"

Pipfile:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
ib-insync = "*"
asyncio = "*"
pandas = "*"

[dev-packages]
black = "*"

[requires]
python_version = "3.10"
python_full_version = "3.10.12"

Python script:

"""
main.py
"""
from ib_insync import *  # pylint: disable=wildcard-import,unused-wildcard-import

ib = IB()
ib.connect("127.0.0.1", 4004, clientId=1)

contract = Forex("EURUSD")
bars = ib.reqHistoricalData(
    contract,
    endDateTime="",
    durationStr="30 D",
    barSizeSetting="1 hour",
    whatToShow="MIDPOINT",
    useRTH=True,
)

df = util.df(bars)
print(df)

Don't forget to add a .env file before running docker compose up --build

TWS_USERID=your-user-id
TWS_PASSWORD=your-pass
TWS_SETTINGS_PATH=
TRADING_MODE=paper
READ_ONLY_API=no
VNC_SERVER_PASSWORD=myVncPassword
TWOFA_TIMEOUT_ACTION=restart
BYPASS_WARNING=
AUTO_RESTART_TIME=11:59 PM
AUTO_LOGOFF_TIME=
SAVE_TWS_SETTINGS=
RELOGIN_AFTER_2FA_TIMEOUT=yes
TIME_ZONE=Etc/UTC
CUSTOM_CONFIG=
SSH_TUNNEL=
SSH_OPTIONS=
SSH_ALIVE_INTERVAL=
SSH_ALIVE_COUNT=
SSH_PASSPHRASE=
SSH_REMOTE_PORT=
SSH_USER_TUNNEL=
SSH_RESTART=
SSH_VNC_PORT=

After running docker compose up --build you can exec into the container and run your python script

❯ docker exec -it $(docker ps --latest --quiet) python3 code/main.py
                         date     open     high      low    close  volume  average  barCount
0   2023-10-22 17:15:00-04:00  1.05970  1.05975  1.05950  1.05965    -1.0     -1.0        -1
1   2023-10-22 18:00:00-04:00  1.05965  1.05970  1.05890  1.05915    -1.0     -1.0        -1
2   2023-10-22 19:00:00-04:00  1.05915  1.05935  1.05885  1.05895    -1.0     -1.0        -1
3   2023-10-22 20:00:00-04:00  1.05895  1.05905  1.05840  1.05870    -1.0     -1.0        -1
4   2023-10-22 21:00:00-04:00  1.05870  1.05875  1.05820  1.05835    -1.0     -1.0        -1
..                        ...      ...      ...      ...      ...     ...      ...       ...
697 2023-11-30 18:00:00-05:00  1.08915  1.08945  1.08895  1.08920    -1.0     -1.0        -1
698 2023-11-30 19:00:00-05:00  1.08920  1.09030  1.08920  1.09020    -1.0     -1.0        -1
699 2023-11-30 20:00:00-05:00  1.09020  1.09100  1.09015  1.09070    -1.0     -1.0        -1
700 2023-11-30 21:00:00-05:00  1.09070  1.09130  1.09065  1.09075    -1.0     -1.0        -1
701 2023-11-30 22:00:00-05:00  1.09075  1.09095  1.08995  1.09010    -1.0     -1.0        -1

[702 rows x 8 columns]

Directory structure should look like this:

❯ tree
.
├── Pipfile
├── Pipfile.lock
├── Dockerfile
├── docker-compose.yml
├── code
│   ├── __init__.py
│   └── main.py

gnzsnz commented 10 months ago

Hi @jcity

Glad to see you have it working.

I'll share a sample docker-compose.yml with a different approach

This compose will

start a container with ib-gateway
start a container with jupyter for research
start a container with trading algos (like your python scripts). this would be based on python base image with your scripts on top of it for example

ib-gateway is not publishing any tws api port to the outside, jupyter and lago services can access ib-gateway on port 4003/4003

ib = IB()
ib.connect("ib-gateway", 4004, clientId=1) # host = service name

and the sample docker file

name: algo-trader
services:
  ib-gateway:
    restart: always
    image: ghcr.io/gnzsnz/ib-gateway:stable
    environment:
      TWS_USERID: ${TWS_USERID}
      TWS_PASSWORD: ${TWS_PASSWORD}
      TRADING_MODE: ${TRADING_MODE:-paper}
# ... yada-yada-yada
    ports:
      #- "127.0.0.1:4001:4003" # don't share outside
      #- "127.0.0.1:4002:4004"
      - "127.0.0.1:5900:5900"
    networks:
      - quant
  jupyterquant:     
      image: ghcr.io/gnzsnz/jupyter-quant:latest
      environment:
        APT_PROXY: ${APT_PROXY}
        QB_CONF: ${QB_CONF}  
# ... yada-yada-yada
      networks:
        - quant
      volumes:        
        - ${PWD}/Documents/Notebooks:/home/gordon/Notebooks
      ports:
        - 8888:8888
  algo:     
      image: python-based # for example your scripts
      environment:
        APT_PROXY: ${APT_PROXY}
        QB_CONF: ${QB_CONF}  
# ... yada-yada-yada
      networks:
        - quant
      volumes_from:        
        - jupyterquant
      ports:
        - 8888:8888

networks:
  - quant

this is just a sample, it won't work without modifications.

it's simpler to keep the ib-gateway or tws-rdesktop images as they are and just connect to the port. with this approach you can't connect to TWS api port from the outside (it's relatively safe)

hope it makes sense

gnzsnz commented 10 months ago

@jcity regarding what migth be causing your errors

on the first Dockerfile, you are using TWS image. and using workdir /home/ibgateway, that won't work.
you are refering to pi wheels. the images is amd64, it won't work on a pi
if you use TWS, make sure that "enable active X and socket API" is checked on settings.
If you could show me the error that your changes on config.ini solved, I would look at it. I never needed to set it up. but after checking IBC's documentation, it might make sense to set it to 'accept', specially for TWS. However I never had problems with it. Probably because i uses save settings on server, but i'm not sure that's why i'm not having any error.

jcity commented 10 months ago

@gnzsnz thanks for the tips!

I like the approach you provided running the "python scripts" as a separate container and linking it with a network.

What I'm doing is essentially building a REST API (using FastAPI) as a "wrapper" over IBC. So I was about to go down the route of creating a new entrypoint that calls your start.sh script and then starts the uvicorn process after. I have that working but I think your approach is much cleaner so I'm going to give that a shot instead.

gnzsnz commented 10 months ago

@jcity , if you feel that AcceptIncomingConnectionAction=accept please let me know. and if possible provide error message, use cases, etc

Mikkel84 commented 9 months ago

I also had the errors above a lot. It actually means that you are trying to use the connection before it is established. The log for the port fork comes before the ibc is started in run.sh and once I see the log entry like IBC: detected dialog entitled: DU******* Trader Workstation Configuration (Simulated Trading); event=Closed I can use the connection to IB. Took me some time to find this, maybe it is of help for someone.

Happy New Year 2024 and thx a lot for providing and maintaining this repo!

gnzsnz commented 8 months ago

@Mikkel84 , do you have reproduction steps? if so, could you please share them. I don't have this issue.

willzhqiang commented 3 months ago

I also had the errors above a lot. It actually means that you are trying to use the connection before it is established. The log for the port fork comes before the ibc is started in run.sh and once I see the log entry like IBC: detected dialog entitled: DU******* Trader Workstation Configuration (Simulated Trading); event=Closed I can use the connection to IB. Took me some time to find this, maybe it is of help for someone.

Happy New Year 2024 and thx a lot for providing and maintaining this repo!

Yes, it's great. I noticed that I need to confirm with my IB Key Authentication app before Docker can log in and allow me to connect.

gnzsnz commented 3 months ago

Thanks, now that I read that again I understand. It only took me 5 months 🤣

gnzsnz / ib-gateway-docker

Socat error connection refused #34

Questions