`substreams run` experiencing `subscription channel at max capacity` errors

[maoueh]

From https://discord.com/channels/666749063386890256/982135810742697984/1162013711368732733

My stream is stuck like this, and doesn't receive any message for more than 30mins.

substreams run -e mainnet.eth.streamingfast.io:443 ./substreams.yaml map_balances --start-block 18333500 --production-mode
Connected (trace ID da39f6b174f859b0b168994f7918d415)
Progress messages received: 0 (0/sec)
Backprocessing history up to requested target block 18333500:
(hit 'm' to switch mode)

...

rpc error: code = Internal desc = subscription channel at max capacity

what does it mean ?

[AngeloCa]

Here is the substream info

Package name: balance_changes
Version: v0.1.0
Modules:
----
Name: map_tokens
Initial block: 18333500
Kind: map
Output Type: proto:aleno.tokens.v1.Token
Hash: 69cf78573e4552043e7390ab513e7414b5c8728e

Name: store_tokens
Initial block: 18333500
Kind: store
Value Type: proto:aleno.tokens.v1.Token
Update Policy: UPDATE_POLICY_SET
Hash: 16d299e2fd60849e838cebcfbadbe2bed37a891f

Name: map_balances
Initial block: 18333500
Kind: map
Output Type: proto:aleno.balance_changes.v1.BalanceChanges
Hash: 7db0852137150abdbd99b3ea2e45e196a535f5f6

Name: eth_tokens:store_first_token_transfer_block
Initial block: 18333500
Kind: store
Value Type: int64
Update Policy: UPDATE_POLICY_SET_IF_NOT_EXISTS
Hash: 41b5aec639f740407222e903c9ea10bee65940e1

Name: eth_tokens:map_tokens
Initial block: 18333500
Kind: map
Output Type: proto:aleno.tokens.v1.Tokens
Hash: bcf39a9ee1dbe866b68a73fa0c5517dbc45fd469

[0xsarvesh]

Hey,

We have similar problem with our different substreams.

This can happen if the rate of incoming data exceeds the rate at which the data is being processed.

@maoueh Can you explain this statement. Does it mean consumer is slow at processing & firehose node is streaming fast? I looked into code, channel size is 128 hardcoded in code. can this be improved to accept as a configuration variable?