pubsub: subscribe/publish errors for large messages (caused by tonic update)

[dezyh]

Currently pulling from a PubSub subscription with large messages (>4MB) will cause the stream to terminate. I have not tried but I believe publishing large messages (>4MB) to a topic will trigger the same stream termination.

The cause is a change in tonic 0.9.0, specifically this PR which adds a maximum encode/decode message size, which defaults to 4MB.

PubSub supports up to 10MB, so we should try update google-cloud-rust to specify a larger size when sending the stream request.

Basic Reproduction Case

Putting together something simple together and running with RUST_LOG=error cargo run

use futures_util::StreamExt;
use google_cloud_default::WithAuthExt;
use google_cloud_pubsub::client::{Client, ClientConfig};
use tracing_subscriber::{fmt, prelude::*, EnvFilter};

#[tokio::main]
async fn main() {
    tracing_subscriber::registry()
        .with(fmt::layer())
        .with(EnvFilter::from_default_env())
        .init();

    let config = ClientConfig::default().with_auth().await.unwrap();
    let client = Client::new(config).await.unwrap();

    let mut subscription = client.subscription("my-subscription");
    let mut messages = subscription.subscribe(None).await.unwrap();

    while let Some(msg) = messages.next().await {
        msg.ack().await.unwrap();
    }
}

Gives the following error:

ERROR google_cloud_pubsub::subscriber: terminated subscriber streaming with error Status { code: OutOfRange, message: "Error, message length too large: found 4750628 bytes, the limit is: 4194304 bytes", source: None } : projects/my-project/subscriptions/my-subscription

Which I've traced to this addition in the above PR.