[Codelab Feedback]: Change the visualiser so that you can subscribe to the topic directly

googlecodelabs / cloud-dataflow-nyc-taxi-tycoon

This is the support code and solutions for the NYC Taxi Tycoon Dataflow Codelab

Apache License 2.0

60 stars 39 forks source link

[Codelab Feedback]: Change the visualiser so that you can subscribe to the topic directly #1

Closed martin-gorner closed 7 years ago

martin-gorner commented 7 years ago

Currently, the visualiser can only connect to PubSub feeds from your own cloud accounts. It cannot connect to a PubSub feed that is not in one of your accounts, even if it is public. It would be nice if you could connect to the public feed without having to go through Dataflow.

lifeofpy commented 6 years ago

The Cloud Dataflow job fails without start. The error is - insufficient quota. This seems very odd. i tried with various values numWorkers - 3, 10, 30. None of them worked. Have tested subscribing to the pubsub topic and it is definitely working. I am using the GCP free trial which provides me $300 free credit. Does anyone have a similar issue. Any help would be great.

Thanks, Sasi

martin-gorner commented 6 years ago

The $300 free trial limits you to 8 CPUs. Since each worker has 4 cores, you need to set numWorkers=2