search

ngulam-ai / Sherlock

0 stars 0 forks source link

clean datasets #50

Open ngulamai opened 6 years ago

ngulamai commented 6 years ago

Can you please remove all unnecesary datasets from bigquery?. Thanks

imagen
akolchin-MM commented 6 years ago

I will review datasets and will remove which is really not needed anymore. Just a couple of clarifications.

  1. DSData - was created by Vera (dashboard team). Maybe it will be better to move it to their own project (it should be possible). In any way, I don't know whether they still need it or not.

  2. The reason for dozens of empty datasets like temp_dataset_beam_job_0366b26ea1134098a74e0df453accd28_dailycostincrementspipelineuser07111048027dff7f57 is the known issue in the Dataflow which they still didn't fix. For now, the only known solution for it is to delete them manually periodically. https://github.com/GoogleCloudPlatform/DataflowJavaSDK/issues/609