Please provide some guidance on the difference between v1beta and v1

[evil-shrike]

Hi!

I needed to run a AutoML model in batch_predict mode and I tried to find some way to pass input BQ's table and output. But it turned out to be not very clear. I'd like to share my exprerience and ask for some improvement for documentation. First of all I looked into BQ's documentation - https://cloud.google.com/vision/automl/docs/predict-batch#automl_batch_predict-python There's a sample where input and output data are being passed via GCS. So I went to API doc to look up the reference of batch_predict method. And found that there's no an ability to specify any other input and output sources besides gcs. Which looked weird. But then I went to sources on github and found that two versions exist in the repo - v1 and v1beta1. I wouldn't care but I found BigQuerySource and BigQueryDestination types in v1beta1. And it was exactly what I needed! With REST APIs it's a similar picture: there was BigQuerySource in v1beta1 and it was removed in v1. So we are having two versions of AutoML side-by-side with different APIs and client libs. Using BigQuery as a source and destination for AutoML is obviously very naturally and handy. It's not quite clear why this ability was removed.

Obviously v1 is newer than v1beta1. And having a feature in v1beta1 that was removed in v1 provoking using v1beta1 what probably is a bad idea. But it'd be nice to have some guidance for migration and differences description from v1beta1 to v1 then.

[munkhuushmgl]

@evil-shrike Sorry to hear that, What type of model are you using for batch predict?

Recently, I helped the customer who had a similar issue.

Please take a look the following issue:

https://github.com/googleapis/python-automl/issues/20

[evil-shrike]

Hi Mike, thanks for looking into this, but I'm not sure I understand, in the mentioned issue you are using REST API v1beta1. No doubt that v1beta1 contains the ability to specify BQ source/destination, but I'm talking about that v1 supersedes v1beta1 and this feature was removed in it. As for type of model, it's Multi-class classification model, but why is it important? But I've realized that my question mostly related to AutoML and not Python client lib.

[munkhuushmgl]

@gguuss I may not be the best person to answer this. Could you answer this question please? Seems like very broad question

[gguuss]

Thanks for reporting the issue, it appears that the bigquery source/destination types that were available during beta were removed for the stable release - this is clear in the REST reference but awkwardly, there are some broken references still left in the RPC reference. Note that sometimes the product teams use the endpoints as an unstable channel and the endpoint for stable but I don't think that's the case in AutoML.

I have reported the issue in the documentation to our team, in the future, when you see these types of issues, you can report it directly from the documentation page and it will reach our writing team. Just look for the Send Feedback button near the "Was this page helpful?" likert range.

I'm following up with the team to make sure this was deliberate, usually there's a good reason for a feature not ending up in GA and hopefully I'll have answer for you.

I'm assuming in your use case you require BigQuery as a src/destination type and cannot use csv files? Is there a reason csv will not work?

[helinwang]

Obviously v1 is newer than v1beta1

I don't think this is true.

My understanding is:

• v1beta1 is for beta products (e.g., AutoML Tables) or GA product's beta features.
• v1 is for GA product's GA features

So the way to decide using v1beta1 vs. v1 is to check if the product (and feature) you are using is GA or beta.

For BigQuerySource and BigQueryDestination found in v1beta1 but not in v1. I think it's due to the GA products don't support BigQuerySource and BigQueryDestination. Which I think is the case for Vision (assuming you are referring to Vision). To verify it, you can check if the UI support importing / exporting from BigQuery.

[evil-shrike]

Hi Helin. I see, thanks for explaining v1 vs v1beta, it helps a lot. Sorry that I wasn't specific, but I was referring to AutoML Tables not Vision. And UI does have support for input/output in BQ. As AutoML Table isn't in GA, so probably there shouldn't be Table support in v1 at all.

[helinwang]

You are welcome. Yes, for AutoML Tables you should be only using v1beta.

[gguuss]

Thank you both, I'll go ahead and close the issue as it seems the high-level problem was some confusion on the release tags (beta vs GA), we'll still be tracking the documentation update internally and hopefully this will be made more clear in the docs.