googleapis / synthtool

Apache License 2.0
89 stars 84 forks source link

Autosynth PR has multiple commits, but snags one of their commit messages #580

Closed JustinBeckwith closed 4 years ago

JustinBeckwith commented 4 years ago

This PR caught my eye today: https://github.com/googleapis/google-api-nodejs-client/pull/2182

It's interesting because the PR has the title of build: do not fail builds on codecov errors, making me think that's the only content. However, it has a second commit, with MANY changes in it: https://github.com/googleapis/google-api-nodejs-client/pull/2182/commits

SurferJeffAtGoogle commented 4 years ago

I notice that in https://github.com/googleapis/google-api-nodejs-client/commit/216135c2fb48fa7260e030fa5f86528bf27d9f16, the following code was removed from synth.py:

subprocess.run(['npm', 'i'])
subprocess.run(['npm', 'run', 'generate'])

What is autodisco and why are we using it?

Anyways, https://github.com/googleapis/google-api-nodejs-client/pull/2182 is a mess because the output of these two npm commands is highly variable.

To reproduce the behavior that created https://github.com/googleapis/google-api-nodejs-client/pull/2182, I

  1. Rewound the repo to 3466b1d0cd869f8c826d0d93213f29aec8776df6 before the code above was deleted.
  2. Ran synth.py and saved the changes in branch A.
  3. Reset the repo to 3466b1d0cd869f8c826d0d93213f29aec8776df6.
  4. Ran synth.py and saved the changes in branch B.
  5. Compared branches A and B. The diff is below.

Context-aware commits require that for the same input, synth.py always generates the same output. Because that's not true for these two npm commands, we should shut off context-aware commits for the target repo.

What does @JustinBeckwith want to do with https://github.com/googleapis/google-api-nodejs-client? Leave a comment in synth.py describing what autodisco is, what it does, and why we use it? Or go back to calling the npm commands and turn of context-aware commits for this one repo?

diff --git a/discovery/adexchangebuyer2-v2beta1.json b/discovery/adexchangebuyer2-v2beta1.json
index 39f15771..778dacb9 100644
--- a/discovery/adexchangebuyer2-v2beta1.json
+++ b/discovery/adexchangebuyer2-v2beta1.json
@@ -2486,7 +2486,7 @@
       }
     }
   },
-  "revision": "20200525",
+  "revision": "20200527",
   "rootUrl": "https://adexchangebuyer.googleapis.com/",
   "schemas": {
     "AbsoluteDateRange": {
diff --git a/discovery/apigee-v1.json b/discovery/apigee-v1.json
index d28cc0ef..6e6d8693 100644
--- a/discovery/apigee-v1.json
+++ b/discovery/apigee-v1.json
@@ -1636,7 +1636,7 @@
                 "keys": {
                   "methods": {
                     "create": {
-                      "description": "Creates a custom consumer key and secret for a developer app. This is\nparticularly useful if you want to migrate existing consumer keys and\nsecrets to Apigee hybrid from another system.\n\nConsumer keys and secrets can contain letters, numbers, underscores, and\nhyphens. No other special characters are allowed.\n\n**Note**: To avoid service disruptions, a consumer key and secret\nshould not exceed 2 KBs each.\n\nAfter creating the consumer key and secret, associate the key with\nan API product using the\nUpdateDeveloperAppKey API.\n\nIf a consumer key and secret already exist, you can keep them or\ndelete them using the\nDeleteDeveloperAppKey API.",
+                      "description": "Creates a custom consumer key and secret for a developer app. This is\nparticularly useful if you want to migrate existing consumer keys and\nsecrets to Apigee hybrid from another system.\n\nConsumer keys and secrets can contain letters, numbers, underscores, and\nhyphens. No other special characters are allowed. To avoid service\ndisruptions, a consumer key and secret should not exceed 2 KBs each.\n\n**Note**: When creating the consumer key and secret, an association to\nAPI products will not be made. Therefore, you should not specify the\nassociated API products in your request. Instead, use the\nUpdateDeveloperAppKey API to\nmake the association after the consumer key and secret are created.\n\nIf a consumer key and secret already exist, you can keep them or\ndelete them using the\nDeleteDeveloperAppKey API.",
                       "flatPath": "v1/organizations/{organizationsId}/developers/{developersId}/apps/{appsId}/keys",
                       "httpMethod": "POST",
                       "id": "apigee.organizations.developers.apps.keys.create",
@@ -1838,7 +1838,7 @@
                     "create": {
                       "methods": {
                         "create": {
-                          "description": "Creates a custom consumer key and secret for a developer app. This is\nparticularly useful if you want to migrate existing consumer keys and\nsecrets to Apigee hybrid from another system.\n\nConsumer keys and secrets can contain letters, numbers, underscores, and\nhyphens. No other special characters are allowed.\n\n**Note**: To avoid service disruptions, a consumer key and secret\nshould not exceed 2 KBs each.\n\nAfter creating the consumer key and secret, associate the key with\nan API product using the\nUpdateDeveloperAppKey API.\n\nIf a consumer key and secret already exist, you can keep them or\ndelete them using the\nDeleteDeveloperAppKey API.",
+                          "description": "Creates a custom consumer key and secret for a developer app. This is\nparticularly useful if you want to migrate existing consumer keys and\nsecrets to Apigee hybrid from another system.\n\nConsumer keys and secrets can contain letters, numbers, underscores, and\nhyphens. No other special characters are allowed. To avoid service\ndisruptions, a consumer key and secret should not exceed 2 KBs each.\n\n**Note**: When creating the consumer key and secret, an association to\nAPI products will not be made. Therefore, you should not specify the\nassociated API products in your request. Instead, use the\nUpdateDeveloperAppKey API to\nmake the association after the consumer key and secret are created.\n\nIf a consumer key and secret already exist, you can keep them or\ndelete them using the\nDeleteDeveloperAppKey API.",
                           "flatPath": "v1/organizations/{organizationsId}/developers/{developersId}/apps/{appsId}/keys/create",
                           "httpMethod": "POST",
                           "id": "apigee.organizations.developers.apps.keys.create.create",
@@ -1992,7 +1992,7 @@
               ],
               "parameters": {
                 "name": {
-                  "description": "Optional. Name of the environment. Alternatively, the name may\nbe specified in the request body in the\nenvironment_id field.",
+                  "description": "Optional. Name of the environment. Alternatively, the name may\nbe specified in the request body in the\nname field.",
                   "location": "query",
                   "type": "string"
                 },
@@ -4554,7 +4554,7 @@
       }
     }
   },
-  "revision": "20200514",
+  "revision": "20200521",
   "rootUrl": "https://apigee.googleapis.com/",
   "schemas": {
     "GoogleApiHttpBody": {
@@ -5577,6 +5577,14 @@
         "name": {
           "description": "The name of the API or shared flow revision to be deployed. Must be of the\nform 'organizations/{org}/apis/{api}/revisions/{rev}' or\n'organizations/{org}/sharedflows/{sf}/revisions/{rev}'.",
           "type": "string"
+        },
+        "proxyUid": {
+          "description": "The uid of the proxy revision.",
+          "type": "string"
+        },
+        "uid": {
+          "description": "A unique id that will only change if the deployment is deleted and\nrecreated.",
+          "type": "string"
         }
       },
       "type": "object"
@@ -5731,7 +5739,7 @@
       "id": "GoogleCloudApigeeV1DeveloperAppKey",
       "properties": {
         "apiProducts": {
-          "description": "List of API products for which the credential can be used.",
+          "description": "List of API products for which the credential can be used.\n\n**Note**: Do not specify the list of API products when creating a consumer\nkey and secret for a developer app. Instead, use the\nUpdateDeveloperAppKey API to\nmake the association after the consumer key and secret are created.",
           "items": {
             "type": "any"
           },
@@ -5913,8 +5921,13 @@
           },
           "type": "array"
         },
+        "revisionId": {
+          "description": "Revision id that defines the ordering on the environment config.\nThe higher the revision, the more recently the configuration\nwas deployed.",
+          "format": "int64",
+          "type": "string"
+        },
         "sequenceNumber": {
-          "description": "A sequence number that defines an ordering on environment configs. No two\nconfigs under an environment will ever have the same sequence number.\nA higher sequence number means that the config was deployed more\nrecently than one with a lower sequence number.",
+          "description": "DEPRECATED: use revision_id",
           "format": "int64",
           "type": "string"
         },
@@ -5924,6 +5937,10 @@
             "$ref": "GoogleCloudApigeeV1TargetServerConfig"
           },
           "type": "array"
+        },
+        "uid": {
+          "description": "A unique id for the environment config that will only change if the\nenvironment is deleted and recreated.",
+          "type": "string"
         }
       },
       "type": "object"
@@ -6947,6 +6964,10 @@
           "format": "int64",
           "type": "string"
         },
+        "description": {
+          "description": "Description of the shared flow revision.",
+          "type": "string"
+        },
         "displayName": {
           "description": "The human readable name of this shared flow.",
           "type": "string"
diff --git a/discovery/bigquery-v2.json b/discovery/bigquery-v2.json
index 9063db9e..47fb87ad 100644
--- a/discovery/bigquery-v2.json
+++ b/discovery/bigquery-v2.json
@@ -1519,7 +1519,7 @@
       }
     }
   },
-  "revision": "20200514",
+  "revision": "20200523",
   "rootUrl": "https://bigquery.googleapis.com/",
   "schemas": {
     "AggregateClassificationMetrics": {
@@ -4312,6 +4312,13 @@
           "description": "The resource type of the request.",
           "type": "string"
         },
+        "labels": {
+          "additionalProperties": {
+            "type": "string"
+          },
+          "description": "The labels associated with this job. You can use these to organize and group your jobs. Label keys and values can be no longer than 63 characters, can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter and each label in the list must have a different key.",
+          "type": "object"
+        },
         "location": {
           "description": "The geographic location where the job should run. See details at https://cloud.google.com/bigquery/docs/locations#specifying_your_location.",
           "type": "string"
@@ -4321,6 +4328,11 @@
           "format": "uint32",
           "type": "integer"
         },
+        "maximumBytesBilled": {
+          "description": "[Optional] Limits the bytes billed for this job. Queries that will have bytes billed beyond this limit will fail (without incurring a charge). If unspecified, this will be set to your project default.",
+          "format": "int64",
+          "type": "string"
+        },
         "parameterMode": {
           "description": "Standard SQL only. Set to POSITIONAL to use positional (?) query parameters or to NAMED to use named (@myparam) query parameters in this query.",
           "type": "string"
@@ -4345,6 +4357,10 @@
           },
           "type": "array"
         },
+        "requestId": {
+          "description": "A unique user provided identifier to ensure idempotent behavior for queries. Note that this is different from the job_id. It has the following properties: 1. It is case-sensitive, limited to up to 36 ASCII characters. A UUID is recommended. 2. Read only queries can ignore this token since they are nullipotent by definition. 3. When a duplicate mutating query request is detected (i.e. having the same request_id as an earlier query), it returns: a. the results of the mutation if it completes successfully within the timeout. b. the running operation if it is still in progress at the end of the timeout. 4. Its lifetime is limited to 15 minutes. In other words, if two requests are sent with the same request_id, but more than 15 minutes apart, idempotency is not guaranteed.",
+          "type": "string"
+        },
         "timeoutMs": {
           "description": "[Optional] How long to wait for the query to complete, in milliseconds, before the request times out and returns. Note that this is only a timeout for the request, not the query. If the query takes longer to run than the timeout value, the call returns without any results and with the 'jobComplete' flag set to false. You can call GetQueryResults() to wait for the query to complete and read the results. The default value is 10000 milliseconds (10 seconds).",
           "format": "uint32",
diff --git a/discovery/bigtableadmin-v2.json b/discovery/bigtableadmin-v2.json
index 99c4a47e..79dc6f1c 100644
--- a/discovery/bigtableadmin-v2.json
+++ b/discovery/bigtableadmin-v2.json
@@ -1531,7 +1531,7 @@
       }
     }
   },
-  "revision": "20200415",
+  "revision": "20200424",
   "rootUrl": "https://bigtableadmin.googleapis.com/",
   "schemas": {
     "AppProfile": {
@@ -1950,7 +1950,7 @@
       "properties": {
         "options": {
           "$ref": "GetPolicyOptions",
-          "description": "OPTIONAL: A `GetPolicyOptions` object for specifying options to\n`GetIamPolicy`."
+          "description": "OPTIONAL: A `GetPolicyOptions` object for specifying options to\n`GetIamPolicy`. This field is only used by Cloud IAM."
         }
       },
       "type": "object"
@@ -2328,7 +2328,7 @@
           "description": "REQUIRED: The complete policy to be applied to the `resource`. The size of\nthe policy is limited to a few 10s of KB. An empty policy is a\nvalid policy but certain Cloud Platform services (such as Projects)\nmight reject them."
         },
         "updateMask": {
-          "description": "OPTIONAL: A FieldMask specifying which fields of the policy to modify. Only\nthe fields in the mask will be modified. If no mask is provided, the\nfollowing default mask is used:\n\n`paths: \"bindings, etag\"`",
+          "description": "OPTIONAL: A FieldMask specifying which fields of the policy to modify. Only\nthe fields in the mask will be modified. If no mask is provided, the\nfollowing default mask is used:\npaths: \"bindings, etag\"\nThis field is only used by Cloud IAM.",
           "format": "google-fieldmask",
           "type": "string"
         }
diff --git a/discovery/sql-v1beta4.json b/discovery/sql-v1beta4.json
index ee0805a8..de3ea9c2 100644
--- a/discovery/sql-v1beta4.json
+++ b/discovery/sql-v1beta4.json
@@ -1818,7 +1818,7 @@
       }
     }
   },
-  "revision": "20200518",
+  "revision": "20200331",
   "rootUrl": "https://sqladmin.googleapis.com/",
   "schemas": {
     "AclEntry": {
@@ -2181,8 +2181,7 @@
             "SQLSERVER_2017_EXPRESS",
             "SQLSERVER_2017_WEB",
             "POSTGRES_10",
-            "POSTGRES_12",
-            "MYSQL_8_0"
+            "POSTGRES_12"
           ],
           "enumDescriptions": [
             "This is an unknown database version.",
@@ -2197,8 +2196,7 @@
             "The database version is SQL Server 2017 Express.",
             "The database version is SQL Server 2017 Web.",
             "The database version is PostgreSQL 10.",
-            "The database version is PostgreSQL 12.",
-            "The database version is MySQL 8."
+            "The database version is PostgreSQL 12."
           ],
           "type": "string"
         },
@@ -2527,10 +2525,6 @@
           "description": "This is always <code>sql#exportContext</code>.",
           "type": "string"
         },
-        "offload": {
-          "description": "Option for export offload.",
-          "type": "boolean"
-        },
         "sqlExportOptions": {
           "description": "Options for exporting data as SQL statements.",
           "properties": {
@@ -2616,8 +2610,7 @@
             "The database version is SQL Server 2017 Express.",
             "The database version is SQL Server 2017 Web.",
             "The database version is PostgreSQL 10.",
-            "The database version is PostgreSQL 12.",
-            "The database version is MySQL 8."
+            "The database version is PostgreSQL 12."
           ],
           "items": {
             "enum": [
@@ -2633,8 +2626,7 @@
               "SQLSERVER_2017_EXPRESS",
               "SQLSERVER_2017_WEB",
               "POSTGRES_10",
-              "POSTGRES_12",
-              "MYSQL_8_0"
+              "POSTGRES_12"
             ],
             "type": "string"
           },
diff --git a/discovery/websecurityscanner-v1.json b/discovery/websecurityscanner-v1.json
index 7f73e451..d9a1d16b 100644
--- a/discovery/websecurityscanner-v1.json
+++ b/discovery/websecurityscanner-v1.json
@@ -526,7 +526,7 @@
       }
     }
   },
-  "revision": "20200516",
+  "revision": "20200522",
   "rootUrl": "https://websecurityscanner.googleapis.com/",
   "schemas": {
     "Authentication": {
diff --git a/discovery/websecurityscanner-v1alpha.json b/discovery/websecurityscanner-v1alpha.json
index 9550232d..d985a531 100644
--- a/discovery/websecurityscanner-v1alpha.json
+++ b/discovery/websecurityscanner-v1alpha.json
@@ -526,7 +526,7 @@
       }
     }
   },
-  "revision": "20200516",
+  "revision": "20200522",
   "rootUrl": "https://websecurityscanner.googleapis.com/",
   "schemas": {
     "Authentication": {
diff --git a/discovery/websecurityscanner-v1beta.json b/discovery/websecurityscanner-v1beta.json
index 4ed41a26..65f11d61 100644
--- a/discovery/websecurityscanner-v1beta.json
+++ b/discovery/websecurityscanner-v1beta.json
@@ -526,7 +526,7 @@
       }
     }
   },
-  "revision": "20200516",
+  "revision": "20200522",
   "rootUrl": "https://websecurityscanner.googleapis.com/",
   "schemas": {
     "Authentication": {
JustinBeckwith commented 4 years ago

What is autodisco and why are we using it?

autodisco is an experiment that has yet to bear fruit :) Node.js apiary was never really connected to the existing discovery-artifact-manager infrastructure, so all re-generations and releases are done by someone, usually me, cloning the repo, running npm run generate, and submitting a PR, followed by a release.

Autodisco is an attempt to automate this process, along with breaking change detection. It runs a pipeline that:

In many ways it's doing similar work to synthtool in a way that's highly stylized for discovery and the nodejs apiary library. This has been a personal 120% project over the last few weeks, largely born out of self preservation.

the output of these two npm commands is highly variable.

Yeah, that's very expected.

What does @JustinBeckwith want to do?

My intent is to forge ahead with autodisco in the name of experimentation, and stop using synthtool. If it's successful, it could be a way we choose to address debt with the discovery-artifact-manager broadly. Worst case, it's a snowflake in the node repo we keep managing.