yoshi-automation
commented
3 years ago Autosynth is still having trouble generating Dataproc. :sob:
Here's the output from running synth.py:
/lib/google_api/dataproc/v1/model/gce_cluster_config.ex.
Writing GetIamPolicyRequest to clients/dataproc/lib/google_api/dataproc/v1/model/get_iam_policy_request.ex.
Writing GetPolicyOptions to clients/dataproc/lib/google_api/dataproc/v1/model/get_policy_options.ex.
Writing HadoopJob to clients/dataproc/lib/google_api/dataproc/v1/model/hadoop_job.ex.
Writing HiveJob to clients/dataproc/lib/google_api/dataproc/v1/model/hive_job.ex.
Writing InjectCredentialsRequest to clients/dataproc/lib/google_api/dataproc/v1/model/inject_credentials_request.ex.
Writing InstanceGroupAutoscalingPolicyConfig to clients/dataproc/lib/google_api/dataproc/v1/model/instance_group_autoscaling_policy_config.ex.
Writing InstanceGroupConfig to clients/dataproc/lib/google_api/dataproc/v1/model/instance_group_config.ex.
Writing InstanceReference to clients/dataproc/lib/google_api/dataproc/v1/model/instance_reference.ex.
Writing InstantiateWorkflowTemplateRequest to clients/dataproc/lib/google_api/dataproc/v1/model/instantiate_workflow_template_request.ex.
Writing Job to clients/dataproc/lib/google_api/dataproc/v1/model/job.ex.
Writing JobMetadata to clients/dataproc/lib/google_api/dataproc/v1/model/job_metadata.ex.
Writing JobPlacement to clients/dataproc/lib/google_api/dataproc/v1/model/job_placement.ex.
Writing JobReference to clients/dataproc/lib/google_api/dataproc/v1/model/job_reference.ex.
Writing JobScheduling to clients/dataproc/lib/google_api/dataproc/v1/model/job_scheduling.ex.
Writing JobStatus to clients/dataproc/lib/google_api/dataproc/v1/model/job_status.ex.
Writing KerberosConfig to clients/dataproc/lib/google_api/dataproc/v1/model/kerberos_config.ex.
Writing LifecycleConfig to clients/dataproc/lib/google_api/dataproc/v1/model/lifecycle_config.ex.
Writing ListAutoscalingPoliciesResponse to clients/dataproc/lib/google_api/dataproc/v1/model/list_autoscaling_policies_response.ex.
Writing ListClustersResponse to clients/dataproc/lib/google_api/dataproc/v1/model/list_clusters_response.ex.
Writing ListJobsResponse to clients/dataproc/lib/google_api/dataproc/v1/model/list_jobs_response.ex.
Writing ListOperationsResponse to clients/dataproc/lib/google_api/dataproc/v1/model/list_operations_response.ex.
Writing ListWorkflowTemplatesResponse to clients/dataproc/lib/google_api/dataproc/v1/model/list_workflow_templates_response.ex.
Writing LoggingConfig to clients/dataproc/lib/google_api/dataproc/v1/model/logging_config.ex.
Writing ManagedCluster to clients/dataproc/lib/google_api/dataproc/v1/model/managed_cluster.ex.
Writing ManagedGroupConfig to clients/dataproc/lib/google_api/dataproc/v1/model/managed_group_config.ex.
Writing NodeGroupAffinity to clients/dataproc/lib/google_api/dataproc/v1/model/node_group_affinity.ex.
Writing NodeInitializationAction to clients/dataproc/lib/google_api/dataproc/v1/model/node_initialization_action.ex.
Writing Operation to clients/dataproc/lib/google_api/dataproc/v1/model/operation.ex.
Writing OrderedJob to clients/dataproc/lib/google_api/dataproc/v1/model/ordered_job.ex.
Writing ParameterValidation to clients/dataproc/lib/google_api/dataproc/v1/model/parameter_validation.ex.
Writing PigJob to clients/dataproc/lib/google_api/dataproc/v1/model/pig_job.ex.
Writing Policy to clients/dataproc/lib/google_api/dataproc/v1/model/policy.ex.
Writing PrestoJob to clients/dataproc/lib/google_api/dataproc/v1/model/presto_job.ex.
Writing PySparkJob to clients/dataproc/lib/google_api/dataproc/v1/model/py_spark_job.ex.
Writing QueryList to clients/dataproc/lib/google_api/dataproc/v1/model/query_list.ex.
Writing RegexValidation to clients/dataproc/lib/google_api/dataproc/v1/model/regex_validation.ex.
Writing ReservationAffinity to clients/dataproc/lib/google_api/dataproc/v1/model/reservation_affinity.ex.
Writing SecurityConfig to clients/dataproc/lib/google_api/dataproc/v1/model/security_config.ex.
Writing SetIamPolicyRequest to clients/dataproc/lib/google_api/dataproc/v1/model/set_iam_policy_request.ex.
Writing ShieldedInstanceConfig to clients/dataproc/lib/google_api/dataproc/v1/model/shielded_instance_config.ex.
Writing SoftwareConfig to clients/dataproc/lib/google_api/dataproc/v1/model/software_config.ex.
Writing SparkJob to clients/dataproc/lib/google_api/dataproc/v1/model/spark_job.ex.
Writing SparkRJob to clients/dataproc/lib/google_api/dataproc/v1/model/spark_r_job.ex.
Writing SparkSqlJob to clients/dataproc/lib/google_api/dataproc/v1/model/spark_sql_job.ex.
Writing Status to clients/dataproc/lib/google_api/dataproc/v1/model/status.ex.
Writing SubmitJobRequest to clients/dataproc/lib/google_api/dataproc/v1/model/submit_job_request.ex.
Writing TemplateParameter to clients/dataproc/lib/google_api/dataproc/v1/model/template_parameter.ex.
Writing TestIamPermissionsRequest to clients/dataproc/lib/google_api/dataproc/v1/model/test_iam_permissions_request.ex.
Writing TestIamPermissionsResponse to clients/dataproc/lib/google_api/dataproc/v1/model/test_iam_permissions_response.ex.
Writing ValueValidation to clients/dataproc/lib/google_api/dataproc/v1/model/value_validation.ex.
Writing WorkflowGraph to clients/dataproc/lib/google_api/dataproc/v1/model/workflow_graph.ex.
Writing WorkflowMetadata to clients/dataproc/lib/google_api/dataproc/v1/model/workflow_metadata.ex.
Writing WorkflowNode to clients/dataproc/lib/google_api/dataproc/v1/model/workflow_node.ex.
Writing WorkflowTemplate to clients/dataproc/lib/google_api/dataproc/v1/model/workflow_template.ex.
Writing WorkflowTemplatePlacement to clients/dataproc/lib/google_api/dataproc/v1/model/workflow_template_placement.ex.
Writing YarnApplication to clients/dataproc/lib/google_api/dataproc/v1/model/yarn_application.ex.
Writing Projects to clients/dataproc/lib/google_api/dataproc/v1/api/projects.ex.
Writing connection.ex.
Writing metadata.ex.
Writing mix.exs
Writing README.md
Writing LICENSE
Writing .gitignore
Writing config/config.exs
Writing test/test_helper.exs
13:48:35.004 [info] Found only discovery_revision and/or formatting changes. Not significant enough for a PR.
fixing file permissions
Traceback (most recent call last):
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>
main()
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main
spec.loader.exec_module(synth_module) # type: ignore
File "/tmpfs/src/github/synthtool/synthtool/metadata.py", line 252, in __exit__
self.observer.stop()
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/watchdog/utils/__init__.py", line 81, in stop
self.on_thread_stop()
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/watchdog/observers/api.py", line 361, in on_thread_stop
self.unschedule_all()
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/watchdog/observers/api.py", line 357, in unschedule_all
self._clear_emitters()
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/watchdog/observers/api.py", line 231, in _clear_emitters
emitter.stop()
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/watchdog/utils/__init__.py", line 81, in stop
self.on_thread_stop()
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/watchdog/observers/inotify.py", line 121, in on_thread_stop
self._inotify.close()
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/watchdog/observers/inotify_buffer.py", line 50, in close
self.stop()
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/watchdog/utils/__init__.py", line 81, in stop
self.on_thread_stop()
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/watchdog/observers/inotify_buffer.py", line 46, in on_thread_stop
self._inotify.close()
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/watchdog/observers/inotify_c.py", line 277, in close
os.close(self._inotify_fd)
OSError: [Errno 9] Bad file descriptor
2020-12-19 05:48:38,161 autosynth [ERROR] > Synthesis failed
2020-12-19 05:48:38,161 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Traceback (most recent call last):
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 354, in <module>
main()
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 189, in main
return _inner_main(temp_dir)
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 291, in _inner_main
).synthesize(synth_log_path / "sponge_log.log")
File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 120, in synthesize
synth_proc.check_returncode() # Raise an exception.
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode
self.stderr)
subprocess.CalledProcessError: Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'clients/dataproc/synth.metadata', 'synth.py', '--', 'Dataproc']' returned non-zero exit status 1.
Google internal developers can see the full log here.
Hello! Autosynth couldn't regenerate Dataproc. :broken_heart:
Here's the output from running
synth.py:Google internal developers can see the full log here.