search

googleapis / synthtool

Apache License 2.0
89 stars 84 forks source link

AttributeError: 'MetadataTrackerAndWriter' object has no attribute 'observer' #891

Closed 0xSage closed 3 years ago

0xSage commented 3 years ago

Package failing with this error: https://github.com/googleapis/nodejs-logging/issues/975

Repasting the error here: Autosynth is still having trouble generating nodejs-logging. :sob:

Here's the output from running synth.py:

BUG:synthtool:Ensuring dependencies.
2020-12-28 04:27:09,525 synthtool [DEBUG] > Cloning googleapis.
DEBUG:synthtool:Cloning googleapis.
2020-12-28 04:27:09,526 synthtool [DEBUG] > Using precloned repo /home/kbuilder/.cache/synthtool/googleapis
DEBUG:synthtool:Using precloned repo /home/kbuilder/.cache/synthtool/googleapis
2020-12-28 04:27:09,531 synthtool [DEBUG] > Generating code for: //google/logging/v2:logging-v2-nodejs.
DEBUG:synthtool:Generating code for: //google/logging/v2:logging-v2-nodejs.
2020-12-28 04:27:15,679 synthtool [SUCCESS] > Generated code into /tmpfs/tmp/tmp0lrr32om.
SUCCESS:synthtool:Generated code into /tmpfs/tmp/tmp0lrr32om.
2020-12-28 04:27:15,712 synthtool [DEBUG] > Using precloned repo /home/kbuilder/.cache/synthtool/synthtool
DEBUG:synthtool:Using precloned repo /home/kbuilder/.cache/synthtool/synthtool
.eslintignore
.eslintrc.json
.gitattributes
.github/ISSUE_TEMPLATE/bug_report.md
.github/ISSUE_TEMPLATE/feature_request.md
.github/ISSUE_TEMPLATE/support_request.md
.github/PULL_REQUEST_TEMPLATE.md
.github/release-please.yml
.github/workflows/ci.yaml
.kokoro/.gitattributes
.kokoro/common.cfg
.kokoro/continuous/node10/common.cfg
.kokoro/continuous/node10/docs.cfg
.kokoro/continuous/node10/test.cfg
.kokoro/continuous/node12/common.cfg
.kokoro/continuous/node12/lint.cfg
.kokoro/continuous/node12/samples-test.cfg
.kokoro/continuous/node12/system-test.cfg
.kokoro/continuous/node12/test.cfg
.kokoro/docs.sh
.kokoro/lint.sh
.kokoro/populate-secrets.sh
.kokoro/presubmit/node10/common.cfg
.kokoro/presubmit/node12/common.cfg
.kokoro/presubmit/node12/samples-test.cfg
.kokoro/presubmit/node12/system-test.cfg
.kokoro/presubmit/node12/test.cfg
.kokoro/publish.sh
.kokoro/release/docs-devsite.cfg
.kokoro/release/docs-devsite.sh
.kokoro/release/docs.cfg
.kokoro/release/docs.sh
.kokoro/release/publish.cfg
.kokoro/samples-test.sh
.kokoro/system-test.sh
.kokoro/test.bat
.kokoro/test.sh
.kokoro/trampoline.sh
.kokoro/trampoline_v2.sh
.mocharc.js
.nycrc
.prettierignore
.prettierrc.js
.trampolinerc
CODE_OF_CONDUCT.md
CONTRIBUTING.md
LICENSE
README.md
api-extractor.json
renovate.json
samples/README.md
2020-12-28 04:27:15,851 synthtool [DEBUG] > Post-processing GAPIC library...
DEBUG:synthtool:Post-processing GAPIC library...
2020-12-28 04:27:15,852 synthtool [DEBUG] > Installing dependencies...
DEBUG:synthtool:Installing dependencies...
npm WARN deprecated debug@4.2.0: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
npm WARN deprecated cheerio-select-tmp@0.1.1: Use cheerio-select instead
npm WARN deprecated fsevents@2.1.3: Please update to v 2.2.x

> protobufjs@6.10.2 postinstall /home/kbuilder/.cache/synthtool/nodejs-logging/node_modules/protobufjs
> node scripts/postinstall

> @google-cloud/logging@9.0.0 prepare /home/kbuilder/.cache/synthtool/nodejs-logging
> npm run compile-protos && npm run compile

> @google-cloud/logging@9.0.0 compile-protos /home/kbuilder/.cache/synthtool/nodejs-logging
> compileProtos src

installing tmp@^0.2.0
installing uglify-js@^3.7.7
installing espree@^7.0.0
installing escodegen@^2.0.0
installing estraverse@^5.1.0

> @google-cloud/logging@9.0.0 precompile /home/kbuilder/.cache/synthtool/nodejs-logging
> gts clean

version: 14
Removing build ...

> @google-cloud/logging@9.0.0 compile /home/kbuilder/.cache/synthtool/nodejs-logging
> tsc -p . && cp -r proto* build/

npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@~2.1.2 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 765 packages from 825 contributors and audited 778 packages in 35.792s

77 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

2020-12-28 04:27:52,382 synthtool [DEBUG] > Running prelint...
DEBUG:synthtool:Running prelint...

> @google-cloud/logging@9.0.0 prelint /home/kbuilder/.cache/synthtool/nodejs-logging
> cd samples; npm link ../; npm install

> @google-cloud/logging@9.0.0 prepare /home/kbuilder/.cache/synthtool/nodejs-logging
> npm run compile-protos && npm run compile

> @google-cloud/logging@9.0.0 compile-protos /home/kbuilder/.cache/synthtool/nodejs-logging
> compileProtos src

> @google-cloud/logging@9.0.0 precompile /home/kbuilder/.cache/synthtool/nodejs-logging
> gts clean

version: 14
Removing build ...

> @google-cloud/logging@9.0.0 compile /home/kbuilder/.cache/synthtool/nodejs-logging
> tsc -p . && cp -r proto* build/

npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@2.1.3 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

audited 778 packages in 20.64s

77 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

/home/kbuilder/.nvm/versions/node/v14.15.3/lib/node_modules/@google-cloud/logging -> /home/kbuilder/.cache/synthtool/nodejs-logging
/home/kbuilder/.cache/synthtool/nodejs-logging/samples/node_modules/@google-cloud/logging -> /home/kbuilder/.nvm/versions/node/v14.15.3/lib/node_modules/@google-cloud/logging -> /home/kbuilder/.cache/synthtool/nodejs-logging
npm WARN deprecated debug@4.2.0: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
npm WARN deprecated fsevents@2.1.3: Please update to v 2.2.x
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@~2.1.2 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 269 packages from 174 contributors and audited 396 packages in 7.75s

82 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

2020-12-28 04:28:22,542 synthtool [DEBUG] > Running fix...
DEBUG:synthtool:Running fix...

> @google-cloud/logging@9.0.0 fix /home/kbuilder/.cache/synthtool/nodejs-logging
> gts fix

version: 14
2020-12-28 04:29:06,848 synthtool [DEBUG] > Compiling protos...
DEBUG:synthtool:Compiling protos...
2020-12-28 04:29:13,086 synthtool [DEBUG] > Post-processing completed
DEBUG:synthtool:Post-processing completed
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>
    main()
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main
    spec.loader.exec_module(synth_module)  # type: ignore
  File "/tmpfs/src/github/synthtool/synthtool/metadata.py", line 250, in __exit__
    self.observer.stop()
AttributeError: 'MetadataTrackerAndWriter' object has no attribute 'observer'
2020-12-28 04:29:15,141 autosynth [ERROR] > Synthesis failed
2020-12-28 04:29:15,142 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at 47b050e docs: add instructions for authenticating for system tests (#974)
2020-12-28 04:29:15,174 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
2020-12-28 04:29:15,201 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Removing build/
Removing node_modules/
Removing package-lock.json
Removing samples/node_modules/
Removing samples/package-lock.json
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 354, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 189, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 334, in _inner_main
    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 65, in synthesize_loop
    has_changes = toolbox.synthesize_version_in_new_branch(synthesizer, youngest)
  File "/tmpfs/src/github/synthtool/autosynth/synth_toolbox.py", line 259, in synthesize_version_in_new_branch
    synthesizer.synthesize(synth_log_path, self.environ)
  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 120, in synthesize
    synth_proc.check_returncode()  # Raise an exception.
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode
    self.stderr)
subprocess.CalledProcessError: Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.

Google internal developers can see the full log here.

daniel-sanche commented 3 years ago

I did a bit more research into this. It looks like the issue is caused because SYNTHTOOL_TRACK_OBSOLETE_FILES defaults to False, then it is enabled as part of the synth.py file. This means when MetadataTrackerAndWriter is exited, should_track_obsolete_files has a different value than when it was entered.

A solution seems to be to enable the flag before MetadataTrackerAndWriter is entered, but that seems to be before the synth.py file is called. For now, I'm going to turn set_track_obsolete_files off for the logging library (https://github.com/googleapis/nodejs-logging/pull/979)

0xSage commented 3 years ago

solved