google-deepmind / reverb

Reverb is an efficient and easy-to-use data storage and transport system designed for machine learning research
Apache License 2.0
704 stars 92 forks source link

[DO NOT MERGE] Attempt and info for merging master into macOS build #39

Open thisiscam opened 3 years ago

thisiscam commented 3 years ago

Hi,

I would like to share my progress in trying to merge the latest master into the macOS build #24 developed by @lifeiteng.

The merge went smoothly; however, there were a few places that I needed to fix/hack, in order to build successfully. I will comment in my commit below for the details about the hacks. Hopefully, these will be somewhat useful info for folks to merge #24 into master.

Currently this fails 10 tests, all of them of the form:

INFO: From Testing //reverb/cc:trajectory_writer_test:
==================== Test output for //reverb/cc:trajectory_writer_test:
dyld: lazy symbol binding failed: Symbol not found: __ZN10tensorflow15TensorShapeBaseINS_18PartialTensorShapeEEC2EN4absl12lts_202103244SpanIKxEE
  Referenced from: /private/var/tmp/_bazel_cyang/b1d79ebef38c9591112b3498fd6e8dab/sandbox/darwin-sandbox/1353/execroot/reverb/bazel-out/darwin-opt/bin/reverb/cc/trajectory_writer_test.runfiles/reverb/reverb/cc/trajectory_writer_test
  Expected in: flat namespace

dyld: Symbol not found: __ZN10tensorflow15TensorShapeBaseINS_18PartialTensorShapeEEC2EN4absl12lts_202103244SpanIKxEE
  Referenced from: /private/var/tmp/_bazel_cyang/b1d79ebef38c9591112b3498fd6e8dab/sandbox/darwin-sandbox/1353/execroot/reverb/bazel-out/darwin-opt/bin/reverb/cc/trajectory_writer_test.runfiles/reverb/reverb/cc/trajectory_writer_test
  Expected in: flat namespace

-- which I assume are caused by incompatible TensorFlow library versions.

google-cla[bot] commented 3 years ago

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

thisiscam commented 3 years ago

@googlebot I consent

google-cla[bot] commented 3 years ago

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

wookayin commented 2 years ago

The lib -> libxxx changes trick was clever to address the following conflicts (ActionConflictException):

(excerpt of log)

ERROR: com.google.devtools.build.lib.skyframe.ArtifactConflictFinder$ConflictException:
 com.google.devtools.build.lib.actions.MutableActionGraph$ActionConflictException:
 for reverb/cc/checkpointing/libcheckpoint_cc_proto.so,
  previous action: action 'Linking reverb/cc/checkpointing/libcheckpoint_cc_proto.so',
  attempted action: action 'Reporting failed target //reverb/cc/checkpointing:checkpoint_cc_proto located at .../reverb/reverb/cc/checkpointing/BUILD:13:24'

The error you are experiencing (Symbol not found) is probably due to the version mismatch between reverb and tensorflow. You should use tf-nightly or any recent versions.