lucasbrambrink
commented
3 months ago Hi!
I'm not sure I've seen this error before. It seems that the file is accessible given that it appears to be failing on parsing of the DeepVariantDatasetConfig proto.
Is is possible for you to share 1 of the files in the mounted bucket? That would allow us to try this on our end. Thanks!
gambalab
I'm attempting to train DeepVariant on a Google VM. However, when I run the training command, I encounter the error below.
All the paths are correct (of course I have edited the pbtxt file to match the path were shuffled example are stored), and everything works when I run the command on my local machine with my GPU or on our cluster with CPUs.
The only difference is that the shuffled examples are stored on a mounted bucket with gcsfuse when I run the command on the Google VM. The bucket is mounted using the following command:
sudo gcsfuse -o allow_other. I have also tried to use singularity, but I got the same error.I've tried to find a solution through various online resources, but nothing has helped so far.
Traceback (most recent call last): File "/tmp/Bazel.runfiles_ebq8nvgq/runfiles/com_google_deepvariant/deepvariant/train.py", line 532, in <module> app.run(main) File "/tmp/Bazel.runfiles_ebq8nvgq/runfiles/absl_py/absl/app.py", line 312, in run _run_main(main, args) File "/tmp/Bazel.runfiles_ebq8nvgq/runfiles/absl_py/absl/app.py", line 258, in _run_main sys.exit(main(argv)) File "/tmp/Bazel.runfiles_ebq8nvgq/runfiles/com_google_deepvariant/deepvariant/train.py", line 518, in main train(FLAGS.config) File "/tmp/Bazel.runfiles_ebq8nvgq/runfiles/com_google_deepvariant/deepvariant/train.py", line 121, in train tune_dataset_config = data_providers.read_dataset_config( File "/tmp/Bazel.runfiles_ebq8nvgq/runfiles/com_google_deepvariant/deepvariant/data_providers.py", line 634, in read_dataset_config dataset_config = text_format.Parse( File "/tmp/Bazel.runfiles_ebq8nvgq/runfiles/com_google_protobuf/python/google/protobuf/text_format.py", line 648, in Parse return ParseLines(text.split(b'\n' if isinstance(text, bytes) else u'\n'), File "/tmp/Bazel.runfiles_ebq8nvgq/runfiles/com_google_protobuf/python/google/protobuf/text_format.py", line 722, in ParseLines return parser.ParseLines(lines, message) File "/tmp/Bazel.runfiles_ebq8nvgq/runfiles/com_google_protobuf/python/google/protobuf/text_format.py", line 776, in ParseLines self._ParseOrMerge(lines, message) File "/tmp/Bazel.runfiles_ebq8nvgq/runfiles/com_google_protobuf/python/google/protobuf/text_format.py", line 804, in _ParseOrMerge self._MergeField(tokenizer, message) File "/tmp/Bazel.runfiles_ebq8nvgq/runfiles/com_google_protobuf/python/google/protobuf/text_format.py", line 894, in _MergeField raise tokenizer.ParseErrorPreviousToken( google.protobuf.text_format.ParseError: 13:1 : Message type "learning.genomics.deepvariant.DeepVariantDatasetConfig" has no field named "s2".