search

google-research / albert

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Apache License 2.0
3.24k stars 569 forks source link

[ALBERT] hub module on tpu #117

Closed seongwook-ham closed 4 years ago

seongwook-ham commented 4 years ago

when i finetune hub module(version 3) with classifer.py on in mnli-m(glue), it throw follwing error tf hub Failed to get matching files on /tmp/tfhub_modules/xxxxxxxxxxxxx /variables/variables: Unimplemented: File system scheme '[local]' not implemented i think it is due to fact that tmp folder is made on local machine, but for tpu checkpoint path should be on cloud storage bucket. uploading hub module on google storage and run classifier with hub module on cloud storage solve problem but i think it it is not convinient way.

theword commented 4 years ago

when i finetune hub module(version 3) with classifer.py on in mnli-m(glue), it throw follwing error tf hub Failed to get matching files on /tmp/tfhub_modules/xxxxxxxxxxxxx /variables/variables: Unimplemented: File system scheme '[local]' not implemented i think it is due to fact that tmp folder is made on local machine, but for tpu checkpoint path should be on cloud storage bucket. uploading hub module on google storage and run classifier with hub module on cloud storage solve problem but i think it it is not convinient way.

I am running into this problem. What did you do to fix it? You created a tmp folder on Google storage and put the 3 folder in it? (the untar hub module 3.tar.gz)

0x0539 commented 4 years ago

I haven't tested this, but you may be able to simply set TFHUB_CACHE_DIR to a gs bucket where you have read+write access. See https://www.tensorflow.org/hub/api_docs/python/hub/load.

If I understand correctly, tensorflow_hub will cache hub files there. No need to upload files there manually.

0x0539 commented 4 years ago

(please reopen if you still have issues)

theword commented 4 years ago

I haven't tested this, but you may be able to simply set TFHUB_CACHE_DIR to a gs bucket where you have read+write access. See https://www.tensorflow.org/hub/api_docs/python/hub/load.

If I understand correctly, tensorflow_hub will cache hub files there. No need to upload files there manually.

Just setting --albert_hub_module_handle='gs://mygooglebucket/3' worked.

ctmckee commented 4 years ago

I am running into the same problem. Could you please describe solution in detail. I have set the "--albert_hub_module_handle='gs://someGoogleBucket/anEmptyFolder'". Do I need to put the 3.tar.gz folder (unzipped) in the "anEmptyFolder" ?

theword commented 4 years ago

I am running into the same problem. Could you please describe solution in detail. I have set the "--albert_hub_module_handle='gs://someGoogleBucket/anEmptyFolder'". Do I need to put the 3.tar.gz folder (unzipped) in the "anEmptyFolder" ?

Unzip the 3.tar.gz file into your Google Cloud Storage. Then have the --albert_hub_module_handle point to the directory that you unzipped.