search

amansrivastava17 / embedding-as-service

One-Stop Solution to encode sentence to fixed length vectors from various embedding techniques
MIT License
204 stars 29 forks source link

Performance issues in server/embedding_as_service/text/xlnet/models/data_utils.py(P2) #65

Open DLPerf opened 3 years ago

DLPerf commented 3 years ago

Hello,I found a performance issue in the definition of parse_files_to_dataset , server/embedding_as_service/text/xlnet/models/data_utils.py, dataset = dataset.cache().map(parser) was called without num_parallel_calls. I think it will increase the efficiency of your program if you add this.

Here is the documemtation of tensorflow to support this thing.

Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.