Hello! I've found a performance issue in /MRC: batch() should be called before map(), which could make your program more efficient. Here is the tensorflow document to support it.
Detailed description is listed below:
/BiDAF/util.py: dataset.batch(config.batch_size)(here) should be called before .map(parser, num_parallel_calls=num_threads)(here).
/MRC/BiDAF/util.py: .batch(config.batch_size)(here) should be called before .map(parser, num_parallel_calls=num_threads)(here).
/MRC/Hybrid/util.py: dataset.batch(config.batch_size)(here) should be called before .map(parser, num_parallel_calls=num_threads)(here).
/MRC/Hybrid/util.py: .batch(config.batch_size)(here) should be called before .map(parser, num_parallel_calls=num_threads)(here).
/MRC/QANet/util.py: dataset.batch(config.batch_size)(here) should be called before .map(parser, num_parallel_calls=num_threads)(here).
/MRC/QANet/util.py: .batch(config.batch_size)(here) should be called before .map(parser, num_parallel_calls=num_threads)(here).
/MRC/RNet/util.py: dataset.batch(config.batch_size)(here) should be called before .map(parser, num_parallel_calls=num_threads)(here).
/MRC/RNet/util.py: .batch(config.batch_size)(here) should be called before .map(parser, num_parallel_calls=num_threads)(here).
Besides, you need to check the function called in map()(e.g., parser called in .map(parser, num_parallel_calls=num_threads)) whether to be affected or not to make the changed code work properly. For example, if parser needs data with shape (x, y, z) as its input before fix, it would require data with shape (batch_size, x, y, z).
Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.
Hello! I've found a performance issue in /MRC:
batch()
should be called beforemap()
, which could make your program more efficient. Here is the tensorflow document to support it.Detailed description is listed below:
dataset.batch(config.batch_size)
(here) should be called before.map(parser, num_parallel_calls=num_threads)
(here)..batch(config.batch_size)
(here) should be called before.map(parser, num_parallel_calls=num_threads)
(here).dataset.batch(config.batch_size)
(here) should be called before.map(parser, num_parallel_calls=num_threads)
(here)..batch(config.batch_size)
(here) should be called before.map(parser, num_parallel_calls=num_threads)
(here).dataset.batch(config.batch_size)
(here) should be called before.map(parser, num_parallel_calls=num_threads)
(here)..batch(config.batch_size)
(here) should be called before.map(parser, num_parallel_calls=num_threads)
(here).dataset.batch(config.batch_size)
(here) should be called before.map(parser, num_parallel_calls=num_threads)
(here)..batch(config.batch_size)
(here) should be called before.map(parser, num_parallel_calls=num_threads)
(here).Besides, you need to check the function called in
map()
(e.g.,parser
called in.map(parser, num_parallel_calls=num_threads)
) whether to be affected or not to make the changed code work properly. For example, ifparser
needs data with shape (x, y, z) as its input before fix, it would require data with shape (batch_size, x, y, z).Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.