search

hassan-mahmood / TIES_DataGeneration

Dataset Generation Code for: S.R. Qasim, H. Mahmood, and F. Shafait, Rethinking Table Parsing using Graph Neural Networks (2019)
MIT License
118 stars 39 forks source link

Why the speed is so slow when I generate the dataset? #9

Open HuiyanWen opened 4 years ago

HuiyanWen commented 4 years ago

Hi, thanks for your code that help me a lot, but I'm still suffer from a problem that the speed is very slow and the RAM's consuming is terrible (It used up 64GB RAM) when I generate the dataset. Specifically, I can only generate 1000 samples per hour approximately. So I want to ask that whether you have the same phenomenon? If it's convenient, may you introduce the concrete environment? And there's another thing makes me confused. How do you compress the TFRECORD which is over 600MB to 6MB?

My environment

I have configured two different environment, however, both of them are suffered from the problem I've mentioned above. 1.python version:3.6.3, tensorflow version:1.7, selenium version:3.141.0 2.python version 3.7, tensorflow version:2.0.0, selenium version:3.141.0

ashish-kubade commented 4 years ago

@HuiyanWen were you able to resolve it? I am also facing slowness. In fact, now able to generate the bounding boxes as well.

linhuaiyi commented 3 years ago

It may something crashed and codes run in circle. Please reference #14