Hi, thanks for your code that help me a lot, but I'm still suffer from a problem that the speed is very slow and the RAM's consuming is terrible (It used up 64GB RAM) when I generate the dataset. Specifically, I can only generate 1000 samples per hour approximately. So I want to ask that whether you have the same phenomenon? If it's convenient, may you introduce the concrete environment? And there's another thing makes me confused. How do you compress the TFRECORD which is over 600MB to 6MB?
My environment
I have configured two different environment, however, both of them are suffered from the problem I've mentioned above.
1.python version:3.6.3, tensorflow version:1.7, selenium version:3.141.02.python version 3.7, tensorflow version:2.0.0, selenium version:3.141.0
Hi, thanks for your code that help me a lot, but I'm still suffer from a problem that the speed is very slow and the RAM's consuming is terrible (It used up 64GB RAM) when I generate the dataset. Specifically, I can only generate 1000 samples per hour approximately. So I want to ask that whether you have the same phenomenon? If it's convenient, may you introduce the concrete environment? And there's another thing makes me confused. How do you compress the TFRECORD which is over 600MB to 6MB?
My environment
I have configured two different environment, however, both of them are suffered from the problem I've mentioned above. 1.python version:3.6.3, tensorflow version:1.7, selenium version:3.141.0 2.python version 3.7, tensorflow version:2.0.0, selenium version:3.141.0