Currently, we have reduced the size of original input images down to (256, 256) because of the limited capability of the default VM in Vertex Pipeline.
In order to overcome this issue, we should use Dataflow. Now, it is tested/verfied that ImportExampleGen component can delegate its job to Dataflow. So, we need to experiment if Dataflow can handle the bigger size of input images.
So, please write a script to make TFRecords with the raw size of images in a separate branch, host the generated TFRecords to GCS bucket. After that, I will create another branch to test the data with Dataflow.
Currently, we have reduced the size of original input images down to (256, 256) because of the limited capability of the default VM in Vertex Pipeline.
In order to overcome this issue, we should use Dataflow. Now, it is tested/verfied that
ImportExampleGencomponent can delegate its job to Dataflow. So, we need to experiment if Dataflow can handle the bigger size of input images.So, please write a script to make TFRecords with the raw size of images in a separate branch, host the generated TFRecords to GCS bucket. After that, I will create another branch to test the data with Dataflow.