Non accesable example TFrecords on Keras IO Example tfrecord.ipynb

[lriosFRBAUTN]

Hi ! Thanks for the great work and the examples.

I was wondering is there is any way to set up a free public bucket / free space to store the TFrecords on this notebook:

https://github.com/keras-team/keras-io/blob/master/examples/keras_recipes/ipynb/tfrecord.ipynb

The path seems not to be accesable

GCS_PATH = "gs://kds-b38ce1b823c3ae623f5691483dbaa0f0363f04b0d6a90b63cf69946e"
FILENAMES = tf.io.gfile.glob(GCS_PATH + "/tfrecords/train*.tfrec")

Best, Leandro

[fchollet]

@mjang2000 Amy, could you please take a look at this, if you have the bandwidth? It seems we should take the same approach as for the pneumonia dataset files.

[mjang2000]

Ah yes, it seems to be the same issue as the pneumonia files. I will reach out to Mark and ask if he can host the files for us.

[lriosFRBAUTN]

Hi ! Hope you are doing fine :) Any progress on this ? Thanks in advance

[mjang2000]

Hi! We are working on a long-term solution to this. However, in the mean time, please use "gs://kds-70223d17d8104e322f13618b2d51198e7d2736b6c575eafd6a84e8a0" for the GCS_PATH instead.

[lriosFRBAUTN]

Thanks for your prompt response! I will try that, right away! Thanks a lot!

[lriosFRBAUTN]

Hi, I am using Google Colab, and the new link works like charm! However, I noticed that notebook only works with TPUs :(

I tried to run the same notebook with Google Colab using GPU and it crashed on the "Visualize input Images" paragraph (so no training at all). When displaying images, it crashed using Google Colab GPU. I also tried with smaller BATCH_SIZE = 32, 16, 4 and even with BATCHSIZE = 1 and also crashed "Your session crashed after using all available RAM."_

I am trying to train my own custom dataset for an Object Detection problem using TF2.x and of course Keras. I have had similar RAM problems with my own dataset (my images are 1280x720, around 1700 JPG files, with class and bonding boxes ). At first, I was not sure if it was my code or my TFrecords. But now that you have provided a working link and the example code https://keras.io/examples/keras_recipes/tfrecord/ is having RAM problems (using Google Colab GPU) I really don't have any clue what it could be so RAM consuming.

Have you noticed this problem? Do you have any idea where the problem might be?

Thanks in advance

[Whisht]

Hi, the recent path @mjang2000 updateing cannot use now. Could you provide a new solution? Thanks.

[mjang2000]

@Whisht Yes, try 'gs://kds-8cea9e04f765f4fee325837a02c37b2e41f9dffa509ad10772052f74' instead.

[gopinathangokul]

Hi @mjang2000 , the above link is not working, could you please provide a working link?

[mjang2000]

@gopinathangokul Try 'gs://kds-4f363bf42156e3caed6cf74ba89089135a522d3dbc34f68c6323844a'

[chdaesung]

@mjang2000 gs://kds-4f363bf42156e3caed6cf74ba89089135a522d3dbc34f68c6323844a was not accessible on my Colab account. I got "NotFoundError: Error executing an HTTP request: HTTP response code 404". Can you help?

[mjang2000]

@chdaesung Try 'gs://kds-c45437956060d652eb2a7563eee2bfd865072fabcbd0905c612697e3'

[mfiro]

@mjang2000 Could you please send a new link? The latest one doesn't work.

[mjang2000]

Try 'gs://kds-91b71e62f3059c9e96bcb2014db88265ed14bcc84fab6762751e3382'

On Sat, May 1, 2021 at 2:15 PM mfiro @.***> wrote:

@mjang2000 https://github.com/mjang2000 Could you please send a new link? The latest one doesn't work.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/keras-team/keras-io/issues/223#issuecomment-830695539, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKNPOG4I2A6NQRKAZEWL3SDTLRVNZANCNFSM4QFCUTSA .

-- Amy MiHyun Jang Biomedical Engineering & Computer Science Egleston Scholar Columbia University | Class of 2022 @.***

[jz247]

Try 'gs://kds-91b71e62f3059c9e96bcb2014db88265ed14bcc84fab6762751e3382' … On Sat, May 1, 2021 at 2:15 PM mfiro @.> wrote: @mjang2000 https://github.com/mjang2000 Could you please send a new link? The latest one doesn't work. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#223 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKNPOG4I2A6NQRKAZEWL3SDTLRVNZANCNFSM4QFCUTSA . -- Amy MiHyun Jang Biomedical Engineering & Computer Science Egleston Scholar Columbia University | Class of 2022 @.

This link doesn't work neither. Could you please a link that works, @mjang2000 ? Thanks!

[mjang2000]

Try 'gs://kds-96526c1ab32bc821b3efc2187816c671aac62349b2c3382038fb695d'

[mikewang6252021]

mjang, can not accesses again; any new link?

when reading gs://kds-96526c1ab32bc821b3efc2187816c671aac62349b2c3382038fb695d/tfrecords

[mjang2000]

Try 'gs://kds-60b7b756ac5ccab177dff545c9b0b7bec83756f087df4aa6cf383987'

On Fri, Jun 25, 2021 at 9:36 AM mikewang6252021 @.***> wrote:

mjang, can not accesses again; any new link?

when reading gs://kds-96526c1ab32bc821b3efc2187816c671aac62349b2c3382038fb695d/tfrecords

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/keras-team/keras-io/issues/223#issuecomment-868689780, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKNPOG2GZMIDZ2X3BZUUJLLTUSWAXANCNFSM4QFCUTSA .

-- Amy MiHyun Jang Biomedical Engineering & Computer Science Egleston Scholar Columbia University | Class of 2022 @.***

[lkarthee]

@mjang2000 what is the dataset being used - is it this one https://www.kaggle.com/datasets/fanconic/skin-cancer-malignant-vs-benign/data ?

Can we use any other dataset from TensorFlow datasets instead of melanoma one as the example is more about TFRecords rather than a specific dataset?