search

tensorflow / datasets

TFDS is a collection of datasets ready to use with TensorFlow, Jax, ...
https://www.tensorflow.org/datasets
Apache License 2.0
4.32k stars 1.55k forks source link

Placeholder for all tfds bugs on windows #1911

Open vijayphoenix opened 4 years ago

vijayphoenix commented 4 years ago

Short description Currently, tfds has no Kokoro windows tests. So this issue is a placeholder for all bug report and fixes related to tfds on windows

Environment information

vijayphoenix commented 4 years ago

Related https://github.com/tensorflow/datasets/issues/817

Problem

Many os methods like os.chown are Unix specific. So, they should not be mocked while unit testing on windows

vijayphoenix commented 4 years ago

Problem

tf.io.gfile.rename sometimes fails to rename files on windows.

Possible fix Try until we successfully rename the file

Also see: https://github.com/tensorflow/tensorflow/issues/21135

vijayphoenix commented 4 years ago

Problem tfds/image_classification has the following bugs on windows

See pytest results

``` ============================= test session starts ============================= platform win32 -- Python 3.7.6, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 rootdir: C:\Users\VIJAY\Desktop\GitHub_Repos\datasets plugins: forked-1.1.3, xdist-1.31.0 collected 355 items tensorflow_datasets\image_classification\beans_test.py .F..s [ 1%] tensorflow_datasets\image_classification\bigearthnet_test.py .F..s [ 2%] tensorflow_datasets\image_classification\binary_alpha_digits_test.py ... [ 3%] .s [ 4%] tensorflow_datasets\image_classification\caltech_birds_test.py .F..s.F.. [ 6%] s [ 7%] tensorflow_datasets\image_classification\caltech_test.py ....s [ 8%] tensorflow_datasets\image_classification\cars196_test.py ....s [ 9%] tensorflow_datasets\image_classification\cassava_test.py ....s [ 11%] tensorflow_datasets\image_classification\cats_vs_dogs_test.py ....s [ 12%] tensorflow_datasets\image_classification\cbis_ddsm_test.py ....s....s... [ 16%] .s [ 16%] tensorflow_datasets\image_classification\chexpert_test.py ....s [ 18%] tensorflow_datasets\image_classification\cifar10_1_test.py ....s [ 19%] tensorflow_datasets\image_classification\cifar10_corrupted_test.py ....s [ 21%] [ 21%] tensorflow_datasets\image_classification\cifar_test.py ....s....s [ 23%] tensorflow_datasets\image_classification\citrus_test.py .F..s [ 25%] tensorflow_datasets\image_classification\cmaterdb_test.py ....s [ 26%] tensorflow_datasets\image_classification\colorectal_histology_test.py .. [ 27%] ..s....s [ 29%] tensorflow_datasets\image_classification\cycle_gan_test.py ....s [ 30%] tensorflow_datasets\image_classification\deep_weeds_test.py ....s [ 32%] tensorflow_datasets\image_classification\diabetic_retinopathy_detection_test.py . [ 32%] ...s [ 33%] tensorflow_datasets\image_classification\dmlab_test.py ....s [ 35%] tensorflow_datasets\image_classification\dtd_test.py ....s [ 36%] tensorflow_datasets\image_classification\eurosat_test.py ....s [ 38%] tensorflow_datasets\image_classification\flowers_test.py ....s [ 39%] tensorflow_datasets\image_classification\food101_test.py ....s [ 40%] tensorflow_datasets\image_classification\geirhos_conflict_stimuli_test.py . [ 41%] ...s [ 42%] tensorflow_datasets\image_classification\horses_or_humans_test.py ....s [ 43%] tensorflow_datasets\image_classification\image_folder_test.py ....s [ 45%] tensorflow_datasets\image_classification\imagenet2012_corrupted_test.py . [ 45%] ...s [ 46%] tensorflow_datasets\image_classification\imagenet2012_subset_test.py .F. [ 47%] .s [ 47%] tensorflow_datasets\image_classification\imagenet_resized_test.py ....s [ 49%] tensorflow_datasets\image_classification\imagenet_test.py .F..s [ 50%] tensorflow_datasets\image_classification\imagenette_test.py ....s [ 52%] tensorflow_datasets\image_classification\imagewang_test.py .F..s.F..s.F. [ 55%] .s [ 56%] tensorflow_datasets\image_classification\inaturalist_test.py ....s [ 57%] tensorflow_datasets\image_classification\lfw_test.py ....s [ 59%] tensorflow_datasets\image_classification\malaria_test.py ....s [ 60%] tensorflow_datasets\image_classification\mnist_corrupted_test.py ....s [ 61%] tensorflow_datasets\image_classification\mnist_test.py ....s....s....s.. [ 66%] ..s [ 67%] tensorflow_datasets\image_classification\oxford_flowers102_test.py ....s [ 69%] [ 69%] tensorflow_datasets\image_classification\oxford_iiit_pet_test.py ....s [ 70%] tensorflow_datasets\image_classification\patch_camelyon_test.py ....s [ 71%] tensorflow_datasets\image_classification\pet_finder_test.py ....s [ 73%] tensorflow_datasets\image_classification\places365_small_test.py .F..s [ 74%] tensorflow_datasets\image_classification\plant_leaves_test.py ....s [ 76%] tensorflow_datasets\image_classification\plant_village_test.py .F..s [ 77%] tensorflow_datasets\image_classification\plantae_k_test.py ....s [ 78%] tensorflow_datasets\image_classification\quickdraw_test.py ....s [ 80%] tensorflow_datasets\image_classification\resisc45_test.py ....s [ 81%] tensorflow_datasets\image_classification\rock_paper_scissors_test.py ... [ 82%] .s [ 83%] tensorflow_datasets\image_classification\smallnorb_test.py ....s [ 84%] tensorflow_datasets\image_classification\so2sat_test.py ....s [ 85%] tensorflow_datasets\image_classification\stanford_dogs_test.py ....s [ 87%] tensorflow_datasets\image_classification\stanford_online_products_test.py . [ 87%] ...s [ 88%] tensorflow_datasets\image_classification\stl10_test.py ....s [ 90%] tensorflow_datasets\image_classification\sun_test.py .F..s.F..s [ 92%] tensorflow_datasets\image_classification\svhn_test.py ....s [ 94%] tensorflow_datasets\image_classification\uc_merced_test.py ....s [ 95%] tensorflow_datasets\image_classification\vgg_face2_test.py .F..s [ 97%] tensorflow_datasets\image_classification\visual_domain_decathlon_test.py . [ 97%] F..s.F..s [100%] ================================== FAILURES =================================== C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\image_classification\beans.py:98: IndexError: list index out of range C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\image_classification\bigearthnet.py:240: ValueError: not enough values to unpack (expected 3, got 1) [while running 'train/ReadArchive'] C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\tfrecords_writer.py:108: AssertionError: No examples were yielded. C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\tfrecords_writer.py:108: AssertionError: No examples were yielded. C:\ProgramData\Miniconda3\lib\site-packages\absl\testing\absltest.py:1704: AssertionError: [{'image': array([[[79, 79, 79], C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\tfrecords_writer.py:108: AssertionError: No examples were yielded. C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\features\class_label_feature.py:115: KeyError: 'n02776631\r' C:\ProgramData\Miniconda3\lib\site-packages\absl\testing\absltest.py:1704: AssertionError: {'https://s3.amazonaws.com/fast-ai-imageclas\\imagewang.tgz'} has length of 1. : Some urls checksums are missing at: C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\url_checksums\imagewang.txt Did you forget to record checksums with `--register_checksums` ? See instructions at: https://www.tensorflow.org/datasets/add_dataset#2_run_download_and_prepare_locally C:\ProgramData\Miniconda3\lib\site-packages\absl\testing\absltest.py:1704: AssertionError: {'https://s3.amazonaws.com/fast-ai-imageclas\\imagewang-320.tgz'} has length of 1. : Some urls checksums are missing at: C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\url_checksums\imagewang.txt Did you forget to record checksums with `--register_checksums` ? See instructions at: https://www.tensorflow.org/datasets/add_dataset#2_run_download_and_prepare_locally C:\ProgramData\Miniconda3\lib\site-packages\absl\testing\absltest.py:1704: AssertionError: {'https://s3.amazonaws.com/fast-ai-imageclas\\imagewang-160.tgz'} has length of 1. : Some urls checksums are missing at: C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\url_checksums\imagewang.txt Did you forget to record checksums with `--register_checksums` ? See instructions at: https://www.tensorflow.org/datasets/add_dataset#2_run_download_and_prepare_locally C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\image_classification\places365_small.py:143: KeyError: '\\a\\airfield\\00001.jpg' C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\tfrecords_writer.py:108: AssertionError: No examples were yielded. C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\tfrecords_writer.py:108: AssertionError: No examples were yielded. C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\tfrecords_writer.py:108: AssertionError: No examples were yielded. C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\features\class_label_feature.py:115: KeyError: '\\n00000' C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\tfrecords_writer.py:108: AssertionError: No examples were yielded. C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\tfrecords_writer.py:108: AssertionError: No examples were yielded. =========================== short test summary info =========================== FAILED tensorflow_datasets/image_classification/beans_test.py::BeansTest::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/image_classification/bigearthnet_test.py::BigearthnetTest::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/image_classification/caltech_birds_test.py::CaltechBirdsTest::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/image_classification/caltech_birds_test.py::CaltechBirds2011Test::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/image_classification/citrus_test.py::CitrusLeavesTest::test_download_and_prepare_as_dataset ## tf glob error FAILED tensorflow_datasets/image_classification/imagenet2012_subset_test.py::Imagenet2012SubsetTest::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/image_classification/imagenet_test.py::Imagenet2012Test::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/image_classification/imagewang_test.py::ImagewangFullSizeTest::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/image_classification/imagewang_test.py::Imagewang320Test::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/image_classification/imagewang_test.py::Imagewang160Test::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/image_classification/places365_small_test.py::Places365SmallTest::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/image_classification/plant_village_test.py::PlantVillageTest::test_download_and_prepare_as_dataset ## tf glob error FAILED tensorflow_datasets/image_classification/sun_test.py::Sun397StandardPartitionTest::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/image_classification/sun_test.py::Sun397TfdsTest::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/image_classification/vgg_face2_test.py::VggFace2Test::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/image_classification/visual_domain_decathlon_test.py::VisualDomainDecathlonGenericTest::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/image_classification/visual_domain_decathlon_test.py::VisualDomainDecathlonImagenetTest::test_download_and_prepare_as_dataset ==== 17 failed, 267 passed, 71 skipped, 557 warnings in 424.49s (0:07:04) ===== ```

vijayphoenix commented 4 years ago

Problem tfds/image has the following bugs on windows

See pytest results

``` ============================= test session starts ============================= platform win32 -- Python 3.7.6, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 rootdir: C:\Users\VIJAY\Desktop\GitHub_Repos\datasets plugins: forked-1.1.3, xdist-1.31.0 collected 120 items tensorflow_datasets\image\abstract_reasoning_test.py ....s [ 4%] tensorflow_datasets\image\aflw2k3d_test.py ....s [ 8%] tensorflow_datasets\image\arc_test.py ....s [ 12%] tensorflow_datasets\image\binarized_mnist_test.py ....s [ 16%] tensorflow_datasets\image\celeba_test.py .F..s [ 20%] tensorflow_datasets\image\celebahq_test.py ....s [ 25%] tensorflow_datasets\image\cityscapes_test.py ....s....s....s....s [ 41%] tensorflow_datasets\image\clevr_test.py ....s [ 45%] tensorflow_datasets\image\coil100_test.py ....s [ 50%] tensorflow_datasets\image\div2k_test.py ....s....s [ 58%] tensorflow_datasets\image\downsampled_imagenet_test.py ....s [ 62%] tensorflow_datasets\image\dsprites_test.py ....s [ 66%] tensorflow_datasets\image\duke_ultrasound_test.py .F..s [ 70%] tensorflow_datasets\image\flic_test.py ....s....s [ 79%] tensorflow_datasets\image\lost_and_found_test.py ....s [ 83%] tensorflow_datasets\image\lsun_test.py .F..s [ 87%] tensorflow_datasets\image\scene_parse_150_test.py ....s [ 91%] tensorflow_datasets\image\shapes3d_test.py ....s [ 95%] tensorflow_datasets\image\the300w_lp_test.py .F..s [100%] ================================== FAILURES =================================== C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\image\celeba.py:208: KeyError: '000001.jpg' C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\features\feature.py:541: OverflowError: Python int too large to convert to C long C:\ProgramData\Miniconda3\lib\site-packages\tensorflow_io\core\python\ops\__init__.py:67: NotImplementedError: unable to open file: libtensorflow_io.so, from paths: ['C:\\ProgramData\\Miniconda3\\lib\\site-packages\\tensorflow_io\\core\\python\\ops\\libtensorflow_io.so'] C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\tfrecords_writer.py:108: AssertionError: No examples were yielded. =========================== short test summary info =========================== FAILED tensorflow_datasets/image/celeba_test.py::CelebATest::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/image/duke_ultrasound_test.py::DukeUltrasoundTest::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/image/lsun_test.py::LsunTest::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/image/the300w_lp_test.py::The300wLpTest::test_download_and_prepare_as_dataset ## tf glob error ====== 4 failed, 92 passed, 24 skipped, 9 warnings in 217.14s (0:03:37) ======= ```

vijayphoenix commented 4 years ago

Problem tfds/object_detection has some bugs

See pytest results

``` ============================= test session starts ============================= platform win32 -- Python 3.7.6, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 rootdir: C:\Users\VIJAY\Desktop\GitHub_Repos\datasets plugins: forked-1.1.3, xdist-1.31.0 collected 50 items tensorflow_datasets\object_detection\coco_test.py ....s....s....s [ 30%] tensorflow_datasets\object_detection\kitti_test.py .F..s [ 40%] tensorflow_datasets\object_detection\open_images_challenge2019_test.py . [ 42%] ...s [ 50%] tensorflow_datasets\object_detection\open_images_test.py ....s [ 60%] tensorflow_datasets\object_detection\voc_test.py ....s....s [ 80%] tensorflow_datasets\object_detection\waymo_open_dataset_test.py ....s [ 90%] tensorflow_datasets\object_detection\wider_face_test.py ....s [100%] ================================== FAILURES =================================== C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\object_detection\kitti.py:269: AssertionError =========================== short test summary info =========================== FAILED tensorflow_datasets/object_detection/kitti_test.py::KittiTest::test_download_and_prepare_as_dataset ===== 1 failed, 39 passed, 10 skipped, 2288 warnings in 148.71s (0:02:28) ===== ```

vijayphoenix commented 4 years ago

Problem tfds/structured has some bugs

See pytest results

``` ============================= test session starts ============================= platform win32 -- Python 3.7.6, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 rootdir: C:\Users\VIJAY\Desktop\GitHub_Repos\datasets plugins: forked-1.1.3, xdist-1.31.0 collected 35 items tensorflow_datasets\structured\amazon_us_reviews_test.py ....s [ 14%] tensorflow_datasets\structured\forest_fires_test.py ....s [ 28%] tensorflow_datasets\structured\german_credit_numeric_test.py ....s [ 42%] tensorflow_datasets\structured\higgs_test.py ....s [ 57%] tensorflow_datasets\structured\iris_test.py .F..s [ 71%] tensorflow_datasets\structured\rock_you_test.py ....s [ 85%] tensorflow_datasets\structured\titanic_test.py ....s [100%] ================================== FAILURES =================================== C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\features\class_label_feature.py:115: KeyError: 'Iris-setosa\r' =========================== short test summary info =========================== FAILED tensorflow_datasets/structured/iris_test.py::IrisTest::test_download_and_prepare_as_dataset ============ 1 failed, 27 passed, 7 skipped, 7 warnings in 21.63s ============= ```

vijayphoenix commented 4 years ago

Problem tfds/text has some bugs

See pytest results

``` ============================= test session starts ============================= platform win32 -- Python 3.7.6, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 rootdir: C:\Users\VIJAY\Desktop\GitHub_Repos\datasets plugins: forked-1.1.3, xdist-1.31.0 collected 260 items tensorflow_datasets\text\blimp_test.py .F..s [ 1%] tensorflow_datasets\text\c4_test.py ....s....s [ 5%] tensorflow_datasets\text\c4_utils_test.py ........s. [ 9%] tensorflow_datasets\text\cfq_test.py ....s [ 11%] tensorflow_datasets\text\civil_comments_test.py ....s [ 13%] tensorflow_datasets\text\cos_e_test.py ....s [ 15%] tensorflow_datasets\text\definite_pronoun_resolution_test.py ....s [ 17%] tensorflow_datasets\text\eraser_multi_rc_test.py ....s [ 19%] tensorflow_datasets\text\esnli_test.py ....s [ 21%] tensorflow_datasets\text\gap_test.py ....s [ 23%] tensorflow_datasets\text\glue_test.py ....s....s....s....s....s....s.... [ 36%] s....s....s....s [ 42%] tensorflow_datasets\text\imdb_test.py ....s [ 44%] tensorflow_datasets\text\librispeech_lm_test.py ....s [ 46%] tensorflow_datasets\text\lm1b_test.py ....s [ 48%] tensorflow_datasets\text\math_dataset_test.py ....s [ 50%] tensorflow_datasets\text\movie_rationales_test.py ....s [ 51%] tensorflow_datasets\text\multi_nli_mismatch_test.py ....s [ 53%] tensorflow_datasets\text\multi_nli_test.py ....s [ 55%] tensorflow_datasets\text\natural_questions_test.py ....s [ 57%] tensorflow_datasets\text\qa4mre_test.py ....s [ 59%] tensorflow_datasets\text\scan_test.py ....s [ 61%] tensorflow_datasets\text\scicite_test.py ....s [ 63%] tensorflow_datasets\text\snli_test.py ....s [ 65%] tensorflow_datasets\text\squad_test.py ....s [ 67%] tensorflow_datasets\text\super_glue_test.py ....s....s....s....s....s... [ 78%] .s....s....s....s....s [ 86%] tensorflow_datasets\text\tiny_shakespeare_test.py ....s [ 88%] tensorflow_datasets\text\triviaqa_test.py ....s [ 90%] tensorflow_datasets\text\web_questions_test.py ....s [ 92%] tensorflow_datasets\text\wiki40b_test.py .F..s [ 94%] tensorflow_datasets\text\wikipedia_test.py ....s [ 96%] tensorflow_datasets\text\xnli_test.py ....s [ 98%] tensorflow_datasets\text\yelp_polarity_test.py ....s [100%] ================================== FAILURES =================================== C:\ProgramData\Miniconda3\lib\site-packages\absl\testing\absltest.py:1704: AssertionError: {'https://raw.githubusercontent.com/alexwarstadt/blimp/master/data\\tough_vs_raising_1.jsonl'} has length of 1. : Some urls checksums are missing at: C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\url_checksums\blimp.txt Did you forget to record checksums with `--register_checksums` ? See instructions at: https://www.tensorflow.org/datasets/add_dataset#2_run_download_and_prepare_locally C:\ProgramData\Miniconda3\lib\site-packages\apache_beam\io\filebasedsource.py:191: OSError: No files found based on the file pattern C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\testing\test_data\fake_examples\wiki40b/train/en_examples-* =========================== short test summary info =========================== FAILED tensorflow_datasets/text/blimp_test.py::BlimpTest::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/text/wiki40b_test.py::Wiki40bTest::test_download_and_prepare_as_dataset ===== 2 failed, 207 passed, 51 skipped, 162 warnings in 295.32s (0:04:55) ===== ```

vijayphoenix commented 4 years ago

Problem tfds/translate has some bugs

See pytest results

``` ============================= test session starts ============================= platform win32 -- Python 3.7.6, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 rootdir: C:\Users\VIJAY\Desktop\GitHub_Repos\datasets plugins: forked-1.1.3, xdist-1.31.0 collected 31 items tensorflow_datasets\translate\para_crawl_test.py ....s [ 16%] tensorflow_datasets\translate\ted_hrlr_test.py ....s [ 32%] tensorflow_datasets\translate\ted_multi_test.py ....s [ 48%] tensorflow_datasets\translate\wmt19_test.py .F..s.F..s [ 80%] tensorflow_datasets\translate\wmt_test.py .F...s [100%] ================================== FAILURES =================================== C:\ProgramData\Miniconda3\lib\site-packages\absl\testing\absltest.py:1704: AssertionError: [{'de': b'ich renne', 'en': b'I am running'}, {'de': b'ich renne', 'en': b'I am running'}, {'de': b'ich schwimme', 'en': b'I am swimming'}, {'de': b'ich renne', 'en': b'I am running'}, {'de': b'ich schwimme\r', 'en': b'I am swimming\r'}, {'de': b'ich renne\r', 'en': b'I am running\r'}, {'de': b'ich renne', 'en': b'I am running'}, {'de': b'ich swimme', 'en': b'I am swimming'}, {'de': b'es verschwand\r', 'en': b'\r'}, {'de': b'es verschwand \r', 'en': b'\r'}, {'de': b'ich schwimme', 'en': b'I am swimming'}, {'de': b'ich schwimme', 'en': b'I am swimming'}, {'de': b'ich schwimme\r', 'en': b'I am swimming\r'}, {'de': b'ich renne\r', 'en': b'I am running\r'}] has length of 14, expected 12. C:\ProgramData\Miniconda3\lib\site-packages\absl\testing\absltest.py:1704: AssertionError: [{'cs': b'b\xc4\x9b\xc5\xbe\xc3\xadm', 'en': b'I am running (tsv)'}, {'cs': b'b\xc4\x9b\xc5\xbe\xc3\xadm', 'en': b'I am running (tsv)'}, {'cs': b'Plavu', 'en': b'I am swimming'}, {'cs': b'Plavu', 'en': b'I am swimming (tsv)'}, {'cs': b'b\xc4\x9b\xc5\xbe\xc3\xadm', 'en': b'I am running'}, {'cs': b'dal\xc5\xa1\xc3\xad p\xc5\x99eklad', 'en': b'another translation'}, {'cs': b'b\xc4\x9b\xc5\xbe\xc3\xadm', 'en': b'I am running (tsv)'}, {'cs': b'b\xc4\x9b\xc5\xbe\xc3\xadm\r', 'en': b'I am running (cc)\r'}, {'cs': b'b\xc4\x9b\xc5\xbe\xc3\xadm', 'en': b'I am running (tmx)'}, {'cs': b'Plavu', 'en': b'I am swimming (tmx)'}, {'cs': b'zmizel\r', 'en': b'\r'}, {'cs': b'Plavu', 'en': b'I am swimming (tsv)'}, {'cs': b'Plavu', 'en': b'I am swimming (tsv)'}, {'cs': b'Plav\r', 'en': b'I am swimming (cc)\r'}] has length of 14, expected 13. C:\ProgramData\Miniconda3\lib\site-packages\absl\testing\absltest.py:1704: AssertionError: [{'cs': b'zmizel\r', 'en': b'\r'}, {'cs': b'b\xc4\x9b\xc5\xbe\xc3\xadm\r', 'en': b'I am running\r'}, {'cs': b'Plav\r', 'en': b'I am swimming\r'}, {'cs': b'zmizel\r', 'en': b'\r'}, {'cs': b'Plav\r', 'en': b'I am swimming\r'}, {'cs': b'b\xc4\x9b\xc5\xbe\xc3\xadm\r', 'en': b'I am running\r'}] has length of 6, expected 4. =========================== short test summary info =========================== FAILED tensorflow_datasets/translate/wmt19_test.py::TranslateDeEnWmt19Test::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/translate/wmt19_test.py::TranslateCsEnWmt19Test::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/translate/wmt_test.py::TranslateWmtCustomConfigTest::test_download_and_prepare_as_dataset ============ 3 failed, 22 passed, 6 skipped, 7 warnings in 24.85s ============= ```

vijayphoenix commented 4 years ago

Problem tfds/audio has some bugs

See pytest results

``` ============================= test session starts ============================= platform win32 -- Python 3.7.6, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 rootdir: C:\Users\VIJAY\Desktop\GitHub_Repos\datasets plugins: forked-1.1.3, xdist-1.31.0 collected 85 items tensorflow_datasets\audio\commonvoice_test.py ....s [ 5%] tensorflow_datasets\audio\crema_d_test.py ....s [ 11%] tensorflow_datasets\audio\dementiabank_test.py ....s [ 17%] tensorflow_datasets\audio\groove_test.py ....s....s....s....s [ 41%] tensorflow_datasets\audio\librispeech_test.py ....s [ 47%] tensorflow_datasets\audio\libritts_test.py ....s [ 52%] tensorflow_datasets\audio\ljspeech_test.py ....s [ 58%] tensorflow_datasets\audio\nsynth_test.py ....s....s.F..s [ 76%] tensorflow_datasets\audio\savee_test.py ....s [ 82%] tensorflow_datasets\audio\speech_commands_test.py .F..s [ 88%] tensorflow_datasets\audio\tedlium_test.py ....s [ 94%] tensorflow_datasets\audio\voxceleb_test.py ....s [100%] ================================== FAILURES =================================== :965: ModuleNotFoundError: No module named 'crepe' C:\ProgramData\Miniconda3\lib\site-packages\absl\testing\absltest.py:1704: AssertionError: [{'audio': array([ 3435, -261, -2600, ..., 1831, 6042, 15647], dtype=int64), 'label': 10}, {'audio': array([-2, 2, 4, ..., 1, 1, 5], dtype=int64), 'label': 3}, {'audio': array([ -1672, 2998, -8908, ..., -18975, -19857, -11291], dtype=int64), 'label': 10}, {'audio': array([ -2, -6, -4, ..., -3, -4, -13], dtype=int64), 'label': 9}, {'audio': array([21662, 10382, 13790, ..., -1910, -761, 5744], dtype=int64), 'label': 10}] has length of 5, expected 4. =========================== short test summary info =========================== FAILED tensorflow_datasets/audio/nsynth_test.py::GANsynthWithF0AndLoudnessTest::test_download_and_prepare_as_dataset FAILED tensorflow_datasets/audio/speech_commands_test.py::SpeechCommandsTest::test_download_and_prepare_as_dataset ===== 2 failed, 66 passed, 17 skipped, 534 warnings in 491.38s (0:08:11) ====== ```

vijayphoenix commented 4 years ago

Problem tfds/core has some bugs on windows.

See pytest results

``` ============================= test session starts ============================= platform win32 -- Python 3.7.6, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 rootdir: C:\Users\VIJAY\Desktop\GitHub_Repos\datasets plugins: forked-1.1.3, xdist-1.31.0 collected 426 items tensorflow_datasets\core\_sharded_files_test.py ....s [ 1%] tensorflow_datasets\core\api_utils_test.py ...s [ 2%] tensorflow_datasets\core\dataset_builder_beam_test.py ..s [ 2%] tensorflow_datasets\core\dataset_builder_test.py ............s.......... [ 8%] s.s..s.....s...s [ 11%] tensorflow_datasets\core\dataset_info_test.py .F....s...... [ 15%] tensorflow_datasets\core\dataset_registered_test.py .s [ 15%] tensorflow_datasets\core\dataset_utils_test.py ......s.......s [ 19%] tensorflow_datasets\core\example_serializer_test.py ........s [ 21%] tensorflow_datasets\core\hashing_test.py ..s [ 21%] tensorflow_datasets\core\lazy_imports_lib_test.py .............s [ 25%] tensorflow_datasets\core\naming_test.py ............s..... [ 29%] tensorflow_datasets\core\registered_test.py ...........s. [ 32%] tensorflow_datasets\core\shuffle_test.py .s....s [ 34%] tensorflow_datasets\core\splits_test.py .s....s..s...... [ 37%] tensorflow_datasets\core\tfrecords_reader_test.py .FsFFFF......s.......s [ 42%] .. [ 43%] tensorflow_datasets\core\tfrecords_writer_test.py ..s...s....s....s... [ 48%] tensorflow_datasets\core\units_test.py ...s [ 49%] tensorflow_datasets\core\decode\base_test.py .s. [ 49%] tensorflow_datasets\core\download\download_manager_test.py FFFFF.F... [ 52%] tensorflow_datasets\core\download\downloader_test.py ..........s...s [ 55%] tensorflow_datasets\core\download\extractor_test.py .F.FsFFFF.FF [ 58%] tensorflow_datasets\core\download\kaggle_test.py ....s [ 59%] tensorflow_datasets\core\download\resource_test.py .s.s [ 60%] tensorflow_datasets\core\features\audio_feature_test.py ....s. [ 61%] tensorflow_datasets\core\features\bounding_boxes_test.py .s [ 62%] tensorflow_datasets\core\features\class_label_feature_test.py .......s. [ 64%] tensorflow_datasets\core\features\features_test.py ....s....s.F. [ 67%] tensorflow_datasets\core\features\image_feature_test.py ...s [ 68%] tensorflow_datasets\core\features\sequence_feature_test.py .........s... [ 71%] .s [ 72%] tensorflow_datasets\core\features\text_feature_test.py .s... [ 73%] tensorflow_datasets\core\features\top_level_feature_test.py ..s. [ 74%] tensorflow_datasets\core\features\translation_feature_test.py s...s.. [ 75%] tensorflow_datasets\core\features\video_feature_test.py s.s. [ 76%] tensorflow_datasets\core\features\text\subword_text_encoder_test.py .... [ 77%] ................s.......s [ 83%] tensorflow_datasets\core\features\text\text_encoder_test.py ...s....s... [ 86%] ....s.....s.......... [ 91%] tensorflow_datasets\core\utils\gcs_utils_test.py Fs [ 91%] tensorflow_datasets\core\utils\image_utils_test.py ....s [ 92%] tensorflow_datasets\core\utils\py_utils_test.py ....sF..s..s [ 95%] tensorflow_datasets\core\utils\tf_utils_test.py .s [ 96%] tensorflow_datasets\core\utils\tqdm_utils_test.py .s [ 96%] tensorflow_datasets\core\utils\version_test.py ........s... [ 99%] tensorflow_datasets\core\visualization\show_examples_test.py s. [100%] ================================== FAILURES =================================== c:\programdata\miniconda3\lib\site-packages\tensorflow_core\python\framework\test_util.py:2839: AssertionError: Exception of type : NewRandomAccessFile failed to Create/Open: C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\non_existent_dir\dataset_info.json : The system cannot find the path specified. C:\ProgramData\Miniconda3\lib\site-packages\absl\testing\absltest.py:1704: AssertionError: Lists differ: [{'filename': '/foo/bar\\mnist-train.tfrecor[381 chars] -1}] != [{'skip': 0, 'take': -1, 'filename': '/foo/b[376 chars]05'}] C:\ProgramData\Miniconda3\lib\site-packages\absl\testing\absltest.py:1704: AssertionError: Lists differ: [{'filename': '/foo/bar\\mnist-train.tfrecor[295 chars] -1}] != [{'skip': 1, 'take': -1, 'filename': '/foo/b[291 chars]05'}] C:\ProgramData\Miniconda3\lib\site-packages\absl\testing\absltest.py:1704: AssertionError: Lists differ: [{'filename': '/foo/bar\\mnist-train.tfrecor[36 chars]: 1}] != [{'skip': 1, 'take': 1, 'filename': '/foo/ba[35 chars]05'}] C:\ProgramData\Miniconda3\lib\site-packages\absl\testing\absltest.py:1704: AssertionError: Lists differ: [{'filename': '/foo/bar\\mnist-train.tfrecor[122 chars]: 1}] != [{'skip': 2, 'take': -1, 'filename': '/foo/b[120 chars]05'}] C:\ProgramData\Miniconda3\lib\site-packages\absl\testing\absltest.py:1704: AssertionError: Lists differ: [{'filename': '/foo/bar\\mnist-train.tfrecor[208 chars]: 1}] != [{'skip': 0, 'take': -1, 'filename': '/foo/b[205 chars]05'}] C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\download\download_manager_test.py:149: KeyError: 'http://foo-bar.ch/a' C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\download\download_manager_test.py:154: KeyError: '/dl_dir\\foo-bar.ch_aRT4otYZNT3fw0YWfAY86H0fTfrB8DwysDCMbMav-eDU.zip' C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\download\download_manager_test.py:149: KeyError: 'http://foo-bar.ch/a.zip' C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\download\download_manager_test.py:154: KeyError: '/dl_dir\\akey_1234jla2OYgu-R9nxZeWXftFHL0uS0HRqM2bAd0yQapRh5Y' C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\download\download_manager_test.py:154: KeyError: '/dl_dir/cached' C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\download\download_manager_test.py:154: KeyError: '/dl_dir\\foo-bar.ch_aLyNxI-LGkoLrOHdfNac3cVR9lB1Qk3JIY64g93hUuQw.tar.gz' C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\download\extractor_test.py:122: AssertionError: b'image,label\n1.jpeg,0\n2.jpeg,1\n3.jpeg,1\n3.jpeg,2\n' != b'image,label\r\n1.jpeg,0\r\n2.jpeg,1\r\n3.jpeg,1\r\n3.jpeg,2\r\n' C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\download\extractor_test.py:116: AssertionError: b'image,label\n1.jpeg,0\n2.jpeg,1\n3.jpeg,1\n3.jpeg,2\n' != b'image,label\r\n1.jpeg,0\r\n2.jpeg,1\r\n3.jpeg,1\r\n3.jpeg,2\r\n' C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\download\extractor_test.py:78: AssertionError: b'image,label\n1.jpeg,0\n2.jpeg,1\n3.jpeg,1\n3.jpeg,2\n' != b'image,label\r\n1.jpeg,0\r\n2.jpeg,1\r\n3.jpeg,1\r\n3.jpeg,2\r\n' : File C:\Users\VIJAY\AppData\Local\Temp\pytest-scriptnu7f8ct2\tmprj_7opnp\extracted_arch\foo.csv has bad content. C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\download\extractor_test.py:78: AssertionError: b'image,label\n1.jpeg,0\n2.jpeg,1\n3.jpeg,1\n3.jpeg,2\n' != b'image,label\r\n1.jpeg,0\r\n2.jpeg,1\r\n3.jpeg,1\r\n3.jpeg,2\r\n' : File C:\Users\VIJAY\AppData\Local\Temp\pytest-scriptnu7f8ct2\tmp6l9kgz8o\extracted_arch\foo.csv has bad content. C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\download\extractor_test.py:78: AssertionError: b'image,label\n1.jpeg,0\n2.jpeg,1\n3.jpeg,1\n3.jpeg,2\n' != b'image,label\r\n1.jpeg,0\r\n2.jpeg,1\r\n3.jpeg,1\r\n3.jpeg,2\r\n' : File C:\Users\VIJAY\AppData\Local\Temp\pytest-scriptnu7f8ct2\tmp93d93605\extracted_arch\foo.csv has bad content. C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\download\extractor_test.py:78: AssertionError: b'image,label\n1.jpeg,0\n2.jpeg,1\n3.jpeg,1\n3.jpeg,2\n' != b'image,label\r\n1.jpeg,0\r\n2.jpeg,1\r\n3.jpeg,1\r\n3.jpeg,2\r\n' : File C:\Users\VIJAY\AppData\Local\Temp\pytest-scriptnu7f8ct2\tmpzu2m8n1a\extracted_arch\foo.csv has bad content. c:\programdata\miniconda3\lib\site-packages\tensorflow_core\python\framework\test_util.py:2839: AssertionError: Exception of type : local variable 'dst_path' referenced before assignment C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\download\extractor_test.py:78: AssertionError: b'image,label\n1.jpeg,0\n2.jpeg,1\n3.jpeg,1\n3.jpeg,2\n' != b'image,label\r\n1.jpeg,0\r\n2.jpeg,1\r\n3.jpeg,1\r\n3.jpeg,2\r\n' : File C:\Users\VIJAY\AppData\Local\Temp\pytest-scriptnu7f8ct2\tmpyss0r9ea\extracted_arch\foo.csv has bad content. c:\programdata\miniconda3\lib\site-packages\tensorflow_core\python\framework\test_util.py:2839: AssertionError: Exception of type : Dtype int32 do not match float32 C:\ProgramData\Miniconda3\lib\site-packages\tensorflow_core\python\lib\io\file_io.py:280: tensorflow.python.framework.errors_impl.UnimplementedError: File system scheme 'gs' not implemented (file: 'gs://tfds-data/datasets\mnist') C:\Users\VIJAY\Desktop\GitHub_Repos\datasets\tensorflow_datasets\core\utils\py_utils_test.py:187: AssertionError: False is not true =========================== short test summary info =========================== FAILED tensorflow_datasets/core/dataset_info_test.py::DatasetInfoTest::test_non_existent_dir FAILED tensorflow_datasets/core/tfrecords_reader_test.py::GetDatasetFilesTest::test_no_skip_no_take FAILED tensorflow_datasets/core/tfrecords_reader_test.py::GetDatasetFilesTest::test_skip FAILED tensorflow_datasets/core/tfrecords_reader_test.py::GetDatasetFilesTest::test_skip_take1 FAILED tensorflow_datasets/core/tfrecords_reader_test.py::GetDatasetFilesTest::test_skip_take2 FAILED tensorflow_datasets/core/tfrecords_reader_test.py::GetDatasetFilesTest::test_take FAILED tensorflow_datasets/core/download/download_manager_test.py::DownloadManagerTest::test_download FAILED tensorflow_datasets/core/download/download_manager_test.py::DownloadManagerTest::test_download_and_extract FAILED tensorflow_datasets/core/download/download_manager_test.py::DownloadManagerTest::test_download_and_extract_already_downloaded FAILED tensorflow_datasets/core/download/download_manager_test.py::DownloadManagerTest::test_download_and_extract_archive_ext_in_fname FAILED tensorflow_datasets/core/download/download_manager_test.py::DownloadManagerTest::test_extract FAILED tensorflow_datasets/core/download/download_manager_test.py::DownloadManagerTest::test_force_download_and_extract FAILED tensorflow_datasets/core/download/extractor_test.py::ExtractorTest::test_bzip2 FAILED tensorflow_datasets/core/download/extractor_test.py::ExtractorTest::test_gzip2 FAILED tensorflow_datasets/core/download/extractor_test.py::ExtractorTest::test_tar FAILED tensorflow_datasets/core/download/extractor_test.py::ExtractorTest::test_tar_stream FAILED tensorflow_datasets/core/download/extractor_test.py::ExtractorTest::test_targz FAILED tensorflow_datasets/core/download/extractor_test.py::ExtractorTest::test_targz_stream FAILED tensorflow_datasets/core/download/extractor_test.py::ExtractorTest::test_wrong_method FAILED tensorflow_datasets/core/download/extractor_test.py::ExtractorTest::test_zip FAILED tensorflow_datasets/core/features/features_test.py::FeatureTensorTest::test_shape_static FAILED tensorflow_datasets/core/utils/gcs_utils_test.py::GcsUtilsTest::test_mnist FAILED tensorflow_datasets/core/utils/py_utils_test.py::PyUtilsTest::test_tfds_dir ==== 23 failed, 336 passed, 66 skipped, 36 warnings in 1127.15s (0:18:47) ===== ```

vijayphoenix commented 4 years ago

No bugs were encountered in tfds/proto, tfds/scripts, tfds/summarization, tfds/testing and tfds/video

lukewys commented 2 years ago

Hi! It seems that https://github.com/tensorflow/datasets/blob/master/tensorflow_datasets/vision_language/wit/wit.py#L25 will cause the importing tensorflow_datasets on windows to throw:

csv.field_size_limit(sys.maxsize)
OverflowError: Python int too large to convert to C long