tensorflow / datasets

TFDS is a collection of datasets ready to use with TensorFlow, Jax, ...
https://www.tensorflow.org/datasets
Apache License 2.0
4.3k stars 1.54k forks source link

Add tests labels for `car196` dataset #1218

Closed sweaterr closed 4 years ago

sweaterr commented 4 years ago

Short description

All labels in the test split of the car196 dataset are 196

image

Environment information

Reproduction instructions

train, data_info = tfds.load(name="cars196", split='test',  with_info=True)
for example in train.take(10): 
  image, label = example["image"], example["label"]
  image_buffer = tf.image.encode_jpeg(image)
  print("Label: %d" % label.numpy())

Link to logs If applicable, <link to gist with logs, stack trace>

Expected behavior What you expected to happen.

Additional context Add any other context about the problem here.

Conchylicultor commented 4 years ago

Thanks for reporting, I believe the dataset do not provide tests labels (labels are private and can be evaluated using their evaluation server): https://ai.stanford.edu/~jkrause/cars/car_dataset.html

However, unknown labels should returns -1. Sending a fix.

sweaterr commented 4 years ago

@Conchylicultor thank you for reply!

sweaterr commented 4 years ago

@Conchylicultor

The site you linked has the following phrase:

Update: For ease of development, a tar of all images is available here and all bounding boxes and labels for both training and test are available here. If you were using the evaluation server before (which is still running), you can use test annotations here to evaluate yourself without using the server.

I think that it now provides labels for the test.

Conchylicultor commented 4 years ago

Oh nice. Don't hesitate to send a PR to add the test labels.

captain-pool commented 4 years ago

Hey, @Conchylicultor can I add this as a task for Google Code-In?

Conchylicultor commented 4 years ago

@captain-pool Yes, sounds good!

Conchylicultor commented 4 years ago

Someone added the tests labels internally, this should appear on GitHub soon.