search

google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://ai.google.dev/edge/mediapipe
Apache License 2.0
27.39k stars 5.14k forks source link

ImageProcessingOptions package is not correct in docs #5283

Open JC3 opened 7 months ago

JC3 commented 7 months ago

Description of issue (what needs changing)

Whatever package ImageProcessingOptions is in is not documented correctly.

Clear description

https://developers.google.com/mediapipe/api/solutions/python/mp/tasks/vision/holistic_landmarker/image_processing_options_module/ImageProcessingOptions shows that ImageProcessingOptions is in mediapipe.tasks.vision.holistic_landmarker.image_processing_options_module but this package does not exist. I have no idea where this class is exported from.

Correct links

No response

Parameters defined

No response

Returns defined

No response

Raises listed and defined

No response

Usage example

No response

Request visuals, if applicable

No response

Submit a pull request?

No response

JC3 commented 7 months ago

If anybody knows what package this is in please let me know, it's currently blocking some development.

I also hand-wavily tried mediapipe.tasks.python.vision.holistic_landmarker.image_processing_options_module to no avail.

JC3 commented 7 months ago

I found it in mediapipe.tasks.python.vision.core.image_processing_options after looking at some actual MediaPipe source code.

Also mediapipe.tasks.components.containers is another one that doesn't exist, it's actually mediapipe.tasks.python.components.containers.

Package names need to be correctly identified in the Python API documentation. The docs are full of nonexistent packages.

kinarr commented 6 months ago

Adding the following code to https://github.com/google/mediapipe/blob/master/mediapipe/tasks/python/vision/__init__.py should fix it most likely:

ImageProcessingOptions = core.image_processing_options.ImageProcessingOptions

del holistic_landmarker
markmcd commented 6 months ago

https://developers.google.com/mediapipe/api/solutions/python/mp/tasks/vision/holistic_landmarker/image_processing_options_module/ImageProcessingOptions shows that ImageProcessingOptions is in mediapipe.tasks.vision.holistic_landmarker.image_processing_options_module but this package does not exist. I have no idea where this class is exported from.

If you take the fully-qualified class from that URL, it exists:

$ pip install mediapipe
...
$ python
>>> import mediapipe as mp
>>> mp.__version__
'0.10.11'

>>> mp.tasks.vision.holistic_landmarker.image_processing_options_module.ImageProcessingOptions()
ImageProcessingOptions(region_of_interest=None, rotation_degrees=0)

>>> mp.tasks.vision.holistic_landmarker.image_processing_options_module
<module 'mediapipe.tasks.python.vision.core.image_processing_options' from '/tmp/mp/lib/python3.11/site-packages/mediapipe/tasks/python/vision/core/image_processing_options.py'>

I found it in mediapipe.tasks.python.vision.core.image_processing_options

This looks like the path from the source directory, not the mediapipe pip module. i.e. if you replace the .s with /, you can find it on GitHub here.

Also mediapipe.tasks.components.containers is another one that doesn't exist, it's actually mediapipe.tasks.python.components.containers.

This is the same. If you use the namespace from the docs directly in Python, you'll get the module you expect.

Are you testing this via the Python CLI from within the Mediapipe repo? Or are you installing the package through pip? If pip, what version of mediapipe did you install? Can you provide a minimal source file that fails to load the class you linked?

Or is the concern that you are unable to find the Mediapipe source from an API symbol? Each page on the API ref should have a link to GitHub at the top where you can find the file. You can call .__file__ on modules to find the file too, though it will be in your virtualenv, so you will need to take the mediapipe/... part and map it to the repository.

kinarr commented 6 months ago

I believe this issue was raised because the following commit didn't remove the holistic_landmarker module in the __init__.py file.

https://github.com/google/mediapipe/commit/d87e8dbaea78488fc07661f13e32e482db4d0ccd

On the other hand when it's comes to the Face Aligner API and the older APIs it's been exposed properly:

https://github.com/google/mediapipe/commit/c6e3f0828248c50ff62fe51113b18bb1b7850698

kuaashish commented 6 months ago

Hi @JC3,

Could you kindly review the above comments?

Thank you!!

JC3 commented 6 months ago

@kuaashish It is not a matter of just fixing the code. My remark that the docs are full of nonexistent packages stands.

I ran a test and attempted to import every package identified in the documentation. Of the 90 packages/modules identified, only 36 exist. Packages identified with FAIL are specified in the documentation but do not actually exist:

  OK   mediapipe
  OK   mediapipe.calculators
  OK   mediapipe.calculators.core
  OK   mediapipe.calculators.core.constant_side_packet_calculator_pb2
  FAIL mediapipe.calculators.core.constant_side_packet_calculator_pb2.mediapipe_dot_framework_dot_calculator__options__pb2
  FAIL mediapipe.calculators.core.constant_side_packet_calculator_pb2.mediapipe_dot_framework_dot_calculator__pb2
  FAIL mediapipe.calculators.core.constant_side_packet_calculator_pb2.mediapipe_dot_framework_dot_calculator__pb2.mediapipe_dot_framework_dot_mediapipe__options__pb2
  FAIL mediapipe.calculators.core.constant_side_packet_calculator_pb2.mediapipe_dot_framework_dot_calculator__pb2.mediapipe_dot_framework_dot_packet__factory__pb2
  FAIL mediapipe.calculators.core.constant_side_packet_calculator_pb2.mediapipe_dot_framework_dot_calculator__pb2.mediapipe_dot_framework_dot_packet__generator__pb2
  FAIL mediapipe.calculators.core.constant_side_packet_calculator_pb2.mediapipe_dot_framework_dot_calculator__pb2.mediapipe_dot_framework_dot_status__handler__pb2
  FAIL mediapipe.calculators.core.constant_side_packet_calculator_pb2.mediapipe_dot_framework_dot_calculator__pb2.mediapipe_dot_framework_dot_stream__handler__pb2
  FAIL mediapipe.calculators.core.constant_side_packet_calculator_pb2.mediapipe_dot_framework_dot_formats_dot_classification__pb2
  FAIL mediapipe.calculators.core.constant_side_packet_calculator_pb2.mediapipe_dot_framework_dot_formats_dot_landmark__pb2
  FAIL mediapipe.calculators.core.constant_side_packet_calculator_pb2.mediapipe_dot_framework_dot_formats_dot_matrix__data__pb2
  FAIL mediapipe.calculators.core.constant_side_packet_calculator_pb2.mediapipe_dot_framework_dot_formats_dot_time__series__header__pb2
  OK   mediapipe.calculators.core.flow_limiter_calculator_pb2
  OK   mediapipe.calculators.core.gate_calculator_pb2
  OK   mediapipe.calculators.core.split_vector_calculator_pb2
  OK   mediapipe.calculators.image
  OK   mediapipe.calculators.image.image_transformation_calculator_pb2
  FAIL mediapipe.calculators.image.image_transformation_calculator_pb2.mediapipe_dot_gpu_dot_scale__mode__pb2
  OK   mediapipe.calculators.image.rotation_mode_pb2
  OK   mediapipe.calculators.image.warp_affine_calculator_pb2
  FAIL mediapipe.calculators.image.warp_affine_calculator_pb2.mediapipe_dot_gpu_dot_gpu__origin__pb2
  OK   mediapipe.calculators.tensor
  OK   mediapipe.calculators.tensor.image_to_tensor_calculator_pb2
  OK   mediapipe.calculators.tensor.inference_calculator_pb2
  OK   mediapipe.calculators.tensor.tensors_to_classification_calculator_pb2
  FAIL mediapipe.calculators.tensor.tensors_to_classification_calculator_pb2.mediapipe_dot_util_dot_label__map__pb2
  OK   mediapipe.calculators.tensor.tensors_to_detections_calculator_pb2
  OK   mediapipe.calculators.tensor.tensors_to_floats_calculator_pb2
  OK   mediapipe.calculators.tensor.tensors_to_landmarks_calculator_pb2
  OK   mediapipe.calculators.tensor.tensors_to_segmentation_calculator_pb2
  OK   mediapipe.calculators.tflite
  OK   mediapipe.calculators.tflite.ssd_anchors_calculator_pb2
  FAIL mediapipe.calculators.tflite.ssd_anchors_calculator_pb2.mediapipe_dot_framework_dot_formats_dot_object__detection_dot_anchor__pb2
  OK   mediapipe.calculators.util
  OK   mediapipe.calculators.util.association_calculator_pb2
  OK   mediapipe.calculators.util.collection_has_min_size_calculator_pb2
  OK   mediapipe.calculators.util.detection_label_id_to_text_calculator_pb2
  OK   mediapipe.calculators.util.detections_to_rects_calculator_pb2
  OK   mediapipe.calculators.util.landmark_projection_calculator_pb2
  OK   mediapipe.calculators.util.landmarks_refinement_calculator_pb2
  OK   mediapipe.calculators.util.landmarks_smoothing_calculator_pb2
  OK   mediapipe.calculators.util.local_file_contents_calculator_pb2
  OK   mediapipe.calculators.util.logic_calculator_pb2
  OK   mediapipe.calculators.util.non_max_suppression_calculator_pb2
  OK   mediapipe.calculators.util.rect_transformation_calculator_pb2
  OK   mediapipe.calculators.util.thresholding_calculator_pb2
  OK   mediapipe.calculators.util.visibility_smoothing_calculator_pb2
  FAIL mediapipe.model_ckpt_util
  FAIL mediapipe.packet_creator
  FAIL mediapipe.packet_getter
  FAIL mediapipe.resource_util
  OK   mediapipe.tasks
  FAIL mediapipe.tasks.audio
  FAIL mediapipe.tasks.components
  FAIL mediapipe.tasks.components.containers
  FAIL mediapipe.tasks.components.containers.keypoint
  FAIL mediapipe.tasks.components.containers.keypoint.location_data_pb2
  FAIL mediapipe.tasks.components.containers.keypoint.location_data_pb2.mediapipe_dot_framework_dot_formats_dot_annotation_dot_rasterization__pb2
  FAIL mediapipe.tasks.components.processors
  FAIL mediapipe.tasks.components.utils
  FAIL mediapipe.tasks.components.utils.cosine_similarity
  FAIL mediapipe.tasks.components.utils.cosine_similarity.embedding_result
  FAIL mediapipe.tasks.components.utils.cosine_similarity.embedding_result.embeddings_pb2
  FAIL mediapipe.tasks.text
  FAIL mediapipe.tasks.text.core
  FAIL mediapipe.tasks.text.core.base_text_task_api
  FAIL mediapipe.tasks.text.core.base_text_task_api.task_runner
  FAIL mediapipe.tasks.vision
  FAIL mediapipe.tasks.vision.holistic_landmarker
  FAIL mediapipe.tasks.vision.holistic_landmarker.base_options_module
  FAIL mediapipe.tasks.vision.holistic_landmarker.base_options_module.acceleration_pb2
  FAIL mediapipe.tasks.vision.holistic_landmarker.base_options_module.base_options_pb2
  FAIL mediapipe.tasks.vision.holistic_landmarker.base_options_module.external_file_pb2
  FAIL mediapipe.tasks.vision.holistic_landmarker.base_vision_task_api
  FAIL mediapipe.tasks.vision.holistic_landmarker.base_vision_task_api.math
  FAIL mediapipe.tasks.vision.holistic_landmarker.base_vision_task_api.rect_module
  FAIL mediapipe.tasks.vision.holistic_landmarker.base_vision_task_api.rect_module.rect_pb2
  FAIL mediapipe.tasks.vision.holistic_landmarker.holistic_landmarker_graph_options_pb2
  FAIL mediapipe.tasks.vision.holistic_landmarker.holistic_landmarker_graph_options_pb2.mediapipe_dot_tasks_dot_cc_dot_vision_dot_face__detector_dot_proto_dot_face__detector__graph__options__pb2
  FAIL mediapipe.tasks.vision.holistic_landmarker.holistic_landmarker_graph_options_pb2.mediapipe_dot_tasks_dot_cc_dot_vision_dot_face__landmarker_dot_proto_dot_face__landmarks__detector__graph__options__pb2
  FAIL mediapipe.tasks.vision.holistic_landmarker.holistic_landmarker_graph_options_pb2.mediapipe_dot_tasks_dot_cc_dot_vision_dot_face__landmarker_dot_proto_dot_face__landmarks__detector__graph__options__pb2.mediapipe_dot_tasks_dot_cc_dot_vision_dot_face__landmarker_dot_proto_dot_face__blendshapes__graph__options__pb2
  FAIL mediapipe.tasks.vision.holistic_landmarker.holistic_landmarker_graph_options_pb2.mediapipe_dot_tasks_dot_cc_dot_vision_dot_hand__landmarker_dot_proto_dot_hand__landmarks__detector__graph__options__pb2
  FAIL mediapipe.tasks.vision.holistic_landmarker.holistic_landmarker_graph_options_pb2.mediapipe_dot_tasks_dot_cc_dot_vision_dot_hand__landmarker_dot_proto_dot_hand__roi__refinement__graph__options__pb2
  FAIL mediapipe.tasks.vision.holistic_landmarker.holistic_landmarker_graph_options_pb2.mediapipe_dot_tasks_dot_cc_dot_vision_dot_pose__detector_dot_proto_dot_pose__detector__graph__options__pb2
  FAIL mediapipe.tasks.vision.holistic_landmarker.holistic_landmarker_graph_options_pb2.mediapipe_dot_tasks_dot_cc_dot_vision_dot_pose__landmarker_dot_proto_dot_pose__landmarks__detector__graph__options__pb2
  FAIL mediapipe.tasks.vision.holistic_landmarker.holistic_result_pb2
  FAIL mediapipe.tasks.vision.holistic_landmarker.image_processing_options_module

This was generated and tested with some Python scripts I hacked together: mpgenimp.tgz

As you can see, it is a bigger, general documentation issue.

This is mediapipe 0.10.9, installed with pip.

kuaashish commented 6 months ago

Hi @markmcd,

Could you please reexamine this issue?

Thank you!!

google-ml-butler[bot] commented 6 months ago

Are you satisfied with the resolution of your issue? Yes No

JC3 commented 6 months ago

That's not the right solution. It also doesn't fix the missing python. in the documented package name.

kinarr commented 6 months ago

@kuaashish As for the remaining issues in the API docs I believe these imports are necessary too. PTAL.

Issue: Unused and internal text core module: https://developers.google.com/mediapipe/api/solutions/python/mp/tasks/text/core Change needed: https://github.com/google/mediapipe/blob/master/mediapipe/tasks/python/text/__init__.py

"""MediaPipe Tasks Text API."""

import mediapipe.tasks.python.text.core

# Remove unnecessary modules to avoid duplication in API docs.
del core

Issue: Wrong keypoint module: https://developers.google.com/mediapipe/api/solutions/python/mp/tasks/components/containers/keypoint Change needed: https://github.com/google/mediapipe/blob/master/mediapipe/tasks/python/components/containers/__init__.py

"""MediaPipe Tasks Components Containers API."""
import mediapipe.tasks.python.components.containers.keypoint

NormalizedKeypoint = keypoint.NormalizedKeypoint

# Remove unnecessary modules to avoid duplication in API docs.
del keypoint

Issue: Improper utils module https://developers.google.com/mediapipe/api/solutions/python/mp/tasks/components/utils/cosine_similarity Change needed: https://github.com/google/mediapipe/blob/master/mediapipe/tasks/python/components/utils/__init__.py

import mediapipe.tasks.python.components.utils.cosine_similarity

cosine_similarity = cosine_similarity.cosine_similarity

# Remove unnecessary modules to avoid duplication in API docs.
del cosine_similarity
del mediapipe
kuaashish commented 6 months ago

@kinarr,

I agree. Thank you for adding more finding here. We should address this issue promptly in our API documentation.