search

mlcommons / training_policies

Issues related to MLPerf™ training policies, including rules and suggested changes
https://mlcommons.org/en/groups/training
Apache License 2.0
92 stars 66 forks source link

[image classification] decode jpeg image in preprocessing #434

Closed delock closed 3 years ago

delock commented 3 years ago

Hi, I want to get clarification whether decode jpeg image and install it in raw (RGB) format in data preprocessing is conforming to training_policies. Decode a jpeg image is part of reference implementation and is non-random part of the data processing, so my understanding is it is allowed. Want to get confirm from WG.

johntran-nv commented 3 years ago

Hi @delock , we discussed this last week, and agreed that jpeg decode alone can be considered a data layout transformation, which is explicitly allowed by the rules, specifically https://github.com/mlcommons/training_policies/blob/master/training_rules.adoc#61-data-state-at-start-of-run.

Note that data augmentation of any kind would invalidate this.

Does that answer your question?

delock commented 3 years ago

@johntran-nv Thanks, that answers my question.