dtinit / data-transfer-project

The Data Transfer Project makes it easy for platforms to build interoperable user data portability features. We are establishing a common framework, including data models and protocols, to enable direct transfer of data both into and out of participating online service providers.
https://dtinit.org/docs/dtp-what-is-it
Apache License 2.0
3.56k stars 482 forks source link

Getting Error Code 6(ALREADY_EXISTS) in Google Photos/Videos uploads #1009

Open abhgarg-fb opened 2 years ago

abhgarg-fb commented 2 years ago

While uploading Photos and Videos to Google Photos, we are receiving Error code 6 in Batch upload response.

Error Logs from Google Photos Adapter can be classified into two categories: java.io.IOException: Photo could not be created. Code: 6 Message: Failed: There was an error while trying to create this media item. at org.datatransferproject.datatransfer.google.photos.GooglePhotosImporter.lambda$importPhotoBatch$5(GooglePhotosImporter.java:285)

java.io.IOException: Video item could not be created. Code: 6 Message: EXISTS: There was an error while trying to create this media item. at org.datatransferproject.datatransfer.google.videos.GoogleVideosImporter.lambda$importVideoBatch$5(GoogleVideosImporter.java:249)

As per grpc documentation for error code, Error code 6 signifies ALREADY_EXISTS which means that the media item is a duplicate entry and as per understanding, this error code can be skipped, and job failure can be avoided.

No retries is being logged at worker level. so, duplicate calls to Google server can be ruled out. Could you please connect with the Google Photos API Library team on the above error scenarios and know more on the failures root cause. Thereby, helping us to improve user experience while uploading Photos/Videos to Google Photos?

chinmay7016 commented 10 months ago

Check for Duplicate Media: As you mentioned, Error code 6 typically indicates that the media item you are trying to upload already exists in the user's Google Photos library. You should double-check your upload requests to ensure that you are not inadvertently trying to upload duplicate photos or videos.

Unique Identifiers: When uploading media, make sure you're using unique identifiers for each item. This can include the file name, checksum, or any other attribute that can distinguish one media item from another.

Metadata Changes: If you are trying to re-upload media items that were previously uploaded but need to make changes (e.g., updating metadata), consider using the Google Photos API to update existing items instead of trying to create new ones. This can help avoid the "ALREADY_EXISTS" error.

Error Handling: Implement proper error handling in your code to capture and log the specific details of the "ALREADY_EXISTS" error, such as the unique identifier of the conflicting media item. This can help you track down the specific items causing the problem.

Rate Limiting: Ensure that you are not hitting rate limits imposed by the Google Photos API. Frequent or rapid uploads may trigger rate limiting, leading to errors. Implement appropriate rate limiting and backoff strategies in your code.

API Documentation and Support: Consult the official Google Photos API documentation and reach out to Google's developer support channels or forums for assistance. They may have specific guidance or solutions for your use case.

Testing: Rigorously test your upload process with a variety of media items to ensure it handles all scenarios, including duplicates, gracefully.