[FR] Avoid uploading data if samples exist in CVAT connected fileshare

ehofesmann commented 3 years ago

CVAT allows files to be accessed either by: 1) Uploading local files (What is currently done) 2) Uploaded through remote URLs 3) Accessed directly through a mounted file share

If the data that is being uploaded exists in a file share connected to CVAT, then it would be preferable to not upload the data to the server. This is especially important in cases where a large number of images or videos are being annotated at one time.

Adding this should be fairly simple. It would require updating this to allow for shared files:

https://github.com/voxel51/fiftyone/blob/e43c00ab96282d1f016e4f57f806c8f48feff6bc/fiftyone/utils/cvat.py#L3096-L3102

 files = {}
 for idx, path in enumerate(paths): 
     # IMPORTANT: CVAT organizes media within a task alphabetically by 
     # filename, so we must give CVAT filenames whose alphabetical order 
     # matches the order of `paths` 
     filename = "%06d%s" % (idx, os.path.splitext(path)[1]) 
     if use_fileshare:
         data["server_files[%d]" % idx] = (filename, path)
     else:
         files["client_files[%d]" % idx] = (filename, open(path, "rb"))

In order to use the correct path, it would be straightforward to follow the Alternate media workflow and store the file share path to every sample as a field on the FiftyOne dataset.

Other points to consider are the options to copy_data and use_cache that will likely need to be incorporated to avoid copying data even for media in the file share. https://github.com/openvinotoolkit/cvat/pull/3544