I maintain an active dataset of image metadata for the Mars 2020 mission images.
At this point there is an update at least once a day or once in two days and the dataset is only going to grow larger.
My issue now is that I have a cron job that handles my update process. I see that the dataset update process uploads all files of the dataset again kind of a replace all situation.
I come here to ask if there is a way to upload only diffs of the dataset i.e., the new items on the dataset and not multiple files.
If not, can we make this a feature request, and I can may be contribute ?
Hello Team and Fellow Kagglers,
I maintain an active dataset of image metadata for the Mars 2020 mission images. At this point there is an update at least once a day or once in two days and the dataset is only going to grow larger.
My issue now is that I have a cron job that handles my update process. I see that the dataset update process uploads all files of the dataset again kind of a replace all situation.
I come here to ask if there is a way to upload only diffs of the dataset i.e., the new items on the dataset and not multiple files. If not, can we make this a feature request, and I can may be contribute ?
thanks in advance! Saketh.