search

Signbank / Global-signbank

An online sign dictionary and sign database management system for research purposes. Developed originally by Steve Cassidy/ This repo is a fork for the Dutch version, previously called 'NGT-Signbank'.
http://signbank.cls.ru.nl
BSD 3-Clause "New" or "Revised" License
19 stars 12 forks source link

Excess number of backup videos being created by the API #1397

Open susanodd opened 3 days ago

susanodd commented 3 days ago

For some reason, numerous video uploads are being done for the same gloss repeatedly, each creating a new backup of the previous video upload. Sometimes upwards of 30 video files. There are matching requests for each of these. "Op hol slaan"

Jetske commented 3 days ago

After asking @rem0g it turns out this was probably an upload script stuck in a loop. Has been fixed.

To clean up the duplicate backup files, is there already command for that (if so I can't find it)? Or what would be the best way to do this?

susanodd commented 3 days ago

There were some more today, so I don't know if it's totally fixed. Regardless, there is nothing to stop the API user from uploading the same video file over and over again.

There isn't a command for them. The files need to be deleted before you can delete the objects. But then the version numbers need to be modified, because they won't be correct anymore. I was working on that for the old format "bak.bak" files. But I didn't delete files. I only removed objects if there was no file. Then enumerated over what was left to give them new version numbers. I'm guessing though that all the backup files created can safely be deleted. I guess we could implement a command to do that in the admin. The files need to be "unlinked". You can see that in the video code. If you just remove the files you can also just remove the objects. We don't really know how extensive this is. If you look at the files in the terminal screen, it's kind of impossible to determine just by looking what is to delete. All the file names start with the same first two characters, then you need to browse to the end to see if it's the same gloss or not. So it would be time consuming to do by hand. It would also be difficult to keep searching in the admin for ones to select, to delete. (Assuming a new admin command.) It's also unfortunate that all the new backup files have version 1. An admin command to "delete a backup video" would need to 1. unlink the file if a file exists 2. delete the object 3. renumber all the versions of all the other backup videos for the same gloss @Jetske I'll make a command as such. Probably in the API request, it ought to check if the "new" video is the same as the "existing" video. There's an OS command for that.
susanodd commented 2 days ago

@susanodd is making a video admin "remove backups" command for selected GlossVideo objects:

If the object is a backup:

  1. unlink the file if a file exists
  2. delete the object
  3. renumber all the versions of all the other backup videos for the same gloss

@Woseseltops:

Probably in the API request, it ought to check if the "new" video is the same as the "existing" video. There's an OS command for that. Ignore the request if so. (Note, there is nothing to stop the API from uploading the same video over and over again. This necessitates the command above. On Signbank, the user would become RSI-exhausted doing repeated uploads by hand.)