Download or sync?

[joto]

The application is called "...Sync". But as far as I can tell from the code there is no "syncing" going on here but just a download of all assets. Files that are removed in Pretalx are not removed from the downloaded directories. I wonder if that is something that we want, say if somebody uploaded a new version of a file under a different name and then removed the old version.

[joto]

Hm. Now that I am thinking about it. If we deleted files that are non-existent in pretalx we'd delete files that we might have uploaded "manually" and placed in the directories, maybe because there was some temporary problem with Pretalx or so. I guess this is less a technical issue then about the question how we want to organize the workflow here.

[mstock]

The 'Sync' in the name certainly isn't ideal and promises more than the application actually delivers, at least right now, as it's clearly much less than that, but since the application optionally also does an upload to Nextcloud, it is a bit more than just 'download' ;).

It probably wouldn't be too hard to at least optionally delete files not currently in Pretalx and directories of talks that are not currently in the schedule at the given 'position' (which could e.g. happen if a talk got cancelled or moved to another room or time slot, or just renamed to fix some typo). But as you mention, that would also delete manually placed files received via other means, which is probably part of the reason why I didn't implement that so far.

In addition to temporary problems with Pretalx, there's also at least one non-temporary 'problem' with Pretalx that we might actually have run into: The maximum file size it accepts for uploads is fairly small (10 MB last time I checked), and some slide decks are larger than this, and if speakers want to show e.g. a video, these are often much larger. In a setup where Nextcloud is involved, one might also consider to share the room folder in a read-write-mode to the presentation PC, which would allow to collect slide decks from speakers that they brought to the talk on a USB stick in the correct folder for the talk - and automatic deletion would remove those, too.

On the technical side, I currently see two features that are likely worth considering, maybe not too hard to implement and which might be useful for certain workflows, but where I don't know if I would be able to implement any of them in time:

• Right now, there's no robust way to identify the original directory of a talk that was renamed or moved around. So maybe adding the talk code/id to the talk directory would be good. Then an additional step which checks if there's an existing directory anywhere else in the structure with the same code before creating a new directory for a talk could move the existing directory to the currently expected position. This could help avoid some confusion (currently, there would be more than one directory for such a talk, both at the old and new position). Sessions with a 'slot count' > 1 in Pretalx seem to share the same code though, which makes this a bit harder, but the id seems unique, so maybe one should use a combination of both, given that the code is generally a bit more 'visible', for example in Pretalx URLs.
• Optionally delete files and/or (empty?) talk directories not in Pretalx, maybe as two separate options, where the latter should probably be combined with the above idea of moving talk directories around. Instead of deleting files or directories, one could also move them to some kind of trash bin or attic directory.

[joto]

It looks like we don't have a clear understanding yet what "the best workflow" would be here. And I suspect there isn't one "best workflow" anyway. Different conferences have different needs. Maybe it would help if we had some kind of "consolidation view" where we can run the software and it will tell me about events/directories/files that are not in sync and then I can manually intervene or not.

I am thinking about something that looks like a diff between what's in pretalx and what's in the local file system/nextcloud. That would allow me to detect "unusual" situations and manually correct them if needed.

[mstock]

I haven't started to implement such a diff functionality, yet, and don't know if I will get around to do so in time, but would you still consider such a diff functionality as useful?

What I have done though is adding the talk code/id at the end of the name of the talk directory so I can easily match an existing directory with an entry in the schedule on the next run of the script. Based on this, I'm now also moving and/or renaming existing directories if the talk was moved in the schedule, renamed, etc., so the script is now slightly closer to something that might be considered as a 'sync'. In a cleanup step, it also removes empty day and room directories which would be left over if e.g. a room got renamed or all talks got moved to a different room or day. The script does not currently detect talks that vanished from the schedule, but that doesn't seem too hard, so I might extend it to handle that case, too: I'd probably move them to special __Attic__ directory or something like that, or optionally delete them - moving them to an __Attic__ directory would also allow the script to move them back should they reappear in the schedule though without loosing resources that got manually added outside Pretalx. Once that would be implemented, calculating some sort of a diff might already be a bit easier, but might require some sort of dry-run mode.

[joto]

For our current setup the diff functionality is not needed, because we can not use NextCloud anyway. In fact it might even hurt if we copy some extra file in there manually and then it gets deleted or moved to the __Attic__. Lets try out what we have now in practice and see how it goes.