In broad strokes, the Uploader deals with applying changes to vN files and with doing deletions. The Uploader serializes these changes to specific file groups (with specific fileGroupUUID's). In the best implementation, I would like the Uploader to run in something like AWS lambda where each file group would be handled with its own lambda queue-- to ensure serialization on a per file group basis. Since handling these vN changes could be a large part of server processing (i.e., with respect to duration and costs), handling them in a lambda could be an efficient way to go-- it could reduce the number of actual server instances required.
However, due to the fact that I'm using Swift server-side, and that AWS lambda support for Swift is limited, I've not yet gone this route. (e.g., lambda support for Swift doesn't include much beyond HTTP handling; handling full-on database access seems not yet there; see https://github.com/swift-server/swift-aws-lambda-runtime/issues/165).
With this in mind, my current implementation of the Uploader puts this inside of the actual server. I deal with serialization effectively, but pretty crudely. I allow only a single instance of an Uploader to run at any one time, even if there are multiple server instances. (I do this using mySQL named locks-- see https://dev.mysql.com/doc/refman/5.7/en/locking-functions.html#function_get-lock). After doing a file upload or a file deletion, the server determines if it should run the Uploader (upon completing a batch of uploads for file uploads, and always for file deletions). If the mySQL lock can be obtained, then the Uploader is run. Otherwise, at least until very recently, the Uploader only runs again when another file upload or file deletion determines it should.
In my testing with iOSBasics (https://github.com/SyncServerII/iOSBasics.git) yesterday (9/9/20), I ran across an issue that early in thinking about the Uploader I realized would arise, but hadn't come across yet. I ran a test where vN files from two file groups were uploaded effectively in parallel. The method in iOSBasics is called queue and this parallel operation is is allowed because in general I am allowing independent upload operations on different file groups. This generates a race condition on the server. Under certain timings, the Uploader will run and finalize the uploads for both of these vN uploads. Under other timings, the Uploader not yet detect the vN upload from one of the uploads, and the ongoing server state -- without further triggering if the Uploader (which, at that point, would only occur with further upload deletions or further specific file uploads) is that one of the uploads would not be finalized.
This random lack of finalization of the uploads makes it difficult for testing. In more general server operation it would also introduce a non-determinism into when vN uploads would complete. A specific client could never be sure when the upload would really complete, and thus when other clients could download that change.
To deal with this issue, I am now adding in another triggering mechanism to have the Uploader run. Currently, in my testing code, it is based purely on a timer. Every 30 seconds (and I will make the number configurable with the server), the Uploader will run and process any pending vN uploads and file deletions that haven't been processed. So, a client can know that no more than 30 seconds can pass before an upload will be available for download.
I'm going to add one further complication to this timer-based triggering. I'm going to run the timer-based triggering of the Uploader, only if the Uploader has not run in that interval by other endpoint requests. This will mean that if the server has a regular stream of vN and/or deletion requests hitting it that the timer-based triggering of the Uploader will never run.
With my efforts on this next version of the SyncServerII server (i.e., those in the https://github.com/SyncServerII/ServerMain.git repo), I have moved most of the conflict resolution server-side (e.g., see https://github.com/SyncServerII/ChangeResolvers.git). This has brought with it the introduction of the
Uploaderinto the server code.In broad strokes, the
Uploaderdeals with applying changes to vN files and with doing deletions. TheUploaderserializes these changes to specific file groups (with specific fileGroupUUID's). In the best implementation, I would like the Uploader to run in something like AWS lambda where each file group would be handled with its own lambda queue-- to ensure serialization on a per file group basis. Since handling these vN changes could be a large part of server processing (i.e., with respect to duration and costs), handling them in a lambda could be an efficient way to go-- it could reduce the number of actual server instances required.However, due to the fact that I'm using Swift server-side, and that AWS lambda support for Swift is limited, I've not yet gone this route. (e.g., lambda support for Swift doesn't include much beyond HTTP handling; handling full-on database access seems not yet there; see https://github.com/swift-server/swift-aws-lambda-runtime/issues/165).
With this in mind, my current implementation of the Uploader puts this inside of the actual server. I deal with serialization effectively, but pretty crudely. I allow only a single instance of an Uploader to run at any one time, even if there are multiple server instances. (I do this using mySQL named locks-- see https://dev.mysql.com/doc/refman/5.7/en/locking-functions.html#function_get-lock). After doing a file upload or a file deletion, the server determines if it should run the Uploader (upon completing a batch of uploads for file uploads, and always for file deletions). If the mySQL lock can be obtained, then the Uploader is run. Otherwise, at least until very recently, the Uploader only runs again when another file upload or file deletion determines it should.
In my testing with iOSBasics (https://github.com/SyncServerII/iOSBasics.git) yesterday (9/9/20), I ran across an issue that early in thinking about the Uploader I realized would arise, but hadn't come across yet. I ran a test where vN files from two file groups were uploaded effectively in parallel. The method in iOSBasics is called
queueand this parallel operation is is allowed because in general I am allowing independent upload operations on different file groups. This generates a race condition on the server. Under certain timings, the Uploader will run and finalize the uploads for both of these vN uploads. Under other timings, the Uploader not yet detect the vN upload from one of the uploads, and the ongoing server state -- without further triggering if the Uploader (which, at that point, would only occur with further upload deletions or further specific file uploads) is that one of the uploads would not be finalized.This random lack of finalization of the uploads makes it difficult for testing. In more general server operation it would also introduce a non-determinism into when vN uploads would complete. A specific client could never be sure when the upload would really complete, and thus when other clients could download that change.
To deal with this issue, I am now adding in another triggering mechanism to have the Uploader run. Currently, in my testing code, it is based purely on a timer. Every 30 seconds (and I will make the number configurable with the server), the Uploader will run and process any pending vN uploads and file deletions that haven't been processed. So, a client can know that no more than 30 seconds can pass before an upload will be available for download.
I'm going to add one further complication to this timer-based triggering. I'm going to run the timer-based triggering of the Uploader, only if the Uploader has not run in that interval by other endpoint requests. This will mean that if the server has a regular stream of vN and/or deletion requests hitting it that the timer-based triggering of the Uploader will never run.