If the request is a GET or HEAD request for a file, then it should check if there is an active lock for the requested path. If there is, it should wait for the lock to be released (or a configurable timeout) before allowing the request to proceed (or erroring in case of timeout).
When a Pipe is processed, it generates output files from input files. At the start of processing, it should place locks on any previously generated output file paths. Once it has finished processing its input files and updating the output files, it should release its locks. Whilst the locks are in place the request middleware will be halting any request on those files.
Together this will help ensure that if you have any lengthy processing happening on a set of a files, which produces a set of output files, and in the meantime you have requests for one or more of those output files, then you wont get a mixture of some processed outputs versus some not yet processed outputs - because the pipe will lock all of its outputs BEFORE processing starts, it will then update them all, and then it will release the locks - in this way the outputs will version together.
It also helps ensure that if you change an input file, and refresh the browser, if the pipeline is actively processing that change, then the request will stall but you will get updated file once the browser request completes. This is often preferable to the request completing but still getting a stale file.
Middleware should sit in the request pipeline:
If the request is a GET or HEAD request for a file, then it should check if there is an active lock for the requested path. If there is, it should wait for the lock to be released (or a configurable timeout) before allowing the request to proceed (or erroring in case of timeout).
When a
Pipeis processed, it generates output files from input files. At the start of processing, it should place locks on any previously generated output file paths. Once it has finished processing its input files and updating the output files, it should release its locks. Whilst the locks are in place the request middleware will be halting any request on those files.Together this will help ensure that if you have any lengthy processing happening on a set of a files, which produces a set of output files, and in the meantime you have requests for one or more of those output files, then you wont get a mixture of some processed outputs versus some not yet processed outputs - because the pipe will lock all of its outputs BEFORE processing starts, it will then update them all, and then it will release the locks - in this way the outputs will version together.
It also helps ensure that if you change an input file, and refresh the browser, if the pipeline is actively processing that change, then the request will stall but you will get updated file once the browser request completes. This is often preferable to the request completing but still getting a stale file.