seqeralabs / tower-agent

Agent app that allows connecting remote batch schedulers to Tower Cloud
https://cloud.tower.nf
Apache License 2.0
8 stars 4 forks source link

Allow to send files #15

Open jordeu opened 3 years ago

jordeu commented 3 years ago

Current implementation when Tower request a file from the Agent a cat <filename> it is executed and the response is returned as a result. This works for small files, but for big files or binary files it may be problematic.

The goal of this issue is to implement a new type of request where Tower requests a specific file and the Agent reads it and sends it back to Tower. It has to support the streaming of big files and binary files of any format.

NOTE: Given the current limit of 5MB per websocket message, this will require a paging mechanism to send big files using several messages.