Open l-vincent-l opened 4 years ago
cc @bzg
Hi @l-vincent-l! Thanks a lot! One question: what is the interval of the data streamed ? Is it "from the beginning of times until today at 4AM"?
@bzg dumps will start to be a couple of gigs soon, is it ok to stream the full dump each time?
It takes everything that has been produce the day before.
I think we should have in instance (process) that does just that, to avoid the duplication problem.
You need to adapt config/prod.exs with the connection options of https://hexdocs.pm/sftp_client/SFTPClient.Operations.Connect.html#connect/1