epam / cloud-pipeline

Cloud agnostic genomics analysis, scientific computation and storage platform
https://cloud-pipeline.com
Apache License 2.0
144 stars 58 forks source link

Node Logger API requests improvements #3577

Closed SilinPavel closed 5 days ago

SilinPavel commented 5 days ago

Introduce new way to push node logs to the system s3 bucket by mounting the bucket with pipe mount and then use cp linux operation. It shout dramatically reduce number of the requests to the API. Plus audit logging can be disabled now for node loggers, it also should reduce number of requests.