apple / batch-processing-gateway

The gateway component to make Spark on K8s much easier for Spark users.
Apache License 2.0
182 stars 38 forks source link

increasing datanode PVC storage from 5g to 6g #4

Closed lixmgl closed 1 year ago

lixmgl commented 1 year ago

… storage for data nodes

What is the Purpose of the Change

Current 5G storage for datanodes will block put-object operation on s3 buckets, more details can be found: https://github.com/apple/batch-processing-gateway/issues/1

Type of Change

Brief Change Log

datanode-statefulset.yaml in ozone

Verifying This Change

(Please pick either of the following options)

I am able to put-object (eventlog/) to new S3 bucket(test) with this change: rainieli-D9VN436:~ rainieli$ aws s3 ls s3://test --recursive --endpoint http://localhost:9878 2022-11-02 10:17:40 0 eventlog/

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

Documentation

Final Checklist: