apache / trafficserver

Apache Traffic Server™ is a fast, scalable and extensible HTTP/1.1 and HTTP/2 compliant caching proxy server.
https://trafficserver.apache.org/
Apache License 2.0
1.8k stars 798 forks source link

How do we configure Tiered Storage in ATS v8.x.x? #9446

Open dhairav opened 1 year ago

dhairav commented 1 year ago

Hi Team,

We are attempting to use ATS 8.1.1 Debian builds - where we want to utilize multiple HDDs and SSDs in order to get the maximum I/O throughput possible from the hardware. We have come across this documentation link - https://docs.trafficserver.apache.org/en/8.1.x/developer-guide/cache-architecture/tiered-storage.en.html This document mentions using Each storage unit in storage.config can be marked with a quality value which is 32 bit number. But we have not been able to specify and run the same.

We have not been able to implement and run it simply due to the lack of documentation on how to write the configuration correctly in the storage.config file.

The documentation explicitly states that we'll be able to Group similar kinds of storage into different volumes. For example split out SSD’s or RAM drives into their own volume., which is what we want to achieve as well.

Our end goal is to allow ATS to decide where to store objects - on fast SSDs or on slower HDDs - depending on the popularity of the objects - similar to how it handles RAM Cache and Disk cache conceptually.

Please help me figure out how to specify a bifurcation between Fast SSD storage & slow HDD storage in the storage.config file. Any help on this is appreciated.

dhairav commented 1 year ago

Hello team, a ping on this issue. I want to understand if it is possible for me to configure/experiment around the above premise.

mlibbey commented 1 year ago

Here's a snippet of a machine in our production, where we have a SSD and RAM Disk.

$ cat storage.config
/dev/disk/by-path/pci-0000:03:00.0-sas-0x3009865483c90bf1-lun-0  volume=1
/dev/disk/by-path/pci-0000:03:00.0-sas-0x3009865483c90bf2-lun-0  volume=1
/dev/disk/by-path/pci-0000:03:00.0-sas-0x3009865483c90bf3-lun-0  volume=1
/dev/ram0 volume=2
$ cat hosting.config
hostname=example.com volume=2
hostname=another.example.com volume=2
github-actions[bot] commented 6 months ago

This issue has been automatically marked as stale because it has not had recent activity. Marking it stale to flag it for further consideration by the community.