This week I will explain Amazon Web Services Storage Gateway monitoring. In my previous posts, I had configured a gateway-cached volume. To proactively maintain our gateway, we have to measure performance of our gateway and volume(s). So today I will tell about the necessary metrics we have to check for. As in AWS documentation, we have to measure performance:
1 – Between our application and our gateway
2 – Between our gateway and AWS
3 – Of Our buffer usage
4 – Of Our cache usage
Let’s see them by examples;
1 – Measure performance between our application and our gateway:
Here we will check for the volume metrics since our application is residing on our volumes.
The metrics we should measure are:data throughput, data latency and operations per second
Data latency -> Check for read time and write time as Average
Data Throughput -> Check for read bytes and write bytes as Sum. Divide by 300 to get per second
IOPS -> Check for read bytes and write bytes as Samples. Divide by 300 to get per second.
As you know I use storage gateway as Veeam backup destination. Let’s see some of the latest backup metrics. My volume’s ID is vol-F2B35E9B.
Data Throughput -> Check for read bytes and write bytes as Sum. Divide by 300 to get per second. Here 1234483200 bytes for 5 minutes period. Whe we divide it to 300, 4114944 bytes per second ,means 3.92 megabytes per second.
2 – Measure performance between our gateway and AWS:
Here we will check for Storage metrics. The same metric will be measure here. We have also additional metrics to be measured below:
Throughput to AWS -> Check for Cloudbytesdownloaded and Cloudbytesuploaded as Sum. Divide by 300 to get per second.
Latency to AWS -> Check for Clouddownloadlatency as Average
Throughput to AWS -> Check for Cloudbytesdownloaded and Cloudbytesuploaded as Sum. Divide by 300 to get per second. As you see uploaded bytes are 752412644 bytes, divided to 300 is 2508042 bytes per second, which is 2,4 megabytes per second.
3 – Measure performance of the upload buffer
Here we will check for the gateway metrics and use the metrics below.
Uploadbufferpercentused as Average.
Uploadbufferused as Average
Uploadbufferfree as Average
Let’s see our buffer metrics while backup was running. As you see Uploadbufferusage was 3215521792 bytes per second which is approxiamtely 3Gigabytes per second ( this is the backuped file size on veeam )
4 – Measure performance of the cache usage
Here we can use both gateway and volume metrics. If you use gateway metrics , we measure the aggregate of all volumes configured for the gateway. The metrics we will use are:
Cachepercentused as Average
Totalcachesize as Average
Cachehitpercent as Average
Cachepercentdirty as Average ( Remember that this is the data not uploded to AWS )
Our totalcachesize is 166564200448 bytes and cachepercentused was 7,8 percent, means 12992007634 bytes.
(150 gb total cache size and 12 Gb iof the cache is used )
You can use this metrics and create Cloudwatch alarms depending on your applications and usage. In my next post I will explain storage gateway snapshot operations. If you have any question or comment, please feel free to write and don’t forget to share please.