Hi everyone,

For a project, I needed to set my CloudWatch logs retention policies for more than 60 log groups. By default, AWS keep logs indefinitely and first 5GB per month of logs storage is free. If you don’t need to keep older logs and predict your logs will exceed 5GB, you should enable retention policy for your log groups ( it’s very cheap by the way, $0.0324 per GB archived per month).

If you have lots of groups like me and don’t want to enable them using AWS console, you can use that single line aws cli command.

 

First of all, let’s describe our groups:

aws logs describe-log-groups --query 'logGroups[*].[logGroupName]' | grep / | sed 's/"//g'

And the output is:

/aws/apigateway/welcome
/aws/lambda/ASG
/aws/lambda/Check_Termination_Protection
/aws/lambda/Crawl
/aws/lambda/GameInfo
/aws/lambda/GetGameDetails
/aws/lambda/GetGameMetrics
/aws/lambda/JWT_Lambda
/aws/lambda/JWT_Wekanban
/aws/lambda/MainCrawl
/aws/lambda/SearchGames
/aws/lambda/ShowUserData
/aws/lambda/searchgame_test
API-Gateway-Execution-Logs_4xygwbh33e/test1
API-Gateway-Execution-Logs_ncccz4j0ll/Demo

Now let’s see our retention policies for each log group:

while read i; do (printf $i": " && aws logs describe-log-groups --log-group-name-prefix $i --query 'logGroups[0].retentionInDays'); done < <(aws logs describe-log-groups --query 'logGroups[*].[logGroupName]' | grep / | sed 's/"//g')

We can see them all of them are set for 14 days:
/aws/apigateway/welcome: 14
/aws/lambda/ASG: 14
/aws/lambda/Check_Termination_Protection: 14
/aws/lambda/Crawl: 14
/aws/lambda/GameInfo: 14
/aws/lambda/GetGameDetails: 14
/aws/lambda/GetGameMetrics: 14
/aws/lambda/JWT_Lambda: 14
/aws/lambda/JWT_Wekanban: 14
/aws/lambda/MainCrawl: 14
/aws/lambda/SearchGames: 14
/aws/lambda/ShowUserData: 14
/aws/lambda/searchgame_test: 14
API-Gateway-Execution-Logs_4xygwbh33e/test1: 14
API-Gateway-Execution-Logs_ncccz4j0ll/Demo: 14

Finally, let’s set them to 3 days:

while read i; do (aws logs put-retention-policy --log-group-name $i --retention-in-days 3 ); done < <(aws logs describe-log-groups --query 'logGroups[*].[logGroupName]' | grep / | sed 's/"//g')

See you on next post…

Onur SALK

AWS Cloud & DevOps Consultant, AWS Certified Solutions Architect, AWS Community Hero

More Posts - Website

Follow Me:
TwitterFacebookLinkedIn