Enable VPC Flow Logs

This step will create the VPC and all components using the example CloudFormation template.

1. Download the latest version of the CloudFormation template here: vpc-alb-app-db.yaml
2. Sign in to the AWS Management Console, select your preferred region, and open the CloudFormation console at https://console.aws.amazon.com/cloudformation/.
3. Click Create Stack, then With new resources (standard).

4. Click Upload a template file and then click Choose file.

5. Choose the CloudFormation template you downloaded in step 1, return to the CloudFormation console page and click Next.
6. Enter the following details:
   • Stack name: The name of this stack. For this lab, use WebApp1-VPC and match the case.
   • Parameters: Parameters may be left as defaults, you can find out more in the description for each.

7. At the bottom of the page click Next.
8. In this lab, we use tags, which are key-value pairs, that can help you identify your stacks. Enter Owner in the left column which is the key, and your email address in the right column which is the value. We will not use additional permissions or advanced options so click Next. For more information, see Setting AWS CloudFormation Stack Options.
9. Review the information for the stack. When you’re satisfied with the configuration, at the bottom of the page check I acknowledge that AWS CloudFormation might create IAM resources with custom names then click Create stack.

10. After a few minutes the final stack status should change from CREATE_IN_PROGRESS to CREATE_COMPLETE. You can click the refresh button to check on the current status. You have now created the VPC stack (well actually CloudFormation did it for you).

Wait until the VPC CloudFormation stack status is CREATE_COMPLETE, then continue. This will take about four minutes.

• Download the CloudFormation template: staticwebapp.yaml
  • You can right-click then choose Save link as; or you can right click and copy the link to use with wget

1. Go to the AWS CloudFormation console at https://console.aws.amazon.com/cloudformation and click Create Stack > With new resources

2. Leave Prepare template setting as-is

   • For Template source select Upload a template file
   • Click Choose file and supply the CloudFormation template you downloaded: staticwebapp.yaml

3. Click Next

4. For Stack name use CloudFormationLab

5. Parameters

   • Look over the Parameters and their default values.

   • Click Next

6. For Configure stack options we recommend configuring tags, which are key-value pairs, that can help you identify your stacks and the resources they create. For example, enter Owner in the left column which is the key, and your email address in the right column which is the value. We will not use additional permissions or advanced options so click Next. For more information, see Setting AWS CloudFormation Stack Options.

7. For Review

   • Review the contents of the page
   • At the bottom of the page, select I acknowledge that AWS CloudFormation might create IAM resources with custom names
   • Click Create stack

8. This will take you to the CloudFormation stack status page, showing the stack creation in progress.

   • Click on the Events tab
   • Scroll through the listing. It shows the activities performed by CloudFormation (newest events at top), such as starting to create a resource and then completing the resource creation.
   • Any errors encountered during the creation of the stack will be listed in this tab.

9. When it shows status CREATE_COMPLETE, then you are finished with this step.

Use aws cli or AWS CloudShell to run below command. This command will create Flow Log in parquet file format with hive-compatible s3 prefixes

1. Navigate to CloudShell from AWS Console from the account where your VPC is located. Note: Please make sure you have correct region selected.

2. Replace <VPC ID> with VPC id from your account. You can find the VPC ID in console

3. Replace <S3 ARN> with S3 buckets arn from central account. Please specify subfolder in case you are storing logs under it.

   e.g. arn:aws:s3:::my-flow-log-bucket/my-custom-flow-logs/

    aws ec2 create-flow-logs \
    --resource-type VPC \
    --resource-ids <VPC ID> \
    --traffic-type ALL \
    --log-destination-type s3 \
    --log-destination <S3 ARN> \
    --destination-options FileFormat=parquet,HiveCompatiblePartitions=True,PerHourPartition=false \
    --log-format '${account-id} ${action} ${az-id} ${bytes} ${dstaddr} ${dstport} ${end} ${flow-direction} ${instance-id} ${interface-id} ${log-status} ${packets} ${pkt-dst-aws-service} ${pkt-dstaddr} ${pkt-src-aws-service} ${pkt-srcaddr} ${protocol} ${region} ${srcaddr} ${srcport} ${start} ${sublocation-id} ${sublocation-type} ${subnet-id} ${tcp-flags} ${traffic-path} ${type} ${version} ${vpc-id}'
   

4. Once you finish replacing ID, ARN paste the command in CloudShell and run it. You will see below result with FlowLogIds, if it is successful.

1. Login to your central AWS account.

2. Run CloudFormation stack to enable VPC Flow Logs.

• Download CloudFormation Template:

  • vpc-flow-logs-custom.yaml

    • This CloudFormation template enables VPC Flow Logs in the account you run it. You will need to run it per VPC.

  • From AWS Console navigate to CloudFormation. Then click on Create stack

  • Create stack page:

    1. In Create stack page Specify template select Upload a template file.
    2. Then Choose File and upload the template vpc-flow-logs-custom.yaml (you have downloaded previously)
    3. Click Next

3. Provide name for the stack e.g., “vpc-flow-logs-stack” and values for the stack parameters and then click Next

   • TrafficType (ACCEPT, REJECT, ALL): Type of traffic you wish to record

     • ACCEPT — The recorded traffic was permitted by the security groups and network ACLs.
     • REJECT — The recorded traffic was not permitted by the security groups or network ACLs.
     • ALL - The recorded traffic that was permitted (ACCEPT) and was not permitted (REJECT) by the security groups or network ACLs.

   • VpcFlowLogsBucketName (Optional): S3 bucket name where VPC flow logs will be stored.

     • If you specify the bucket name then it is assumed that the bucket already exists. If you want to centralize the storage of the logs, then create the bucket before and specify the bucket name here. If you are enabling VPC Flow Logs in additional account then please make sure to modify S3 bucket’s policy from the central account to grant access to additional account and provide the name of the central bucket to this parameter.

     • If you leave it blank CloudFormation template will create a bucket for you.

     Note: VpcFlowLogsBucketName - This bucket will be used to gather vpc flow logs for all of your vpcs from one or more accounts. So please make sure this is the central account where you want your VPC flow logs to be collected and QuickSight dashboard to be hosted.

   • VpcFlowLogsFilePrefix (Optional): VPC Flow logfile prefix in S3 bucket. See bold text in below example

     e.g., bucket_name/vpc-flow-logs/AWSLogs/aws_account_id/vpcflowlogs/region/year/month/day/

   • VpcId: You can find the VPC ID in console

   

4. In Configure stack options page, add below tags and click on Next

   • Name=VPCFlowLogs-CFN
   • Purpose=WALabVPCFlowLogs

5. On Review screen verify the inputs you have provided

6. Last click on Create stack

7. As shown below you will see progress of the stack creation under Events tab. Please wait for the stack to complete the execution. Once complete it will show the status CREATE_COMPLETE in green then proceed to the next step.