Audit log streaming

HCP Packer supports near real-time streaming of audit events. Audit logs allow administrators to track user activity and enable security teams to ensure compliance in accordance with regulatory requirements.

The following sections outline the steps required to enable and configure audit logs streaming to supported providers. HCP Packer supports streaming audit logs to Datadog and Amazon CloudWatch. Note that you can only stream to one external account at a time.

Log retention

The HCP Packer platform stores audit logs for at least one year, and you can access logs for both active and deleted registries.

Amazon CloudWatch

1. From the HCP Packer Overview page, select the Audit Logs view.
2. Click Enable Log Streaming.
3. From the Enable audit logs streaming view, select Amazon CloudWatch as the provider.
4. From the Amazon CloudWatch as the provider page, keep AWS ID and External ID handy. You would need that in the next steps.
5. Create IAM Policy & Role
6. Under the provider, enter the Destination name, Role ARN (copied from the previous step), and select the AWS region where you are intend to store your data.
7. Click Test connection to receive a test event and confirm the connection has been established.
8. Click Save.

Logs should arrive within your Amazon CloudWatch in a few minutes of using Packer.

Refer to the AWS documentation for details on log exploration.

Datadog

1. From the HCP Packer Overview page, select the Audit Logs view.
2. Click Enable Log Streaming.
3. From the Enable audit logs streaming view, select Datadog as the provider.
4. Under the provider, enter your Destination name, API Key and select the Datadog site region that matches your existing Datadog environment.
5. Click Test connection to receive a test event and confirm the connection has been established.
6. Click Save.

Logs should arrive within your Datadog environment in a few minutes of using Packer. Refer to the Datadog documentation for details on log exploration.

Testing streaming configuration

During the streaming configuration setup, you can test that your streaming configuration works within HCP. Testing verifies that your credentials are correct and that other parameters on the configuration plan work. To test, enter the necessary parameters for the logging provider you wish to test, then click Test connection button.

HCP sends a test message to the logging provider and shares the success or failure status on the Enable log streaming page.

You can also test any updated streaming configurations to ensure they still work as intended.

Updating streaming configuration

After configuring streaming, you could update your configuration for a variety of reasons. You may want to rotate a secret used for your logging provider, or switch logging providers altogether.

1. Select Edit streaming configuration from the Manage menu on the Audit logs page.
2. If you want to select a new provider, do so now.
3. Enter new parameters for the provider.
4. (Optional) Test the connection by clicking the Test connection.
5. Click Save.

AWS Setup

As a part of the Amazon CloudWatch setup, you need to create an IAM Policy and Role. Refer following section to do that per your preferred method.

Before starting these steps, you must have AWS ID and External ID from the HCP Packer Audit Logs page within your HCP Portal.

AWS Management Console

1. Create a new IAM policy using the AWS Management Console.

2. Choose the JSON option and copy and paste the policy below.

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "HCPLogStreaming",
               "Effect": "Allow",
               "Action": [
                   "logs:PutLogEvents",
                   "logs:DescribeLogStreams",
                   "logs:DescribeLogGroups",
                   "logs:CreateLogStream",
                   "logs:CreateLogGroup",
                   "logs:TagLogGroup"
               ],
               "Resource": "*"
           }
       ]
   }
   

   Copy

3. Finish the rest of the setup to create and save policy.

4. Create a new IAM role with the AWS Management Console.

5. Choose AWS account as your trusted entity type.

6. Select Another AWS account for the AWS account field.

7. Use AWS ID value from the HCP Packer Audit Logs page as your account ID.

8. From options, select Require external ID.

9. For External ID, use the External ID value from your HCP Packer Audit Logs page.

10. Finish the process of saving and creating your custom role.

11. Attach the policy that you created in the previous steps to this role.

12. Finish the role creation setup.

13. Copy the ARN of the role and go back to Amazon CloudWatch section to finish rest of the steps.

Terraform

data "aws_iam_policy_document" "allow_hcp_to_stream_logs" {
 statement {
   effect = "Allow"
   actions = [
     "logs:PutLogEvents",       # To write logs to cloudwatch
     "logs:DescribeLogStreams", # To get the latest sequence token of a log stream
     "logs:DescribeLogGroups",  # To check if a log group already exists
     "logs:CreateLogGroup",     # To create a new log group
     "logs:CreateLogStream"     # To create a new log stream
   ]
   resources = [
     "*"
   ]
 }
}

data "aws_iam_policy_document" "trust_policy" {
 statement {
   sid     = "HCPLogStreaming"
   effect  = "Allow"
   actions = ["sts:AssumeRole"]
   principals {
     identifiers = ["<AWS ID-generated-by-Hashicorp>"]
     type        = "AWS"
   }
   condition {
     test     = "StringEquals"
     variable = "sts:ExternalId"
     values = [
       "<External ID-generated-by-Hashicorp>"
     ]
   }
 }
}

resource "aws_iam_role" "role" {
 name               = "hcp-log-streaming"
 description        = "iam role that allows hcp to send logs to cloudwatch logs"
 assume_role_policy = data.aws_iam_policy_document.trust_policy.json
 inline_policy {
   name   = "inline-policy"
   policy = data.aws_iam_policy_document.allow_hcp_to_stream_logs.json
 }
}