Following the Linux install instructions here will lead to a signature error when performing ‘apt update’:

Failed to fetch https://d3nt0h4h6pmmc4.cloudfront.net/ubuntu/dists/jammy/InRelease  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 04B0588859EF5026

The reason for this is the key from Amazon is not in the binary format apt is expecting. Changing the key retrieval command from

wget -q -O - https://workspaces-client-linux-public-key.s3-us-west-2.amazonaws.com/ADB332E7.asc | sudo tee /etc/apt/trusted.gpg.d/amazon-workspaces-clients.gpg > /dev/null

To:

wget -q -O - https://workspaces-client-linux-public-key.s3-us-west-2.amazonaws.com/ADB332E7.asc | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/amazon-workspaces-clients.gpg

Writes the key in the expected format and allows apt to verify the source signatures and install the package correctly. Note that the 22.04 client only supports DCV, you will need to modify your Workspace to switch to DCV if it is provisioned as PCoIP.

TOKEN=`curl --no-progress-meter -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`
curl --no-progress-meter -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/identity-credentials/ec2/security-credentials/ec2-instance

The response should look something like this:

{
  "Code" : "Success",
  "LastUpdated" : "2025-02-21T12:13:35Z",
  "Type" : "AWS-HMAC",
  "AccessKeyId" : "ASIAZKEYKEYKEY",
  "SecretAccessKey" : "SECRETSECRETSECRET",
  "Token" : "AVERYLONGTOKENSTRING",
  "Expiration" : "2025-02-21T18:37:11Z"
}

To extract the credentials into environment variables for use with the AWS CLI:

CREDS=`curl --no-progress-meter -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/identity-credentials/ec2/security-credentials/ec2-instance`
export AWS_ACCESS_KEY_ID=`echo $CREDS | jq -r .AccessKeyId`
export AWS_SECRET_ACCESS_KEY=`echo $CREDS | jq -r .SecretAccessKey`
export AWS_SESSION_TOKEN=`echo $CREDS | jq -r .Token`

After which you can use the CLI as usual until the credentials expire and are rotated.

aws sts get-caller-identity
{
    "UserId": "112233445566:aws:ec2-instance:i-1122334455667788",
    "Account": "112233445566",
    "Arn": "arn:aws:sts::112233445566:assumed-role/aws:ec2-instance/i-1122334455667788"
}

Firstly an Athena query for finding unused VPC interface endpoints in cost and usage report data. This requires you to have cost and usage reporting enabled (you should already) and configured for querying in Athena, see this guide. The query searches for all VPC interface endpoints that have not accrued any data transfer charges during the query period. VPC endpoints incur both hourly and data transfer charges, if you are not transferring any data it is very likely the endpoint is not being used and can be deleted. This will save around $7 a month ($0.01 x 24hours x 30 days) in each availability zone configured for the unused endpoint (by default all AZs). Annual cost saving potential of around $260 for each (3 AZ) endpoint removed in each account.

select 
  line_item_usage_account_id, line_item_resource_id
from 
  cur
where 
  line_item_usage_type like '%VpcEndpoint-Hour%'
  and year='2021'
  and month in ('7','07')
  and line_item_resource_id not in (
    select distinct(line_item_resource_id)
    from cur
    where line_item_usage_type like '%VpcEndpoint-Byte%'
    and year='2021'
    and month in ('7','07')
  )
group by line_item_usage_account_id, line_item_resource_id

The second approach is only recommended for dev/test environments where availability zone fault tolerance is not a requirement. When you create a VPC interface endpoint an ENI is created in each subnet (availability zone) unless you explicitly deselect the subnets. This is a best practice for interface endpoints as it provides continued availability in the case of an AZ failure - as long as you are using the (default) regional interface endpoint DNS name with private DNS. When using private DNS the regional endpoint DNS name, ec2.eu-west-1.amazonaws.com for example, is resolved to the private IP of the VPC interface endpoint ENI rather than the public IP of the service. In the case where one AZ is not reachable the request will be routed to an interface endpoint ENI in a different AZ, allowing your client applications to continue functioning. The down side of this is that you will incur intra-region data transfer charges (of $0.01/GB) for sending requests to the ENI in a different AZ. The potential cost saving opportunity is to remove this availability mechanism to reduce the hourly interface endpoint charges.

As long as the data transferred to and from the interface endpoint is around 1GB/hour it is cheaper to configure the VPC interface endpoint to only be available in one (or two) availability zones. This makes sense for any API specific endpoints (EC2 for example) but may not make sense for high throughput data intensive endpoints (such as S3/Kinesis/Cloudwatch Logs). The cost and usage report data can once again be used to help identify interface endpoints where the data transfer is low enough to justify exposing the endpoint in only one availability zone. Note that the endpoint data charges are aggregated across all ENIs so the query will not correctly identify cases where traffic is not more-or-less equally balanced across AZs. The query assumes that the cost and usage report is configured for hourly granularity, the line_item_usage_amount would need to be modified for daily/monthly reports and would be less accurate. Annual cost saving potential would depend on data transfer and number of AZs provisioned but should be around $175 for API endpoints reduced from 3 AZs to 1 AZ.

select 
  line_item_usage_account_id, line_item_resource_id
from 
  cur
where 
  line_item_usage_type like '%VpcEndpoint-Byte%'
  and year='2021'
  and month in ('7','07')
  and line_item_usage_amount < 1.0
  and line_item_resource_id not in (
    select distinct(line_item_resource_id)
    from cur
    where line_item_usage_type like '%VpcEndpoint-Byte%'
    and year='2021'
    and month in ('7','07')
    and line_item_usage_amount > 1.0
  )
group by line_item_usage_account_id, line_item_resource_id

aws ssm get-parameters-by-path \
--path /aws/service/global-infrastructure/services/[SERVICE]/regions/[REGION] \
--output text

As an example, checking if Athena is available in ap-northeast-3:

aws ssm get-parameters-by-path --path /aws/service/global-infrastructure/services/athena/regions/ap-northeast-3 --output text

An empty result indicates the service is not available. The list of service names can be retrieved as below (from the documentation linked above):

aws ssm get-parameters-by-path --path /aws/service/global-infrastructure/services --query 'Parameters[].Name | sort(@)'

Similarly the list of regions can be found using:

aws ssm get-parameters-by-path --path /aws/service/global-infrastructure/regions --query 'Parameters[].Name'

Earlier this year the S3 team announced that S3 will stop accepting API requests signed using AWS Signature Version 2 after June 24th, 2019. Customers will need to update their SDKs, CLIs, and custom implementations to make use of AWS Signature Version 4 to avoid impact after this date. It might be difficult to find older applications or instances using outdated versions of the AWS CLI or SDKs that need to be updated, the purpose of this post is to explain how AWS CloudTrail data events and Amazon Athena can be used to help identify applications that may need to be updated. We will cover the setup of the CloudTrail data events, the Athena table creation, and some Athena queries to filter and refine the results to help with this process.

Update (January/February 2019)

Setting up CloudTrail data events in the AWS console

Setting up CloudTrail data events using the AWS CLI

A word on cost

Creating the Athena table

Analysing the data events with Athena

Tracing the source of the requests