Cloud Storage

Google Cloud Storage Buckets

Google Cloud Storage (GCS) is Google’s object storage service and is commonly used by web applications, mobile applications and cloud infrastructure to store files and application data.

Just like AWS S3, the existence of a Google Cloud Storage bucket isn’t itself interesting from a security perspective. Many buckets intentionally contain publicly accessible application resources.

What we want to determine is whether the bucket exposes more information or permissions than intended.

This may include anonymous access to sensitive objects, bucket listing, excessive IAM permissions or storage locations that were never intended to be publicly discoverable.

Finding Bucket Names

The first step is normally discovering potential bucket names.

Bucket names may be based on:

1
Company name
2
Domain name
3
Application name
4
Project name
5
Environment
6
Firebase project

Common examples might look like:

1
example
2
example-prod
3
example-production
4
example-backups
5
example.appspot.com
6
example-app.appspot.com

Bucket names can appear in JavaScript, configuration files, mobile applications and API responses.

GCPBucketBrute

GCPBucketBrute from Rhino Security Labs can be used to enumerate Google Storage bucket names and test the access available to discovered buckets.

1
https://github.com/RhinoSecurityLabs/GCPBucketBrute

Navigate to the tool:

1
cd ~/Hacking/Pentest_Tools/GCPBucketBrute

A keyword can then be supplied:

1
sudo python3 gcpbucketbrute.py -k company_name -u

The keyword should generally be based on something associated with the target organisation, such as the company name, application name or domain.

GCPBucketBrute is useful because discovering that a bucket exists is only the first part of the problem. We also want to understand what level of access is available to it.

Checking a Bucket Manually

Google’s Cloud Storage JSON API can also be queried directly.

The following endpoint lists objects within a bucket where the caller has the required permission:

1
https://www.googleapis.com/storage/v1/b/<BUCKET NAME>/o

This gives us a very quick way of checking a discovered bucket.

For example:

1
https://www.googleapis.com/storage/v1/b/example.appspot.com/o

The response itself can provide useful information.

In my testing I generally interpret the responses as:

1
404    Bucket does not exist
2

3
401/403
4
       Bucket exists, but anonymous access to the requested
5
       operation is not permitted
6

7
Object listing
8
       Bucket exists and the current caller is able to list objects

The exact status code can vary depending on the request and Google’s access controls, so the important distinction is whether the bucket exists and whether the requested operation is permitted.

Google documents storage.objects.list as the permission required to list objects using this API.

Listing Bucket Contents

If object listing is permitted, the API response will contain information about the objects stored within the bucket.

This immediately gives us considerably more information to work with.

Interesting filenames may include:

1
backup
2
config
3
database
4
export
5
credentials
6
users
7
production
8
private
9
secret

We can then investigate individual objects to determine whether sensitive information has actually been exposed.

Remember that the ability to list objects and the ability to retrieve an object are separate permissions. A bucket may expose one without necessarily exposing the other. Google documents these as storage.objects.list and storage.objects.get respectively.

Using gsutil

Google’s gsutil utility can also be used to interact with Cloud Storage.

Documentation:

1
https://cloud.google.com/storage/docs/gsutil_install

To attempt to list the contents of a bucket:

1
gsutil ls gs://example.appspot.com

This normally uses the currently authenticated Google identity, but it is also useful when determining whether the bucket permits the requested access without appropriate authorisation.

If access is granted, recursively listing the contents can provide a better picture of what information is stored within the bucket.

Modern Google Cloud environments may also use the equivalent gcloud storage commands:

1
gcloud storage ls gs://example.appspot.com

Google currently documents gcloud storage ls as the command-line method for listing objects.

Retrieving the Bucket IAM Policy

If our current identity has sufficient permissions, the bucket’s IAM policy can be retrieved through the API:

1
https://www.googleapis.com/storage/v1/b/<BUCKET NAME>/iam

For example:

1
https://www.googleapis.com/storage/v1/b/example.appspot.com/iam

The policy can reveal which users, groups and service accounts have access to the bucket.

Pay particular attention to principals such as:

1
allUsers
2
allAuthenticatedUsers

allUsers represents anyone on the Internet, while allAuthenticatedUsers can provide access far beyond the organisation.

For example, granting allUsers the roles/storage.objectViewer role can make the bucket’s objects publicly readable and listable.

As with AWS, however, the complete configuration needs to be considered rather than treating a single IAM entry as the entire security picture.

Bucket ACLs

Older Google Cloud Storage configurations may also use ACLs.

Where the caller has sufficient permission, bucket ACL information can be retrieved through:

1
https://storage.googleapis.com/storage/v1/b/<BUCKET>/acl

Modern environments may use Uniform bucket-level access, in which case ACL operations are disabled and access is controlled through IAM instead.

This distinction is useful when testing because an ACL request failing does not necessarily mean that permissions cannot be assessed—it may simply mean that the bucket is using IAM exclusively.

Finding Buckets in Android Applications

Mobile applications are a particularly useful source of Google Cloud Storage bucket names.

Firebase and Google Cloud configuration values frequently survive application compilation and can therefore be identified after decompiling an APK.

One quick method is to recursively search the decompiled application for:

1
google_storage_bucket

Using PowerShell:

1
Get-ChildItem -Path .\app\ -Recurse |
2
Select-String -Pattern "google_storage_bucket"

We may find something similar to:

1
"google_storage_bucket": "example-mobile-app-production.appspot.com"

This gives us a bucket name that can immediately be tested using the same techniques discussed above.

For example:

1
https://www.googleapis.com/storage/v1/b/example-mobile-app-production.appspot.com/o

This is one reason mobile application testing and cloud enumeration often overlap. Information embedded in the client application can expose cloud resources that were not obvious from the organisation’s website.

Don’t Stop at Bucket Discovery

Finding the bucket is only the beginning.

Once a bucket has been identified, determine what the current user can actually do.

Questions worth asking include:

1
Can I determine that the bucket exists?
2

3
Can I list its objects?
4

5
Can I retrieve individual objects?
6

7
Can I upload objects?
8

9
Can I overwrite objects?
10

11
Can I delete objects?
12

13
Can I retrieve its IAM policy?
14

15
Can I read its ACLs?
16

17
Do I have access when unauthenticated?
18

19
Do I have additional access using my supplied account?

Different permissions have very different security implications.

A bucket containing intentionally public images is unlikely to be interesting.

A bucket containing private documents that can be downloaded anonymously is.

A bucket that allows an unauthenticated user to upload or overwrite application content could have significantly greater impact.

What Should We Look For?

Interesting Cloud Storage content may include:

1
Application configuration
2
Database backups
3
API keys
4
Service-account information
5
Credentials
6
Customer documents
7
Internal documents
8
Source code
9
Environment files
10
Logs
11
Exports
12
Mobile application data
13
Infrastructure configuration
14
Backups

Don’t report a bucket simply because it exists.

The important part is determining what information or capability is exposed and whether that exposure crosses the intended security boundary.

AWS S3 Buckets

Amazon Simple Storage Service (S3) is AWS’s object storage service. Applications use S3 buckets to store almost anything, including images, documents, backups, application data, log files and static website content.

From a penetration testing perspective, S3 becomes interesting when buckets or individual objects have been exposed unintentionally, permissions are too broad, or sensitive information has been uploaded to storage that was intended to be public.

A bucket being accessible from the Internet isn’t automatically a vulnerability. S3 is commonly used intentionally for public content. What we really want to establish is what an unauthenticated or low-privileged user can actually do with the bucket and what information is exposed.

Useful resources:

1
https://buckets.grayhatwarfare.com/
2

3
https://book.hacktricks.xyz/pentesting/pentesting-web/buckets/aws-s3#amazon-s3-buckets

Finding S3 Buckets

The first problem is normally identifying the bucket name.

Bucket names frequently contain information relating to the organisation, application or environment:

1
company
2
company-backups
3
company-prod
4
company-production
5
company-dev
6
company-assets
7
company-static
8
application-name
9
application-name-prod

References to S3 buckets can also appear throughout an application’s client-side resources and configuration.

When assessing a web application, search JavaScript, HTML, API responses, configuration files and other client-side resources for strings such as:

1
amazonaws.com
2
s3.amazonaws.com
3
s3-
4
.amazonaws.com

Mobile applications are another useful source because bucket names may be embedded in configuration files or application code.

We can also use search engines, certificate data, source-code repositories and other reconnaissance sources to identify references to an organisation’s storage infrastructure.

GrayHatWarfare

GrayHatWarfare maintains a searchable database of publicly exposed cloud storage buckets and files:

1
https://buckets.grayhatwarfare.com/

Searching for the organisation name, domain name, application names and other unique identifiers may identify buckets or files that would otherwise be difficult to discover.

This can be particularly useful during external reconnaissance where we do not initially know the organisation’s bucket naming convention.

Checking Whether a Bucket Exists

Once we have a possible bucket name, we can attempt to access it through S3.

For example:

1
https://BUCKET.s3.amazonaws.com/

Or:

1
https://s3.amazonaws.com/BUCKET/

The exact response can provide useful information about whether the bucket exists and whether anonymous access is permitted.

An access-denied response should not immediately be discarded. It may confirm that the bucket exists even though directory-style listing is not available.

We can then continue testing individual objects or assess the bucket using the AWS CLI.

Anonymous Bucket Listing

One of the first things worth checking is whether the bucket allows its contents to be listed without authentication.

Using the AWS CLI:

1
aws s3 ls s3://BUCKET --no-sign-request

--no-sign-request tells the AWS CLI not to use configured AWS credentials.

If the bucket permits anonymous listing, the returned object names can reveal considerably more information about the application or organisation.

For example:

1
backup/
2
database/
3
documents/
4
exports/
5
logs/
6
production/
7
users/

Even when the individual objects are not sensitive, filenames and directory structures can reveal internal application functionality and naming conventions.

Recursively Listing Objects

If anonymous listing is permitted, recursively enumerate the bucket:

1
aws s3 ls s3://BUCKET --recursive --no-sign-request

For a large bucket we may want to search the results rather than manually reviewing everything:

1
aws s3 ls s3://BUCKET --recursive --no-sign-request | grep -Ei 'backup|config|database|sql|password|secret|key|env|zip|tar|log'

Interesting extensions may include:

1
.env
2
.config
3
.json
4
.xml
5
.yml
6
.yaml
7
.sql
8
.db
9
.sqlite
10
.bak
11
.zip
12
.tar
13
.gz
14
.pem
15
.key
16
.pfx
17
.log

The objective isn’t simply to prove that listing is possible. We want to determine whether the exposed objects contain information that creates meaningful security impact.

Accessing Individual Objects

A bucket may prevent directory listing while still allowing individual objects to be downloaded.

This distinction is important.

Knowing:

1
https://BUCKET.s3.amazonaws.com/

does not necessarily tell us whether:

1
https://BUCKET.s3.amazonaws.com/backup.zip

is accessible.

If object names are discovered elsewhere in the application, test those objects directly.

Using the AWS CLI:

1
aws s3 cp s3://BUCKET/path/to/file.txt - --no-sign-request

Or with HTTP:

1
curl https://BUCKET.s3.amazonaws.com/path/to/file.txt

This means “I can’t list the bucket” does not necessarily mean “the bucket contains nothing publicly accessible.”

Downloading Public Bucket Content

If anonymous listing and object retrieval are both allowed, content can be copied for further analysis:

1
aws s3 cp s3://BUCKET ./bucket-data/ --recursive --no-sign-request

Only download information required for the assessment. Large storage buckets may contain significant amounts of data, so indiscriminately synchronising an entire bucket is usually unnecessary.

Authenticated Testing

During an AWS assessment we may have access to AWS credentials rather than testing anonymously.

First determine which identity we are currently using:

1
aws sts get-caller-identity

We can then test whether that identity can access the identified bucket:

1
aws s3 ls s3://BUCKET

Authenticated testing is particularly important because a bucket may correctly block anonymous users while granting excessive access to IAM users, roles or external AWS accounts.

The security question therefore becomes:

What can this particular AWS principal do with the bucket?

Bucket ACL

Where our current identity has permission, retrieve the bucket ACL:

1
aws s3api get-bucket-acl --bucket BUCKET

Review the returned grants and determine which principals have access.

Historically, S3 ACLs were a common source of unintended public access. Modern AWS environments increasingly rely on bucket policies and IAM instead, but ACLs are still worth reviewing where they are enabled.

Bucket Policy

Retrieve the bucket policy:

1
aws s3api get-bucket-policy --bucket BUCKET

Bucket policies can be particularly interesting because they may grant access to:

1
Everyone
2
Specific AWS accounts
3
IAM users
4
IAM roles
5
AWS services
6
External organisations

Pay particular attention to statements containing broad principals such as:

1
"Principal": "*"

However, the presence of "Principal": "*" does not automatically mean that the bucket is publicly exploitable. Conditions and S3 Block Public Access settings can significantly change the effective permissions.

The complete policy needs to be reviewed.

Is the Bucket Public?

AWS can report whether a bucket policy is considered public:

1
aws s3api get-bucket-policy-status --bucket BUCKET

We can also inspect the bucket’s public-access-block configuration:

1
aws s3api get-public-access-block --bucket BUCKET

S3 Block Public Access provides several protections designed to prevent public ACLs and bucket policies from exposing data.

AWS applies the most restrictive applicable configuration between bucket, account and organisation-level controls, so simply seeing a permissive bucket policy does not necessarily mean the policy is effective.

What Permissions Matter?

When assessing an S3 bucket, think in terms of actions rather than simply “public” or “private”.

We may want to determine whether our current identity can:

1
List objects
2
Read objects
3
Upload objects
4
Overwrite objects
5
Delete objects
6
Read the bucket policy
7
Read ACLs
8
Modify ACLs
9
Modify the bucket policy

These have very different security implications.

A publicly readable marketing image bucket may be completely intentional.

A publicly writable bucket is a very different situation.

Testing Upload Permissions

Where explicitly permitted by the engagement scope, test whether the current principal can upload an object using a harmless test file:

1
echo "pentest" > test.txt
2

3
aws s3 cp test.txt s3://BUCKET/test.txt

For anonymous testing:

1
aws s3 cp test.txt s3://BUCKET/test.txt --no-sign-request

If the upload succeeds, verify the object and remove it afterwards:

1
aws s3 rm s3://BUCKET/test.txt

Write access can have considerably greater impact than read access, particularly where the bucket hosts application resources or static website content.

Do not overwrite or modify existing objects merely to demonstrate write access.

What Should We Look For?

Interesting S3 exposures may include:

1
Database backups
2
Application backups
3
Environment files
4
Configuration files
5
Source code
6
API keys
7
Access keys
8
Private keys
9
Certificates
10
Customer documents
11
User uploads
12
Logs
13
Internal documentation
14
Infrastructure configuration
15
Terraform state
16
CI/CD artefacts

Files such as Terraform state can be particularly valuable because they may contain infrastructure information and sensitive values.

The impact of an exposed bucket should therefore be based on what is actually accessible, rather than simply reporting that an S3 bucket exists.

The interesting question is always:

What information or capability has been exposed to someone who shouldn’t have it?