AWS S3 Signed URLs in Django

Amazon web services or AWS is one of the most popular cloud platforms that provide for server, storage, networking, remote computing, hosting and many more services based on the requirements. One of the most popular storage services of AWS is the Simple Storage Service commonly known as S3. S3 is an object storage service that offers storage for a variety of applications with high scalability, data availability and performance.

Django is an open-source, high-level python web framework for rapid development and clean, expedient design. It is exceedingly scalable, robust and comes with batteries included meaning it has a lot of in-built functions to support rapid development.

Objective

Using Django and AWS S3, create custom signed URLs for the project’s files stored in the S3 bucket so that it is not publicly accessible and files are accessible for a brief duration like one hour.

Create AWS S3 bucket and IAM User

We need to create a private S3 bucket in AWS so that all the data inside the bucket cannot be accessible using public S3 URLs like https://your-bucket-name.s3.amazonaws.com/media/file.png.

Steps to create a private bucket:

  1. Go to AWS management console
  2. From Services > Storage > S3, click on Create bucket button.
  3. Enter “your-bucket-name”, select your preferred AWS region for creating the bucket as it will be created in that region like the Asia Pacific (Mumbai) ap-south-1.
  4. In the section “Block Public Access settings for this bucket” check to Block all public access as we do not want the bucket to allow any public access.

5. Leave all other settings to default, then click on Create bucket.

Once the bucket is created we need to create an IAM user who has the access to this S3 bucket and we can use the keys in our Django application for interacting with the S3 bucket.

Steps to create an IAM user:

  1. Select IAM from Services, navigate to Users appearing inside Access management on the left-hand side.
  2. Click on Add users, provide a username and select programmatic access then click next.

3. If you already have a user group that has AmazonS3FullAccess attached policy, add the user to that user group or else create a new user group, provide the group name and attach AmazonS3FullAccess as the policy for the user group, click on Create group.

4. After creating a user group the group will be automatically selected for the IAM user we created in Step 2. Click on next until Create user option appears then click on Create user.

5. Download the CSV file which contains Access Key ID and Secret Access Key.

Now that we have created our private S3 bucket and have the AWS credentials, let’s move to the Django application.

Django Application

Let’s start with the settings configuration. We assume that the initial Django project is already done. We need to install two packages boto3 and django-storages.

pip install boto3
pip install django-storages

After both these packages are successfully installed, add the django-storages to the INSTALLED_APPS variable in settings.py.

INSTALLED_APPS = [..."storages",...]

Add the AWS configurations for interacting with the AWS S3 bucket we created above.

settings.py

Add the AWS access key, AWS secret access key, S3 bucket name, S3 bucket region, your Django project’s name toYOUR_AWS_ACCESS_KEY_ID , YOUR_AWS_SECRET_ACCESS_KEY , YOUR_AWS_STORAGE_BUCKET_NAME , YOUR_AWS_S3_REGION_NAME and your_django_project_name respectively.

Here AWS_QUERYSTRING_EXPIRE is the time in seconds for which the URL will be accessible, the time is set to 3600 seconds i.e. 1 hour, change this according to the requirements. The max time for the URL to be accessible is 7 days.

Now we have to define custom storage for our application. Create a file storage_backends.py inside the same directory where settings.py exits.

| your_django_project_name
| - |- ...
|- settings.py
|- storage_backends.py
|- ...

storage_backends.py

We have created two storage classes public, private for handling public and private files so that if there is a need for public files we can use PublicMediaStorage class to handle it. We will be using the PrivateMediaStorage class for handling the private media files which is the main objective of this application.

Now add this private storage class to the Django model through which we are storing files.

models.py

The FileField will use PrivateMediaStorage as the default storage for this model, the files stored using this model will have a signature attached to the URL so that it is restricted and available for a brief amount of time.

The URL is constructed using various parameters:

  • X-AMZ-Algorithm
  • X-AMZ-Credential
  • X-AMZ-Date
  • X-AMZ-Expires
  • X-AMZ-Signature
  • X-AMZ-SignedHeaders

All these parameters are automatically added to the URL generated while serving the file. These parameters make the URL private and accessible only for a brief amount of time.

After implementing all this logic, now the application is ready to serve signed URLs which will be accessible for one hour by default. Hooray!!

Signed URL

https://random-bucket.s3.amazonaws.com/private/file.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAZ6YLEAJKUYUJANFF%2F20210721%2Fap-south-1%2Fs3%2Faws4_request&X-Amz-Date=20210721T081020Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=2da3dd1f187aace29e6488855736060e988dbda6841a105c05661990964aeaa8

Public URL

https://random-bucket.s3.amazonaws.com/private/file.jpg

If anyone tries to access the public URL, it will not work, it will show a message Access Denied!

That’s it

This approach worked for me. I found most of the content through searching on Google, reading documentations and some trial and error. Hopefully, this will be useful to you.

References

  1. AWS documentation — https://docs.aws.amazon.com/AmazonS3/latest/userguide/ShareObjectPreSignedURL.html
  2. Django storages — https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html

Thank you for reading, hope it helps ✨

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store