AWS S3 Bucket

I have worked in a Chatbot product Lily Health that helps women through facebook chatting. I need to implement a new feature that can receive attachment from facebook and display it. When a facebook user sends any kinds of attachment, facebook first uploads the attachement in their fbcdn then returns the attachment url link in webhooks. Facebook provides the attachment in webhook post data as json format is given below:

{
   'entry':[
      {
         'messaging':[
            {
               'timestamp':1569844017208,
               'message':{
                  'mid':'RiEeqOXoTsxQUYdACamMx1FsMuxYuZpLd2Ti_QQepc4VjRjf1jL9L-ff3G8RwWOR14MTS94Ckmp12mh0pQXg2Q',
                  'attachments':[
                     {
                        'payload':{
                           'url':'https://scontent.xx.fbcdn.net/v/t1.15752-9/71384073_2440261556255816_5718908083162316800_n.png?_nc_cat=101&_nc_oc=AQlqBPPDfOFmpQTGj5-zAqI_LXbeGGk1WPztdaNjQeIisgqHUL5A8aFhRzJPgU37tmfpvFogpa763U-5am9A_4tL&_nc_ad=z-m&_nc_cid=0&_nc_zor=9&_nc_ht=scontent.xx&oh=c40e387ab65015190feb4bce4f03b7f1&oe=5E29EC37'
                        },
                        'type':'image'
                     },
                     {
                        'payload':{
                           'url':'https://scontent.xx.fbcdn.net/v/t1.15752-9/71818349_482421819282412_5182082100815200256_n.png?_nc_cat=111&_nc_oc=AQn7peXAPEqJ5fivE0W3jnwhfcrkeuC44XSVw6yg84w_ipClORs9IJwgd4Eme0MsFgQZb0_XNCT3H4AkJlajjOwm&_nc_ad=z-m&_nc_cid=0&_nc_zor=9&_nc_ht=scontent.xx&oh=c772cac3fb42df19406e5ab712441819&oe=5DF13B7E'
                        },
                        'type':'image'
                     }
                  ]
               },
               'sender':{
                  'id':'2369507619774719'
               },
               'recipient':{
                  'id':'2100837706646730'
               }
            }
         ],
         'id':'2100837706646730',
         'time':1569844017767
      }
   ],
   'object':'page'
}

Facebook Messenger Platform supports the following four attachment types, specified in the attachment.type property of the message:

I have planned to store the attachment in AWS S3 Bucket from the facebook attachments payload url and upload the attachment S3 bucket url in my database so that I can display it later. Here I have uploaded the attachment folder wise consider the attachment type.

So First I have to create a S3 bucket and you can follow this tutorial Create an Amazon S3 Bucket. After creating the bucket I need to public the bucket and you can follow my tutorial to make S3 Bucket Public. From S3 bucket we need the following attributes

  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_KEY
  • AWS_REGION
  • AWS_BUCKET_NAME
  • AWS_S3_ENDPOINT

I have written a python function upload_file_to_aws_s3 that takes two parameter one is url and another is file type and this function return upload attachment S3 Bucket url.

import boto3
import requests
import os
from urllib.parse import urlparse

AWS_ACCESS_KEY_ID = "AWS_ACCESS_KEY_ID"
AWS_SECRET_KEY = "AWS_SECRET_KEY"
AWS_REGION = "AWS_REGION"
AWS_BUCKET_NAME = "AWS_BUCKET_NAME"
AWS_S3_ENDPOINT = "AWS_S3_ENDPOINT"

def upload_file_to_aws_s3(url='', file_type='image'):
    file_url = ''
    # get the connection of AWS S3 Bucket
    s3 = boto3.resource(
        's3',
        aws_access_key_id = AWS_ACCESS_KEY_ID,
        aws_secret_access_key = AWS_SECRET_KEY,
        region_name = AWS_REGION
    )
    
    response = requests.get(url)
    if response.status_code==200:
        raw_data = response.content
        url_parser = urlparse(url)
        file_name = os.path.basename(url_parser.path)
        key = file_type + "/" + file_name

        try:
            # Write the raw data as byte in new file_name in the server
            with open(file_name, 'wb') as new_file:
                new_file.write(raw_data)
    
            # Open the server file as read mode and upload in AWS S3 Bucket.
            data = open(file_name, 'rb')
            s3.Bucket(AWS_BUCKET_NAME).put_object(Key=key, Body=data)
            data.close()
            
            # Format the return URL of upload file in S3 Bucjet
            file_url = 'https://%s.%s/%s' % (AWS_BUCKET_NAME, AWS_S3_ENDPOINT, key)
        except Exception as e:
            print("Error in file upload %s." % (str(e)))
        
        finally:
            # Close and remove file from Server
            new_file.close()
            os.remove(file_name)
            print("Attachment Successfully save in S3 Bucket url %s " % (file_url))
    else:
        print("Cannot parse url")
    return file_url

If your AWS_BUCKET_NAME is mrk-lily-bucket, AWS_S3_ENDPOINT is s3-ap-southeast-1.amazonaws.com and key is image/71384073_2440261556255816_5718908083162316800_n.png then return url will be:

https://mrk-lily-bucket.s3-ap-southeast-1.amazonaws.com/image/71384073_2440261556255816_5718908083162316800_n.png.

After uploading different type of file in your S3 bucket you will get below folder wise upload structure in that bucket: AWS S3 Bucket File Upload

Some useful link that I have followed:

  • https://simpleisbetterthancomplex.com/tutorial/2017/08/01/how-to-setup-amazon-s3-in-a-django-project.html

Note: If you have any query or any upgradation of my writing or any mistakes please comment and suggest me. You are warmly welcomed always.