Getting started with GCP Storage Bucket with Python

Soumil Nitin Shah¶

Bachelor in Electronic Engineering | Masters in Electrical Engineering | Master in Computer Engineering |

• Website : https://soumilshah.herokuapp.com
• Github: https://github.com/soumilshah1995
• Linkedin: https://www.linkedin.com/in/shah-soumil/
• Blog: https://soumilshah1995.blogspot.com/
• Youtube : https://www.youtube.com/channel/UC_eOodxvwS_H7x2uLQa-svw?view_as=subscriber
• Facebook Page : https://www.facebook.com/soumilshah1995/
• Email : shahsoumil519@gmail.com
• projects : https://soumilshah.herokuapp.com/project

Hello! I’m Soumil Nitin Shah, a Software and Hardware Developer based in New York City. I have completed by Bachelor in Electronic Engineering and my Double master’s in Computer and Electrical Engineering. I Develop Python Based Cross Platform Desktop Application , Webpages , Software, REST API, Database and much more I have more than 2 Years of Experience in Python

Welcome We will be Learning about GCP Buckets¶

In [9]:

# pip install --upgrade google-api-python-client
# pip install --upgrade google-cloud-storage
# https://cloud.google.com/storage/docs/reference/libraries?authuser=1#client-libraries-install-python

Make sure to download the Keys¶

In [136]:

try:
    from google.cloud import storage
    import google.cloud.storage
    import json
    import os
    import sys
except Exception as e:
    print("Error : {} ".format(e))

In [142]:

PATH = os.path.join(os.getcwd() , 'propane-calling-262419-9f249a3aa132.json')
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = PATH

• Create a Client Object

In [143]:

storage_client = storage.Client(PATH)

In [144]:

storage_client

Out[144]:

<google.cloud.storage.client.Client at 0x22e8de945e0>

Getting all Files from GCP Bucket¶

In [145]:

bucket = storage_client.get_bucket('myfirst-bucket-1995')

In [146]:

filename = [filename.name for filename in list(bucket.list_blobs(prefix='')) ]
filename

Out[146]:

['C:\\Users\\s.shah\\Desktop\\Personal Projects\\netflix_titles.csv',
 'netflix_titles.csv',
 'subscribe.webp']

Downloading a File from Bucket¶

In [147]:

blop = bucket.blob(blob_name = 'netflix_titles.csv').download_as_string()

In [148]:

with open ('doo.csv', "wb") as f:
        f.write(blop)

Pushing a file on GCP bucket¶

In [150]:

filename= 'doo.csv'

In [151]:

UPLOADFILE = os.path.join(os.getcwd(), filename)

In [152]:

bucket = storage_client.get_bucket('myfirst-bucket-1995')
blob = bucket.blob(filename)
blob.upload_from_filename(UPLOADFILE)

Reading CSV File Drieectly from GCP bucket¶

In [153]:

import pandas as pd 
import io
from io import BytesIO

In [154]:

df = pd.read_csv(
    io.BytesIO(
                 bucket.blob(blob_name = 'netflix_titles.csv').download_as_string() 
              ) ,
                 encoding='UTF-8',
                 sep=',')

In [155]:

df.head(1)

Out[155]:

	show_id	type	title	director	cast	country	date_added	release_year	rating	duration	listed_in	description
0	81145628	Movie	Norm of the North: King Sized Adventure	Richard Finn, Tim Maltby	Alan Marriott, Andrew Toth, Brian Dobson, Cole...	United States, India, South Korea, China	September 9, 2019	2019	TV-PG	90 min	Children & Family Movies, Comedies	Before planning an awesome wedding for his gra...

Deleting a file from GCP bucket¶

In [156]:

filename = [filename.name for filename in list(bucket.list_blobs(prefix='')) ]
filename

Out[156]:

['C:\\Users\\s.shah\\Desktop\\Personal Projects\\netflix_titles.csv',
 'doo.csv',
 'netflix_titles.csv',
 'subscribe.webp']

In [157]:

DELETE_FILE = 'doo.csv'

In [158]:

bucket = storage_client.get_bucket('myfirst-bucket-1995')
blob = bucket.blob(DELETE_FILE)

In [159]:

blob.delete()

In [160]:

filename = [filename.name for filename in list(bucket.list_blobs(prefix='')) ]
filename

Out[160]:

['C:\\Users\\s.shah\\Desktop\\Personal Projects\\netflix_titles.csv',
 'netflix_titles.csv',
 'subscribe.webp']

In [ ]: