There’s yet another Python library for Google Cloud Storage library out there, mine ;-). It’s called gcs-client and it’s a somewhat mix of Google’s Api Python Client (without the discovery) and Google’s App Engine GCS Client.
A couple of weeks ago I wanted to play with Google’s Cloud Storage using Python and found that none of the big libraries fit my profile. On one had I had the Google Api Python Client which is really neat, but it is too generic, the upload functionality is somewhat limited when you have dynamic data and it’s not very pythonic not to mention that it doesn’t retry on 429 errors (too many requests) as Google documentation recommends, on the other had I had Google’s App Engine GCS Client which has everything I wanted but is meant to be used inside Google’s App Engine.
So I decided to create yet another library, this one specific to Google’s Cloud Storage and a little more pythonic. And while it is still on the early stages of development and missing a lot of features, it still has the most basic functionality:
- Creating buckets
- Deleting buckets
- Listing buckets in a project
- Getting default bucket for a project
- Getting bucket attributes
- Listing objects in a bucket
- Getting objects attributes
- Deleting objects
- Reading object contents
- Writing an object
- Configurable retries with Truncated Exponential Backoff
The library can be easily installed with from pypi:
pip install gcs-client.
Here’s an example of the library usage that prints the buckets of a project called ‘project_name’ using ‘private_key.json’ credentials, display some info on one of the buckets and list the objects inside that bucket (or creates one if it’s empty).
import gcs_client credentials = gcs_client.Credentials('private_key.json') project = gcs_client.Project('project_name', credentials) # Print buckets in the project buckets = project.list() print 'Buckets:\n\t- ','\n\t- '.join(map(str, buckets)) # Print some information from first bucket bucket = buckets print 'Bucket %s is located in %s with storage class %s' % (bucket, bucket.location, bucket.storageClass) # List the objects in the bucket objects = bucket.list() if not objects: print 'There are no objects, creating one' filename = '/tmp/my_file.txt' with bucket.open(filename, 'w') as f: f.write('this is a test file\n' * 100) objects = [gcs_client.Object(bucket.name, filename, credentials=credentials)] if objects: print '\t','\n\t'.join(map(lambda o: o.name + ' has %s bytes' % o.size, objects)) # Read the contents from the first file with objects.open() as obj: print 'Contents of file %s are:\n' % obj.name, obj.read() else: print 'There are no objects, nothing to do'