Setting Up GOOGLE_APPLICATION_CREDENTIALS for Python in GCP
Written on
Configuring Application Default Credentials
In this guide, we will explore how to set up the GOOGLE_APPLICATION_CREDENTIALS for Python when working with Google Cloud Platform (GCP). Proper authentication is crucial for programmatic interaction with GCP services like Google BigQuery.
When this step is overlooked, you may encounter the error:
oauth2client.client.ApplicationDefaultCredentialsError: The Application Default Credentials are not available.
Subscribe to Data Pipeline, your go-to newsletter for Data Engineering insights.
How Application Default Credentials Operate
Application Default Credentials (ADC) provide a method for GCP to infer credentials based on the application environment. This feature allows your application code to operate across various environments without needing to alter the authentication method for GCP services or APIs.
For local development, there are primarily two methods to provide credentials to ADC:
- User Credentials
- Service Account Keys
Creating the JSON Credentials File
To generate the JSON file containing the required credentials, ensure you have the gcloud CLI installed on your machine.
For local development, the recommended approach is to use user credentials linked to your personal Google Cloud account. Execute the following command, which will prompt you to log in via your default web browser:
gcloud auth application-default login
After logging in, your credentials will be saved in a JSON file located in the following default directories:
- Mac/Linux: $HOME/.config/gcloud/application_default_credentials.json
- Windows: %APPDATA%gcloudapplication_default_credentials.json
Alternatively, if you prefer using a Service Account, you can generate the JSON token by accessing the Service Account service on GCP. However, be cautious, as service account keys can pose security risks. More secure methods include impersonation and Workload Identity Pool.
Setting the GOOGLE_APPLICATION_CREDENTIALS Environment Variable
To specify the credentials JSON file location, you need to set the GOOGLE_APPLICATION_CREDENTIALS environment variable. In Python, you can do this programmatically with the following code snippet:
import os
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = '$HOME/.config/gcloud/application_default_credentials.json'
Alternatively, you can create an instance of google.oauth2.service_account.Credentials and pass it to the Google client. Here’s an example demonstrating how to authenticate the Gmail Client in Python:
from google.oauth2 import service_account
from googleapiclient.discovery import build
credentials = service_account.Credentials.from_service_account_file(
'$HOME/.config/gcloud/application_default_credentials.json'
)
service = build('gmail', 'v1', credentials=credentials)
Note that these code snippets assume your JSON credentials file is in the default directory created by gcloud. Adjust the path accordingly if it differs.
Conclusion
This tutorial provided a comprehensive overview of setting up Application Default Credentials (ADC) for Google Cloud and Python, ensuring proper authentication for seamless interaction with GCP services. The ADC strategy allows for code execution across various environments without changing the authentication process.
We also discussed how to create the necessary JSON credentials file using either user credentials or a Service Account, and how to set the GOOGLE_APPLICATION_CREDENTIALS environment variable to define the file's location.
Subscribe to Data Pipeline for more insights on Data Engineering!