Cloud Storage Authentication

[cclarkson]

When I try to authenticate for cloud storage on datalab using the code below, I get a URL to follow for an authentication code, but the URL leads to a page with this error message:

"Sign in with Google temporarily disabled for this app. This app has not been verified yet by Google in order to use Google Sign In."

I'm not sure if this is simply a permissions issue or something more sinister.

# Data storage, uses about 34 MB
output_cloud_zarr_path_template = 'vo_agam_production/ag3_data_paper/{}.pca_umap_input_alleles.zarr'
# Writing the PCA data to the cloud will require the appropriate authentication and authorization.

import gcsfs
# UNCOMMENT THIS TO AUTHENTICATE. YOU ONLY NEED TO RUN THIS ONCE.
# After running this once, your authentication token should then be cached in `~/.gcs_tokens`
# Once you have authenticated, you should comment this out again to avoid re-authenticating.
gcs_browser = gcsfs.GCSFileSystem(project='malariagen-jupyterhub', token='browser')

# Use `cache_timeout=0` to prevent object list cache, to avoid recreating map for Zarr consolidated metadata
auth_fs = gcsfs.GCSFileSystem(project='malariagen-jupyterhub', token='cache', cache_timeout=0)

for sp in species_group:

    # Check that the output's Zarr metadata file is not already on the cloud.
    # We don't want to accidentally overwrite or delete existing data, which might have been used in downstream analysis.
    # We don't simply check for the existence of the Zarr file here (i.e. output_cloud_zarr_path),
    # We might want to re-run the first parts of this notebook again,
    # so the Zarr store might legitimately exist but be incomplete.
    # The Zarr store is not considered complete until the Zarr metadata file (.zmetadata) is present and correct.
    # The final part of this notebook includes steps to create the Zarr metadata file and then validate it.

    output_cloud_zarr_metadata_path = f'{output_cloud_zarr_path_template.format(sp)}/.zmetadata'
    print(f'Checking for {output_cloud_zarr_metadata_path}')
    assert not auth_fs.exists(output_cloud_zarr_metadata_path)