search

spatie / laravel-google-cloud-storage

Google Cloud Storage filesystem driver for Laravel
https://spatie.be/open-source
MIT License
196 stars 51 forks source link

version 2.0.3 broke google auth #25

Closed veneliniliev closed 2 years ago

veneliniliev commented 2 years ago

When I use the latest version of the package (2.0.3) i have the following problem:

Google\Cloud\Core\Exception\BadRequestException
<?xml version='1.0' encoding='UTF-8'?><Error><Code>InvalidAuthentication</Code><Message>The provided authentication header is invalid.</Message><Details>Cannot use OAuth Authorization header with form POST.</Details></Error>

When reverting to 2.0.2... all work fine.

freekmurze commented 2 years ago

@hotrush can you take a look at this?

hotrush commented 2 years ago

@veneliniliev share please your config file (without secrets of course, just structure)

hotrush commented 2 years ago

Tested:

  1. with path to config file

        'gcs' => [
        'driver' => 'gcs',
        'key_file_path' => '/app/gcs.json',
        'key_file' => [],
        'projectId' => null,
        'bucket' => 'BUCKET',
        'pathPrefix' => '',
        'apiEndpoint' => null,
        'visibility' => 'private',
        'metadata' => ['cacheControl'=> 'public,max-age=86400'],
    ],

  2. with copied config file 

        'gcs' => [
        'driver' => 'gcs',
        'key_file_path' => null,
        'key_file' => [
            'type' => 'service_account',
            'project_id' => 'alien-device-123123',
            'private_key_id' => '12312',
            'private_key' => "-----BEGIN PRIVATE KEY-----...",
            'client_email' => '123.iam.gserviceaccount.com',
            'client_id' => '123',
            'auth_uri' => 'https://accounts.google.com/o/oauth2/auth',
            'token_uri' => 'https://oauth2.googleapis.com/token',
            'auth_provider_x509_cert_url' => 'https://www.googleapis.com/oauth2/v1/certs',
            'client_x509_cert_url' => 'https://www.googleapis.com/robot/v1/metadata/x...'
        ],
        'projectId' => null,
        'bucket' => 'BUCKET',
        'pathPrefix' => '',
        'apiEndpoint' => null,
        'visibility' => 'private',
        'metadata' => ['cacheControl'=> 'public,max-age=86400'],
    ],

  3. with exported env var

export GOOGLE_APPLICATION_CREDENTIALS=/app/gcs.json plus

        'gcs' => [
            'driver' => 'gcs',
            'key_file_path' => null,
            'key_file' => [],
            'projectId' => null,
            'bucket' => 'BUCKET',
            'pathPrefix' => '',
            'apiEndpoint' => null,
            'visibility' => 'private',
            'metadata' => ['cacheControl'=> 'public,max-age=86400'],
        ],

Works perfect to me...

veneliniliev commented 2 years ago

it also works well for me locally. the interesting thing is that when I built it on the docker image and put it in the kubernetes cluster then the problem happens. I checked many times all the settings in the config and in env.

if it can help ... I use media-library to upload pictures with gcs.

/vendor/google/cloud-core/src/RequestWrapper.php in Google\Cloud\Core\RequestWrapper::convertToGoogleException at line 368

/vendor/google/cloud-core/src/RequestWrapper.php in Google\Cloud\Core\RequestWrapper::send at line 207

/vendor/google/cloud-core/src/Upload/MultipartUploader.php in Google\Cloud\Core\Upload\MultipartUploader::upload at line 44

/vendor/google/cloud-storage/src/Bucket.php in Google\Cloud\Storage\Bucket::upload at line 295

/vendor/league/flysystem-google-cloud-storage/GoogleCloudStorageAdapter.php in League\Flysystem\GoogleCloudStorage\GoogleCloudStorageAdapter::createDirectory at line 226

/vendor/league/flysystem/src/Filesystem.php in League\Flysystem\Filesystem::createDirectory at line 96

/vendor/laravel/framework/src/Illuminate/Filesystem/FilesystemAdapter.php in Illuminate\Filesystem\FilesystemAdapter::makeDirectory at line 781

/vendor/spatie/laravel-medialibrary/src/MediaCollections/Filesystem.php in Spatie\MediaLibrary\MediaCollections\Filesystem::getMediaDirectory at line 317

/vendor/spatie/laravel-medialibrary/src/MediaCollections/Filesystem.php in Spatie\MediaLibrary\MediaCollections\Filesystem::copyToMediaLibrary at line 119

/vendor/spatie/laravel-medialibrary/src/MediaCollections/Filesystem.php in Spatie\MediaLibrary\MediaCollections\Filesystem::add at line 28

/vendor/spatie/laravel-medialibrary/src/MediaCollections/FileAdder.php in Spatie\MediaLibrary\MediaCollections\FileAdder::processMediaItem at line 428

/vendor/spatie/laravel-medialibrary/src/MediaCollections/FileAdder.php in Spatie\MediaLibrary\MediaCollections\FileAdder::attachMedia at line 410

/vendor/spatie/laravel-medialibrary/src/MediaCollections/FileAdder.php in Spatie\MediaLibrary\MediaCollections\FileAdder::toMediaCollection at line 331
hotrush commented 2 years ago

Might help if you tell us how you configure the package 😄

Do you use GKE and creds from automatically mounted GOOGLE_APPLICATION_CREDENTIALS?

veneliniliev commented 2 years ago

i don't use GOOGLE_APPLICATION_CREDENTIALS (I tested with them but still the same problem)

in filesystems

 'gcs' => [
            'driver' => 'gcs',
            'key_file_path' => resource_path(env('GOOGLE_CLOUD_KEY_FILE', null)), // optional: /path/to/service-account.json
            //'key_file' => [], // optional: Array of data that substitutes the .json file (see below)
            'project_id' => env('GOOGLE_CLOUD_PROJECT_ID', 'your-project-id'), // optional: is included in key file
            'bucket' => env('GOOGLE_CLOUD_STORAGE_BUCKET', 'your-bucket'),
            'path_prefix' => env('GOOGLE_CLOUD_STORAGE_PATH_PREFIX', ''), // optional: /default/path/to/apply/in/bucket
            'apiEndpoint' => env('GOOGLE_CLOUD_STORAGE_API_URI', null), // see: Public URLs below
            'visibility' => 'public', // optional: public|private
            'metadata' => ['cacheControl' => 'public,max-age=86400'], // optional: default metadata
        ],

in .env

GOOGLE_CLOUD_PROJECT_ID=xxx
GOOGLE_CLOUD_KEY_FILE="xxx/service-account.json"
GOOGLE_CLOUD_STORAGE_BUCKET=xxx
GOOGLE_CLOUD_STORAGE_API_URI="https://xxxx/"

env loaded correctly in pod

# env | grep 'GOOGLE'
GOOGLE_CLOUD_STORAGE_API_URI="https://xxxx/"
GOOGLE_CLOUD_STORAGE_BUCKET=xxxx
GOOGLE_CLOUD_PROJECT_ID=xxxx
GOOGLE_CLOUD_KEY_FILE="xxxx/service-account.json"

the configuration is the same as in 2.0.2 where it works without a problem.

hotrush commented 2 years ago

Changes i made fixes passing apiEndpoint into the client, so in your case i think it makes sense to remove GOOGLE_CLOUD_STORAGE_API_URI="https://xxxx/". It doesn't make sense to override it usually, only when you use some side gcs-compatible storage (and before my changes it wasn't passed at all 😄 ).

If won't help - some questions:

hotrush commented 2 years ago

@veneliniliev

shawnlindstrom commented 2 years ago

@hotrush I can confirm removing the GOOGLE_CLOUD_STORAGE_API_URI from env resolved this issue for me.

veneliniliev commented 2 years ago

@hotrush I can confirm removing the GOOGLE_CLOUD_STORAGE_API_URI from env resolved this issue for me.

resolve the issue but if you use a custom domain for cloud storage?

without this parameter, the domain is https://storage.googleapis.com/... but I want to use my custom domain ex https://cdn.custom-domain.com/...

how to set my own domain?

hotrush commented 2 years ago

I'm pretty sure that custom domain doesn't host google API, but only proxies access to files. To hit API you still need to use storage.googleapis.com. And for example, when you generate a signed URL you need to pass bucket-bound hostname as a parameter to receive a link pointing to your domain, see - https://github.com/googleapis/google-cloud-php/blob/main/Storage/src/StorageObject.php#L820.

So, I think the problem with these two methods:

They use apiEndpoint from the config as a domain for URLs generated, when signedUrl method doesn't receive bucketBoundHostname at all. It makes sense to add a new config parameter for bucket-bound hostname option because bucket-bound hostname is not the same as API endpoint.

cc @freekmurze

mstaack commented 2 years ago

2.0.3 broken my setup too. issue is related to storageApiUrl config... 2.0.2 works perfectly for me

hotrush commented 2 years ago

Still need some attention from @freekmurze, as I wrote above - 2.0.3 fixed one issue and broke another feature, I think need to add one more config option and release v3 (?) because of the breaking change. @freekmurze confirm please and I'll make a pr

sergiofalcon commented 2 years ago

Same problem of @veneliniliev here, and with a custom domain as bucket name, too.

After several hours trying to fix it, it worked when i downgraded to 2.0.2, too.