search

SCRT-HQ / PSGSuite

Powershell module for Google / G Suite API calls wrapped in handy functions. Authentication is established using a service account via P12 key to negate the consent popup and allow for greater handsoff automation capabilities
https://psgsuite.io/
Apache License 2.0
235 stars 67 forks source link

Add -Fields parameter to Get-GSDriveFileList #177

Closed WJurecki closed 5 years ago

WJurecki commented 5 years ago

I would like to be able to specify what fields to have returned to increate performance.

For example I know that in my code I am only ever going to use the parents, name, id, and folderColorRgb properties of the objects returned and therefore would like to increase performance by not having the remaining properties collected and returned.

I would like a -Fields parameter added that accepts property names.

Such as: Get-GSDriveFileList -Fields 'parents,name,id,folderColorRgb' or Get-GSDriveFileList -Fields 'parents','name','id','folderColorRgb'

This example would result in the value assigned to $request.Fields at line 116 being files(parents,name,id,folderColorRgb),kind,nextPageToken

API Explorer confirms this is the proper format for the $request.Fields

For compatibility with existing behavior you could simply have this new -Fields parameter default to * since files(*),kind,nextPageToken is a valid request that returns the current result set.

Performance Tips discusses this and shows proper formatting.

scrthq commented 5 years ago

hey @WJurecki - This was actually brought up in an issue a while ago, and it appears that the difference between the .NET SDK when leaving the default value for Fields (or even wide open as *) on a request vs specifying a subset of Fields is negligible: https://github.com/scrthq/PSGSuite/issues/63#issuecomment-401243201

I'm adding it in, but forewarning that it doesn't add a performance gain as you'd expect. Here is my testing after adding, you can see that getting a subset of fields performs roughly the same as getting all of them.

image

Note that the previous setup was defaulting to 'files,kind,nextPageToken', so I've retained that as the default value for the parameter.

scrthq commented 5 years ago

Hey @WJurecki - v2.26.2 is out now with Fields support for Get-GSDriveFileList. Please try it out and let me know if you're having any issues! Thanks for your patience!

WJurecki commented 5 years ago

Hi @scrthq, thanks for adding the -Fields option.

It appears these is an issue with a logic of this new feature.

When executing Get-GSDriveFileList without the -Fields option the returned objects are different in version 2.26.2 than it was in 2.26.1. This is because $request.Fields is only set when -Fields is specified and therefore never being set with the default of 'files,kind,nextPageToken' and results in the google default field set.

This further shows that the issue with Get-GSDriveFileList limits pagesize to 100 #175 must be a vendor limit of the returned result size because when specifying only a few fields pagesize works up to 1,000.

scrthq commented 5 years ago

Good catch @WJurecki ! See my note here on that PR where that code bit changed. Very accurate analysis.

Checking out the difference in behavior now and will get it resolved asap if impacting!

scrthq commented 5 years ago

Confirmed the difference, getting a fix out in a few minutes 💪 thanks again!

scrthq commented 5 years ago

Fixed and deployed in v2.26.4!