shotgunsoftware / python-api

A Python-based library for accessing Flow Production Tracking API.
https://developer.shotgridsoftware.com/python-api
Other
308 stars 198 forks source link

SG-36677 Optimize payload by prevent unnecessary data #360

Open carlos-villavicencio-adsk opened 2 days ago

carlos-villavicencio-adsk commented 2 days ago

Summary

Remove unnecessary data in the payload when combining related queries before sending it to the server. This would improve overall performance decreasing network latency and server processing.

Affected methods

Detailed Description

The change should transform calls like:

sg.find('Asset', [['project', 'is', {'created_at': datetime.datetime(2015, 12, 16, 11, 2, 10, tzinfo), 'id': 72, 'name': 'Demo: Game', 'type': 'Project'}]]) 

as if the following had been called:

sg.find('Asset', [['project', 'is', {'id': 72, 'type': 'Project'}]]) 

This version also includes a new environmental variable to disable this feature. Please use it if you want to compare results or if you are getting any unexpected behavior.

export SHOTGUN_API_DISABLE_ENTITY_OPTIMIZATION=1

Testing

Install this v3.7.0-beta.1 version.

pip install git+https://github.com/shotgunsoftware/python-api.git@v3.7.0-beta.1

In my first tests, I added 40 fake tests to a project dict and with this feature applied, the result time decreased from 0.4285s to 0.2598s

# Getting project fields
project_fields = list(sg.schema_field_read("Project").keys())

# Getting a project
sg_full_project = sg.find_one('Project', [['id', 'is', PROJECT_ID]], project_fields)

# At this point, `sg_full_project` can be "hacked" with more unnecessary fields to simulate a larger payload.

# Getting an asset
sg_asset = sg.find('Asset', [['project', 'is', sg_full_project]])

# A/B test
os.environ["SHOTGUN_API_DISABLE_ENTITY_OPTIMIZATION"] = "1"
sg_asset_no_optimization = sg.find('Asset', [['project', 'is', sg_full_project]])

TODO