[performance] Look into reducing number of API calls during create

datopian / metastore-lib

🗄️ Library for storing dataset metadata, with versioning support and pluggable backends including GitHub.

https://tech.datopian.com/versioning/

MIT License

10 stars 1 forks source link

Open shevron opened 4 years ago

shevron commented 4 years ago

When calling create we are doing quite a lot of API calls (see analysis in #17). Look into ways of reducing the number of calls.

Perhaps, using GraphQL mutations (if supported by GitHub) can work to create a tree of objects in a single call.
See if we can avoid calling create_file (not likely) to create the base commit for the repo
See if we can avoid calling get_branch to get the sha of the tip of the main branch.

shevron commented 4 years ago

This applies to update as well as to create.