hk21702 / YA-GCal-Notion-Sync-Script

Script to synchronize between a Notion database and Google Calendar both ways. Uses Google App Script.
GNU General Public License v3.0
142 stars 11 forks source link

Service invoked too many times for one day: premium urlfetch #51

Closed emisaghi closed 1 year ago

emisaghi commented 1 year ago

I'm getting the "Service invoked too many times for one day: premium urlfetch" error. I initially put my trigger time to every 1 minute and I thought maybe that was the reason for the error. It sounds like I have a 20,000 per day quota though, so not sure how I reached that unless it included every event on my calendar in that (as it does urlfetch for every event)? I changed it to every 15 minutes now, hoping that might fix it, but I'm just wondering what is counted under urlfetch and whether there is a way to limit that in the script code without Google throwing that error.

hk21702 commented 1 year ago

URLFetch is called pretty much every time the script talks to Notion so generally you might see the rate limit being hit after you've just set up the script, or you ran a full sync. Talking to GCal is done directly using it's API.

During normal operation, when a script trigger that doesn't result in any events being updated is ran, URLfetch is not called that much. It should be some small O(1) constant number of calls. When things are being changed, it's some O(n) linear amount scaling with the number of events changed.

Since you just setup the script, and you probably ran it in full sync mode a bunch of times because of the descriptions stuff, you prob used up most of your limit already. The only other time where URLfetch will be called a ton is if you are for some reason changing or adding a bunch of events.

I personally use a 15 min timer and a calendar change trigger, but you can experiment with having a faster frequency. Just wait a day or two after setup to increase the frequency since the initial import of events can be rather expensive.

There isn't anything in the code that counts the number of fetches. If you'd like to, you can add a few quick line that just counts the number of times requests are made using URLFetch or the batch version of that. Then just print the count to console at the end of the script.

emisaghi commented 1 year ago

Thanks for the prompt reply. I think the URLFetch count would be a nice addition. Do I just add a variable and add to it every time a line calls UrlFetchApp.fetch(url, options)?

On that note, I also seem to have some duplicate events on Notion, but not on Google Calendar. I wonder what might have caused that? (it's not universal, only some events are affected and I can't find a pattern between them)

emisaghi commented 1 year ago

One more thing I noticed that might help people like me who have almost everything scheduled on their Google Calendar is to have a certain limit on how many events are being synced. I changed the future and past limit to 90 and 7 days, respectively, and I finally had a successful execution.

ghost commented 1 year ago

Hey! @emisaghi how would one go about reducing the future and past limit to 90 and 7 days, as you did? I think I have the same issue as you, but I'm not savvy enough to locate the change in the code itself. Thanks!

emisaghi commented 1 year ago

You could change it at the top of the main code:

Changing const RELATIVE_MAX_DAY = 1825; // 5 years const RELATIVE_MIN_DAY = 30;

To const RELATIVE_MAX_DAY = 90; // sync until 3 months in the future const RELATIVE_MIN_DAY = 7; // sync since a week in the past

worked for me.

ghost commented 1 year ago

const RELATIVE_MAX_DAY = 90; // sync until 3 months in the future const RELATIVE_MIN_DAY = 7; // sync since a week in the past

Thank you very much! @emisaghi 👍 The 5 year limit was making fullSync too slow (I have too many events on Gcal) and my calendars, other than the primary, weren't loading correctly. Your solution worked alright. I hope anyone with the same issue sees this, the script itself works great.