Changes to frequency of /updates endpoint

[SirSparkles]

Has there been a change arounf 28/12 that caused the /updates endpoint to have much more data returned?

Prior to that point you would only need 1-2 calls to catch up a few days worth of updates, since then each page of responses only adds 10 minutes of updates (approx)

2023-01-09 07:12:30|INFO| Obtained 500 responses from lastupdated query #1 - until 9/01/2023 1:44:15 AM (1673199855) 2023-01-09 07:12:30|INFO| Obtained 500 responses from lastupdated query #2 - until 9/01/2023 2:03:35 AM (1673201015) 2023-01-09 07:12:31|INFO| Obtained 500 responses from lastupdated query #3 - until 9/01/2023 2:31:05 AM (1673202665) 2023-01-09 07:12:32|INFO| Obtained 500 responses from lastupdated query #4 - until 9/01/2023 2:58:51 AM (1673204331) 2023-01-09 07:12:32|INFO| Obtained 500 responses from lastupdated query #5 - until 9/01/2023 3:21:30 AM (1673205690) 2023-01-09 07:12:33|INFO| Obtained 500 responses from lastupdated query #6 - until 9/01/2023 3:44:25 AM (1673207065) 2023-01-09 07:12:33|INFO| Obtained 500 responses from lastupdated query #7 - until 9/01/2023 3:57:25 AM (1673207845) 2023-01-09 07:12:34|INFO| Obtained 500 responses from lastupdated query #8 - until 9/01/2023 4:09:00 AM (1673208540) 2023-01-09 07:12:35|INFO| Obtained 500 responses from lastupdated query #9 - until 9/01/2023 4:16:40 AM (1673209000) 2023-01-09 07:12:36|INFO| Obtained 500 responses from lastupdated query #10 - until 9/01/2023 4:30:35 AM (1673209835) 2023-01-09 07:12:36|INFO| Obtained 500 responses from lastupdated query #11 - until 9/01/2023 4:43:13 AM (1673210593) 2023-01-09 07:12:37|INFO| Obtained 500 responses from lastupdated query #12 - until 9/01/2023 4:57:20 AM (1673211440) 2023-01-09 07:12:38|INFO| Obtained 500 responses from lastupdated query #13 - until 9/01/2023 5:16:14 AM (1673212574) 2023-01-09 07:12:38|INFO| Obtained 500 responses from lastupdated query #14 - until 9/01/2023 5:41:42 AM (1673214102) 2023-01-09 07:12:39|INFO| Obtained 500 responses from lastupdated query #15 - until 9/01/2023 6:08:13 AM (1673215693) 2023-01-09 07:12:39|INFO| Obtained 500 responses from lastupdated query #16 - until 9/01/2023 6:32:40 AM (1673217160) 2023-01-09 07:12:40|INFO| Obtained 500 responses from lastupdated query #17 - until 9/01/2023 6:59:37 AM (1673218777) 2023-01-09 07:12:41|INFO| Obtained 186 responses from lastupdated query #18 - until 9/01/2023 7:09:59 AM (1673219399)

[szsori]

There were some data issues we needed to resolve, which required that I inject a number of update and deletion records into the updates queue. This was a one time process, so once you get through those it'll go back to the normal amount of updates.

[LarryGT]

The /updates endpoint still returns a huge amount of data. I did a call today with "since" timestamp 11th january. The call returned, that there are about 58.000 changes ("total_items":58465). So much changes in one and a half day? When i read the data of all pages (following the "next" url), i noticed the first 71 pages contained data. The other 47 pages which were returned contained no data. So in fact there were about 35.500 changes (which contained about 10.000 duplicates).

So this is very strange and it seems there is a problem with this endpoint.

[antheaezzell]

Our developer looked into it and it's a bug. I've ticketed a fix (internal ticket - https://mediamorph.atlassian.net/browse/TVD-3599). I'll get back to you @LarryGT when it has been fixed. Thanks for reporting.

[SirSparkles]

Can this be reopened until it's fixed please?

It is an issue still - see below:

2023-01-22 21:57:33|INFO| Obtained 500 responses from lastupdated query #30 - until 19/01/2023 3:46:38 AM (1674071198) 2023-01-22 21:57:34|INFO| Obtained 500 responses from lastupdated query #31 - until 19/01/2023 4:10:54 AM (1674072654) 2023-01-22 21:57:36|INFO| Obtained 500 responses from lastupdated query #32 - until 19/01/2023 4:34:05 AM (1674074045) 2023-01-22 21:57:36|INFO| Obtained 500 responses from lastupdated query #33 - until 19/01/2023 4:55:46 AM (1674075346) 2023-01-22 21:57:37|INFO| Obtained 500 responses from lastupdated query #34 - until 19/01/2023 5:29:44 AM (1674077384) 2023-01-22 21:57:38|INFO| Obtained 500 responses from lastupdated query #35 - until 19/01/2023 6:05:09 AM (1674079509) 2023-01-22 21:57:39|INFO| Obtained 500 responses from lastupdated query #36 - until 19/01/2023 6:32:01 AM (1674081121) 2023-01-22 21:57:40|INFO| Obtained 500 responses from lastupdated query #37 - until 19/01/2023 6:54:05 AM (1674082445) 2023-01-22 21:57:41|INFO| Obtained 500 responses from lastupdated query #38 - until 19/01/2023 7:17:36 AM (1674083856) 2023-01-22 21:57:42|INFO| Obtained 500 responses from lastupdated query #39 - until 19/01/2023 7:34:41 AM (1674084881) 2023-01-22 21:57:43|INFO| Obtained 500 responses from lastupdated query #40 - until 19/01/2023 7:59:22 AM (1674086362) 2023-01-22 21:57:44|INFO| Obtained 500 responses from lastupdated query #41 - until 19/01/2023 8:36:53 AM (1674088613) 2023-01-22 21:57:45|INFO| Obtained 500 responses from lastupdated query #42 - until 19/01/2023 9:43:33 AM (1674092613) 2023-01-22 21:57:46|INFO| Obtained 500 responses from lastupdated query #43 - until 19/01/2023 10:26:30 AM (1674095190) 2023-01-22 21:57:47|INFO| Obtained 500 responses from lastupdated query #44 - until 19/01/2023 11:15:03 AM (1674098103) 2023-01-22 21:57:48|INFO| Obtained 500 responses from lastupdated query #45 - until 19/01/2023 12:39:16 PM (1674103156) 2023-01-22 21:57:50|INFO| Obtained 500 responses from lastupdated query #46 - until 19/01/2023 1:39:55 PM (1674106795) 2023-01-22 21:57:51|INFO| Obtained 500 responses from lastupdated query #47 - until 19/01/2023 3:37:52 PM (1674113872) 2023-01-22 21:57:52|INFO| Obtained 500 responses from lastupdated query #48 - until 19/01/2023 4:12:30 PM (1674115950) 2023-01-22 21:57:53|INFO| Obtained 500 responses from lastupdated query #49 - until 19/01/2023 4:59:56 PM (1674118796) 2023-01-22 21:57:54|INFO| Obtained 500 responses from lastupdated query #50 - until 19/01/2023 6:16:59 PM (1674123419) 2023-01-22 21:57:55|INFO| Obtained 500 responses from lastupdated query #51 - until 19/01/2023 6:54:28 PM (1674125668) 2023-01-22 21:57:55|INFO| Obtained 500 responses from lastupdated query #52 - until 19/01/2023 7:34:33 PM (1674128073) 2023-01-22 21:57:56|INFO| Obtained 500 responses from lastupdated query #53 - until 19/01/2023 8:00:05 PM (1674129605) 2023-01-22 21:57:57|INFO| Obtained 500 responses from lastupdated query #54 - until 19/01/2023 8:31:58 PM (1674131518) 2023-01-22 21:57:58|INFO| Obtained 500 responses from lastupdated query #55 - until 19/01/2023 8:56:04 PM (1674132964) 2023-01-22 21:57:59|INFO| Obtained 500 responses from lastupdated query #56 - until 19/01/2023 9:10:32 PM (1674133832) 2023-01-22 21:58:00|INFO| Obtained 500 responses from lastupdated query #57 - until 19/01/2023 9:26:06 PM (1674134766) 2023-01-22 21:58:02|INFO| Obtained 500 responses from lastupdated query #58 - until 19/01/2023 9:40:55 PM (1674135655) 2023-01-22 21:58:03|INFO| Obtained 500 responses from lastupdated query #59 - until 19/01/2023 10:02:25 PM (1674136945) 2023-01-22 21:58:05|INFO| Obtained 500 responses from lastupdated query #60 - until 19/01/2023 10:43:59 PM (1674139439) 2023-01-22 21:58:06|INFO| Obtained 500 responses from lastupdated query #61 - until 19/01/2023 11:14:46 PM (1674141286) 2023-01-22 21:58:07|INFO| Obtained 500 responses from lastupdated query #62 - until 19/01/2023 11:46:08 PM (1674143168) 2023-01-22 21:58:08|INFO| Obtained 500 responses from lastupdated query #63 - until 20/01/2023 12:20:42 AM (1674145242) 2023-01-22 21:58:08|INFO| Obtained 500 responses from lastupdated query #64 - until 20/01/2023 12:40:33 AM (1674146433) 2023-01-22 21:58:09|INFO| Obtained 500 responses from lastupdated query #65 - until 20/01/2023 1:09:22 AM (1674148162) 2023-01-22 21:58:10|INFO| Obtained 500 responses from lastupdated query #66 - until 20/01/2023 1:27:37 AM (1674149257) 2023-01-22 21:58:11|INFO| Obtained 500 responses from lastupdated query #67 - until 20/01/2023 1:49:19 AM (1674150559) 2023-01-22 21:58:12|INFO| Obtained 500 responses from lastupdated query #68 - until 20/01/2023 2:32:24 AM (1674153144) 2023-01-22 21:58:13|INFO| Obtained 500 responses from lastupdated query #69 - until 20/01/2023 3:07:44 AM (1674155264) 2023-01-22 21:58:14|INFO| Obtained 500 responses from lastupdated query #70 - until 20/01/2023 3:42:18 AM (1674157338) 2023-01-22 21:58:15|INFO| Obtained 500 responses from lastupdated query #71 - until 20/01/2023 4:11:02 AM (1674159062) 2023-01-22 21:58:16|INFO| Obtained 500 responses from lastupdated query #72 - until 20/01/2023 4:44:08 AM (1674161048) 2023-01-22 21:58:16|INFO| Obtained 500 responses from lastupdated query #73 - until 20/01/2023 5:16:21 AM (1674162981) 2023-01-22 21:58:17|INFO| Obtained 500 responses from lastupdated query #74 - until 20/01/2023 5:49:46 AM (1674164986) 2023-01-22 21:58:18|INFO| Obtained 500 responses from lastupdated query #75 - until 20/01/2023 6:20:58 AM (1674166858) 2023-01-22 21:58:19|INFO| Obtained 500 responses from lastupdated query #76 - until 20/01/2023 6:56:45 AM (1674169005) 2023-01-22 21:58:20|INFO| Obtained 500 responses from lastupdated query #77 - until 20/01/2023 7:35:52 AM (1674171352) 2023-01-22 21:58:21|INFO| Obtained 500 responses from lastupdated query #78 - until 20/01/2023 8:02:32 AM (1674172952) 2023-01-22 21:58:22|INFO| Obtained 500 responses from lastupdated query #79 - until 20/01/2023 8:48:31 AM (1674175711) 2023-01-22 21:58:23|INFO| Obtained 500 responses from lastupdated query #80 - until 20/01/2023 9:45:36 AM (1674179136) 2023-01-22 21:58:24|INFO| Obtained 500 responses from lastupdated query #81 - until 20/01/2023 10:14:51 AM (1674180891) 2023-01-22 21:58:25|INFO| Obtained 500 responses from lastupdated query #82 - until 20/01/2023 10:36:07 AM (1674182167) 2023-01-22 21:58:26|INFO| Obtained 500 responses from lastupdated query #83 - until 20/01/2023 11:14:58 AM (1674184498) 2023-01-22 21:58:27|INFO| Obtained 500 responses from lastupdated query #84 - until 20/01/2023 12:36:18 PM (1674189378) 2023-01-22 21:58:27|INFO| Obtained 500 responses from lastupdated query #85 - until 20/01/2023 1:42:54 PM (1674193374) 2023-01-22 21:58:28|INFO| Obtained 500 responses from lastupdated query #86 - until 20/01/2023 3:12:51 PM (1674198771) 2023-01-22 21:58:29|INFO| Obtained 500 responses from lastupdated query #87 - until 20/01/2023 4:42:06 PM (1674204126) 2023-01-22 21:58:30|INFO| Obtained 500 responses from lastupdated query #88 - until 20/01/2023 5:10:13 PM (1674205813) 2023-01-22 21:58:31|INFO| Obtained 500 responses from lastupdated query #89 - until 20/01/2023 5:44:57 PM (1674207897) 2023-01-22 21:58:32|INFO| Obtained 500 responses from lastupdated query #90 - until 20/01/2023 6:19:22 PM (1674209962) 2023-01-22 21:58:33|INFO| Obtained 500 responses from lastupdated query #91 - until 20/01/2023 6:46:00 PM (1674211560) 2023-01-22 21:58:34|INFO| Obtained 500 responses from lastupdated query #92 - until 20/01/2023 7:59:51 PM (1674215991) 2023-01-22 21:58:35|INFO| Obtained 500 responses from lastupdated query #93 - until 20/01/2023 8:38:24 PM (1674218304) 2023-01-22 21:58:36|INFO| Obtained 500 responses from lastupdated query #94 - until 20/01/2023 9:06:25 PM (1674219985) 2023-01-22 21:58:37|INFO| Obtained 500 responses from lastupdated query #95 - until 20/01/2023 9:32:50 PM (1674221570) 2023-01-22 21:58:38|INFO| Obtained 500 responses from lastupdated query #96 - until 20/01/2023 9:54:14 PM (1674222854) 2023-01-22 21:58:39|INFO| Obtained 500 responses from lastupdated query #97 - until 20/01/2023 10:11:52 PM (1674223912) 2023-01-22 21:58:40|INFO| Obtained 500 responses from lastupdated query #98 - until 20/01/2023 10:27:42 PM (1674224862) 2023-01-22 21:58:41|INFO| Obtained 500 responses from lastupdated query #99 - until 20/01/2023 10:58:54 PM (1674226734) 2023-01-22 21:58:42|INFO| Obtained 500 responses from lastupdated query #100 - until 20/01/2023 11:19:42 PM (1674227982) 2023-01-22 21:58:43|INFO| Obtained 500 responses from lastupdated query #101 - until 20/01/2023 11:25:52 PM (1674228352) 2023-01-22 21:58:44|INFO| Obtained 500 responses from lastupdated query #102 - until 20/01/2023 11:37:23 PM (1674229043) 2023-01-22 21:58:45|INFO| Obtained 500 responses from lastupdated query #103 - until 21/01/2023 12:08:02 AM (1674230882) 2023-01-22 21:58:46|INFO| Obtained 500 responses from lastupdated query #104 - until 21/01/2023 12:26:07 AM (1674231967) 2023-01-22 21:58:47|INFO| Obtained 500 responses from lastupdated query #105 - until 21/01/2023 12:43:49 AM (1674233029) 2023-01-22 21:58:48|INFO| Obtained 500 responses from lastupdated query #106 - until 21/01/2023 1:00:50 AM (1674234050) 2023-01-22 21:58:49|INFO| Obtained 500 responses from lastupdated query #107 - until 21/01/2023 1:25:59 AM (1674235559) 2023-01-22 21:58:50|INFO| Obtained 500 responses from lastupdated query #108 - until 21/01/2023 1:39:18 AM (1674236358) 2023-01-22 21:58:51|INFO| Obtained 500 responses from lastupdated query #109 - until 21/01/2023 1:55:20 AM (1674237320) 2023-01-22 21:58:52|INFO| Obtained 500 responses from lastupdated query #110 - until 21/01/2023 2:14:01 AM (1674238441) 2023-01-22 21:58:53|INFO| Obtained 500 responses from lastupdated query #111 - until 21/01/2023 2:29:00 AM (1674239340) 2023-01-22 21:58:55|INFO| Obtained 500 responses from lastupdated query #112 - until 21/01/2023 2:45:44 AM (1674240344) 2023-01-22 21:58:56|INFO| Obtained 500 responses from lastupdated query #113 - until 21/01/2023 3:01:41 AM (1674241301) 2023-01-22 21:58:57|INFO| Obtained 500 responses from lastupdated query #114 - until 21/01/2023 3:14:28 AM (1674242068) 2023-01-22 21:58:58|INFO| Obtained 500 responses from lastupdated query #115 - until 21/01/2023 3:34:54 AM (1674243294) 2023-01-22 21:58:59|INFO| Obtained 500 responses from lastupdated query #116 - until 21/01/2023 3:56:02 AM (1674244562) 2023-01-22 21:59:00|INFO| Obtained 500 responses from lastupdated query #117 - until 21/01/2023 4:19:31 AM (1674245971) 2023-01-22 21:59:01|INFO| Obtained 500 responses from lastupdated query #118 - until 21/01/2023 4:47:25 AM (1674247645) 2023-01-22 21:59:01|INFO| Obtained 500 responses from lastupdated query #119 - until 21/01/2023 5:06:06 AM (1674248766) 2023-01-22 21:59:02|INFO| Obtained 500 responses from lastupdated query #120 - until 21/01/2023 5:33:38 AM (1674250418) 2023-01-22 21:59:03|INFO| Obtained 500 responses from lastupdated query #121 - until 21/01/2023 5:59:18 AM (1674251958) 2023-01-22 21:59:04|INFO| Obtained 500 responses from lastupdated query #122 - until 21/01/2023 6:34:38 AM (1674254078) 2023-01-22 21:59:05|INFO| Obtained 500 responses from lastupdated query #123 - until 21/01/2023 6:56:23 AM (1674255383) 2023-01-22 21:59:06|INFO| Obtained 500 responses from lastupdated query #124 - until 21/01/2023 7:13:54 AM (1674256434) 2023-01-22 21:59:07|INFO| Obtained 500 responses from lastupdated query #125 - until 21/01/2023 7:27:02 AM (1674257222) 2023-01-22 21:59:08|INFO| Obtained 500 responses from lastupdated query #126 - until 21/01/2023 7:40:53 AM (1674258053) 2023-01-22 21:59:09|INFO| Obtained 500 responses from lastupdated query #127 - until 21/01/2023 8:03:58 AM (1674259438) 2023-01-22 21:59:10|INFO| Obtained 500 responses from lastupdated query #128 - until 21/01/2023 8:21:06 AM (1674260466) 2023-01-22 21:59:11|INFO| Obtained 500 responses from lastupdated query #129 - until 21/01/2023 8:52:33 AM (1674262353) 2023-01-22 21:59:12|INFO| Obtained 500 responses from lastupdated query #130 - until 21/01/2023 9:12:26 AM (1674263546) 2023-01-22 21:59:12|INFO| Obtained 500 responses from lastupdated query #131 - until 21/01/2023 9:42:36 AM (1674265356) 2023-01-22 21:59:13|INFO| Obtained 500 responses from lastupdated query #132 - until 21/01/2023 10:21:51 AM (1674267711) 2023-01-22 21:59:15|INFO| Obtained 500 responses from lastupdated query #133 - until 21/01/2023 10:52:28 AM (1674269548) 2023-01-22 21:59:16|INFO| Obtained 500 responses from lastupdated query #134 - until 21/01/2023 11:45:21 AM (1674272721) 2023-01-22 21:59:17|INFO| Obtained 500 responses from lastupdated query #135 - until 21/01/2023 12:34:24 PM (1674275664) 2023-01-22 21:59:18|INFO| Obtained 500 responses from lastupdated query #136 - until 21/01/2023 12:57:51 PM (1674277071) 2023-01-22 21:59:19|INFO| Obtained 500 responses from lastupdated query #137 - until 21/01/2023 1:45:13 PM (1674279913) 2023-01-22 21:59:20|INFO| Obtained 500 responses from lastupdated query #138 - until 21/01/2023 3:16:23 PM (1674285383) 2023-01-22 21:59:21|INFO| Obtained 500 responses from lastupdated query #139 - until 21/01/2023 4:15:55 PM (1674288955) 2023-01-22 21:59:22|INFO| Obtained 500 responses from lastupdated query #140 - until 21/01/2023 4:49:28 PM (1674290968) 2023-01-22 21:59:24|INFO| Obtained 500 responses from lastupdated query #141 - until 21/01/2023 5:40:03 PM (1674294003) 2023-01-22 21:59:25|INFO| Obtained 500 responses from lastupdated query #142 - until 21/01/2023 6:29:48 PM (1674296988) 2023-01-22 21:59:26|INFO| Obtained 500 responses from lastupdated query #143 - until 21/01/2023 7:07:27 PM (1674299247) 2023-01-22 21:59:27|INFO| Obtained 500 responses from lastupdated query #144 - until 21/01/2023 7:38:35 PM (1674301115) 2023-01-22 21:59:28|INFO| Obtained 500 responses from lastupdated query #145 - until 21/01/2023 8:09:26 PM (1674302966) 2023-01-22 21:59:29|INFO| Obtained 500 responses from lastupdated query #146 - until 21/01/2023 9:06:21 PM (1674306381) 2023-01-22 21:59:30|INFO| Obtained 500 responses from lastupdated query #147 - until 21/01/2023 9:34:51 PM (1674308091) 2023-01-22 21:59:31|INFO| Obtained 500 responses from lastupdated query #148 - until 21/01/2023 10:11:44 PM (1674310304) 2023-01-22 21:59:32|INFO| Obtained 500 responses from lastupdated query #149 - until 21/01/2023 10:46:10 PM (1674312370) 2023-01-22 21:59:33|INFO| Obtained 500 responses from lastupdated query #150 - until 21/01/2023 11:09:13 PM (1674313753) 2023-01-22 21:59:34|INFO| Obtained 500 responses from lastupdated query #151 - until 21/01/2023 11:34:19 PM (1674315259) 2023-01-22 21:59:35|INFO| Obtained 500 responses from lastupdated query #152 - until 22/01/2023 12:00:03 AM (1674316803) 2023-01-22 21:59:36|INFO| Obtained 500 responses from lastupdated query #153 - until 22/01/2023 12:35:29 AM (1674318929) 2023-01-22 21:59:37|INFO| Obtained 500 responses from lastupdated query #154 - until 22/01/2023 1:00:08 AM (1674320408) 2023-01-22 21:59:38|INFO| Obtained 500 responses from lastupdated query #155 - until 22/01/2023 1:18:18 AM (1674321498) 2023-01-22 21:59:39|INFO| Obtained 500 responses from lastupdated query #156 - until 22/01/2023 1:50:24 AM (1674323424) 2023-01-22 21:59:40|INFO| Obtained 500 responses from lastupdated query #157 - until 22/01/2023 2:05:46 AM (1674324346) 2023-01-22 21:59:41|INFO| Obtained 500 responses from lastupdated query #158 - until 22/01/2023 2:28:39 AM (1674325719) 2023-01-22 21:59:42|INFO| Obtained 500 responses from lastupdated query #159 - until 22/01/2023 2:50:24 AM (1674327024) 2023-01-22 21:59:43|INFO| Obtained 500 responses from lastupdated query #160 - until 22/01/2023 3:32:00 AM (1674329520) 2023-01-22 21:59:44|INFO| Obtained 500 responses from lastupdated query #161 - until 22/01/2023 3:46:39 AM (1674330399) 2023-01-22 21:59:44|INFO| Obtained 500 responses from lastupdated query #162 - until 22/01/2023 4:02:53 AM (1674331373) 2023-01-22 21:59:45|INFO| Obtained 500 responses from lastupdated query #163 - until 22/01/2023 4:36:22 AM (1674333382) 2023-01-22 21:59:46|INFO| Obtained 500 responses from lastupdated query #164 - until 22/01/2023 5:17:43 AM (1674335863) 2023-01-22 21:59:47|INFO| Obtained 500 responses from lastupdated query #165 - until 22/01/2023 5:30:11 AM (1674336611) 2023-01-22 21:59:48|INFO| Obtained 500 responses from lastupdated query #166 - until 22/01/2023 5:32:56 AM (1674336776) 2023-01-22 21:59:49|INFO| Obtained 500 responses from lastupdated query #167 - until 22/01/2023 5:36:25 AM (1674336985) 2023-01-22 21:59:50|INFO| Obtained 500 responses from lastupdated query #168 - until 22/01/2023 5:40:17 AM (1674337217) 2023-01-22 21:59:52|INFO| Obtained 500 responses from lastupdated query #169 - until 22/01/2023 6:05:05 AM (1674338705) 2023-01-22 21:59:53|INFO| Obtained 500 responses from lastupdated query #170 - until 22/01/2023 6:44:41 AM (1674341081) 2023-01-22 21:59:53|INFO| Obtained 500 responses from lastupdated query #171 - until 22/01/2023 7:31:12 AM (1674343872) 2023-01-22 21:59:54|INFO| Obtained 500 responses from lastupdated query #172 - until 22/01/2023 8:02:55 AM (1674345775) 2023-01-22 21:59:55|INFO| Obtained 500 responses from lastupdated query #173 - until 22/01/2023 8:28:27 AM (1674347307) 2023-01-22 21:59:56|INFO| Obtained 500 responses from lastupdated query #174 - until 22/01/2023 9:03:11 AM (1674349391) 2023-01-22 21:59:57|INFO| Obtained 500 responses from lastupdated query #175 - until 22/01/2023 9:25:34 AM (1674350734) 2023-01-22 21:59:58|INFO| Obtained 500 responses from lastupdated query #176 - until 22/01/2023 9:59:10 AM (1674352750) 2023-01-22 21:59:59|INFO| Obtained 500 responses from lastupdated query #177 - until 22/01/2023 10:21:22 AM (1674354082) 2023-01-22 22:00:00|INFO| Obtained 500 responses from lastupdated query #178 - until 22/01/2023 11:06:11 AM (1674356771) 2023-01-22 22:00:01|INFO| Obtained 500 responses from lastupdated query #179 - until 22/01/2023 12:21:48 PM (1674361308) 2023-01-22 22:00:02|INFO| Obtained 500 responses from lastupdated query #180 - until 22/01/2023 1:32:56 PM (1674365576) 2023-01-22 22:00:03|INFO| Obtained 500 responses from lastupdated query #181 - until 22/01/2023 2:35:34 PM (1674369334) 2023-01-22 22:00:04|INFO| Obtained 500 responses from lastupdated query #182 - until 22/01/2023 3:48:31 PM (1674373711) 2023-01-22 22:00:05|INFO| Obtained 500 responses from lastupdated query #183 - until 22/01/2023 4:27:46 PM (1674376066) 2023-01-22 22:00:06|INFO| Obtained 500 responses from lastupdated query #184 - until 22/01/2023 5:10:15 PM (1674378615) 2023-01-22 22:00:07|INFO| Obtained 500 responses from lastupdated query #185 - until 22/01/2023 5:40:21 PM (1674380421) 2023-01-22 22:00:08|INFO| Obtained 500 responses from lastupdated query #186 - until 22/01/2023 6:18:07 PM (1674382687) 2023-01-22 22:00:09|INFO| Obtained 500 responses from lastupdated query #187 - until 22/01/2023 7:22:41 PM (1674386561) 2023-01-22 22:00:10|INFO| Obtained 500 responses from lastupdated query #188 - until 22/01/2023 8:09:59 PM (1674389399) 2023-01-22 22:00:10|INFO| Obtained 500 responses from lastupdated query #189 - until 22/01/2023 8:54:15 PM (1674392055) 2023-01-22 22:00:11|INFO| Obtained 500 responses from lastupdated query #190 - until 22/01/2023 9:18:15 PM (1674393495) 2023-01-22 22:00:12|INFO| Obtained 500 responses from lastupdated query #191 - until 22/01/2023 9:37:02 PM (1674394622) 2023-01-22 22:00:13|INFO| Obtained 425 responses from lastupdated query #192 - until 22/01/2023 9:59:58 PM (1674395998)

[antheaezzell]

Hi @SirSparkles our developers checked and do not see an inordinate number of updates beyond what he explained a few weeks ago when we has to inject a number of update and deletion records into the updates queue. The one time process has run and this should be the normal amount of updates. We did have a bug that @LarryGT alerted us to where there was empty data on the updates endpoint and a wrong count of records. This was fixed in the v4.7.3 release on Friday and the documentation updated. Does this solve your issue? Or is there something else I might be missing?

[LarryGT]

Hi, there are no more empty pages, but i am wondering why there are duplicates. I did a call for the last 24 hours, and the endpoint returned 22035 updates, but there are 6794 updates which are duplicate (same entityType and recordId). It would be enough to list each id only once. If a record is changed three times there is no need to list all three changes but only one. So the endpoint has to deliver less data.

[antheaezzell]

@LarryGT let me look into this for you.

[LarryGT]

Any news about this?

[antheaezzell]

We are having our developers consider an approach here. I've just followed up with them to weigh in and investigate whether this would be a breaking change.

[antheaezzell]

Hi @LarryGT an approach was agreed to and I have ticketed the work. We are working with a resource limitation right now, but I can keep you posted on progress of the work intermittently.

Ticket for our internal purposes - https://mediamorph.atlassian.net/browse/TVD-3633

[LarryGT]

What is the status of this ticket? I noticed the endpoint returns many empty pages again.

[maxifwortman]

Hi @LarryGT, do you have an example request that is failing?

[LarryGT]

I just did a call to https://api4.thetvdb.com/v4/updates?since=1681564673 and only the first page contained data. The other 54 pages did not contain data.

[maxifwortman]

@LarryGT thanks for reporting this. There was an issue that was returning an empty page for updates. If you could try it again, you should see the correct info.

[LarryGT]

The pages no contain data, but there are still duplicates. I think there is still work to be done there.

[antheaezzell]

@LarryGT can you please provide an example of the duplicates, so I can add to the investigatory ticket?

[LarryGT]

I have checked the updates of the last 24 hours. The file attached contains the duplicates.

output.txt

[antheaezzell]

Thanks. I'll add it to the ticket.