search

vmware-archive / kafka-trigger

Kubernetes CRD controller for Kafka topic as event source for Kubeless functions
Apache License 2.0
28 stars 34 forks source link

Add backoff #33

Closed sepetrov closed 4 years ago

sepetrov commented 4 years ago

Issue Ref: None

Description:

Adds exponential backoff.

To enable this behaviour, one must set environment variable BACKOFF_INTERVAL in format compatible with time.ParseDuration.

If handler function responds with non-200 status code, the controller will wait for duration d before sending the next message / request. Once the handler function responds with 200 OK, d will be reset to 0.

TODOs:

sepetrov commented 4 years ago

The sequence of exponential backoff intervals for 1000 unsuccessful requests (in seconds):

Screenshot 2020-10-07 at 10 01 34
Sequence of delays in seconds ```console 0.552330 1.080382 1.310130 1.582393 2.340489 4.506219 3.221480 5.608623 7.649838 15.394871 29.271138 56.813325 42.855832 52.839431 49.083490 58.133391 46.982049 47.586111 70.745081 43.113183 42.191213 51.652285 64.240397 81.749486 47.586855 47.824954 75.154382 42.394960 81.920101 71.803150 61.429218 31.698185 39.499697 66.435206 88.514497 34.767217 65.688516 33.547239 71.521475 48.091361 40.395974 62.465991 62.649334 46.710457 55.389132 61.835143 45.212430 46.924860 77.316295 51.708329 82.832587 47.826736 83.661704 35.847277 88.615012 34.457460 43.337365 70.864699 44.490905 48.691347 85.970786 74.510938 78.063303 73.813889 40.975495 55.701425 83.819517 70.959209 88.735761 85.332736 35.450237 59.588520 85.619208 87.296726 50.877238 71.450330 72.654432 63.826776 68.969368 63.105903 75.349410 54.228197 37.839067 89.157884 83.780505 49.325038 73.268866 68.672387 35.131230 70.174518 67.363699 52.181571 44.209353 62.116913 41.234766 44.330442 67.685890 37.605176 46.879818 54.619371 56.094748 67.505702 63.008815 67.416530 73.750844 79.832035 30.030829 74.164116 53.999026 59.872087 66.238686 54.577097 31.780277 30.114234 30.170582 84.949279 65.390051 63.563547 78.924310 82.680706 57.506549 66.009936 31.575909 80.749967 44.981592 68.507057 44.847996 40.419351 65.557425 78.863673 71.630288 31.819353 62.352606 88.540489 75.045783 47.640379 75.189677 39.057843 51.346036 79.915851 43.909803 67.670076 59.903658 35.390165 31.511638 53.532971 65.362985 85.776698 64.325208 65.314581 54.705761 63.154823 59.496444 87.477235 77.832512 36.442867 76.982098 53.595060 37.824831 41.401966 74.389547 69.242485 35.903027 61.222817 35.983780 39.110604 34.571416 48.912485 39.579055 38.268244 49.356641 62.344471 64.251098 60.766905 71.050508 69.182412 61.469986 69.256208 72.982102 68.198653 30.769555 31.840932 35.881852 52.146703 79.587248 50.860903 50.658901 45.179989 42.988269 63.300128 54.124251 60.389824 40.120780 49.882096 79.675686 72.017272 33.475576 89.949569 54.692422 36.700478 76.845245 35.527057 33.209677 72.881749 45.045737 80.917975 88.432912 42.753657 31.292027 86.711686 35.578209 68.750002 48.713133 56.907862 59.234355 34.948781 70.309746 54.011297 84.016509 86.992992 49.159876 59.963130 54.025939 31.188520 68.702332 55.721306 50.375805 83.246850 44.179648 75.900493 32.145279 73.654635 67.550198 60.785250 34.346901 73.453744 82.790691 88.665809 80.850016 79.931876 44.870671 84.803944 34.502233 80.106228 67.759901 75.104435 67.920206 35.816053 30.889642 65.030085 34.125372 89.896429 68.951305 89.127935 80.088346 49.923365 69.683591 87.361238 48.630617 41.063442 88.025660 79.994509 48.572907 78.352306 55.039551 73.111827 54.404207 83.748196 87.490582 31.122793 77.500339 55.413189 30.910877 55.961894 84.286574 81.342265 32.575299 69.541832 50.871543 60.209207 80.396845 31.386574 37.461811 45.670537 80.096850 48.888288 30.460872 83.985008 52.216052 36.011965 68.592242 76.193345 77.467520 45.742914 50.811833 42.879223 79.325574 51.068058 65.951655 64.701075 54.814859 37.191031 84.696822 33.227135 43.735055 49.450438 51.045908 50.957325 48.228128 88.124770 70.291593 42.476588 87.788364 48.132143 78.476465 38.045050 86.865617 68.451889 87.195525 78.592454 40.895451 86.565442 79.874462 59.680826 81.318621 72.644635 46.409685 54.457972 84.585677 86.663828 59.917947 47.318299 88.553715 57.155069 32.699442 48.921719 87.114369 75.093785 62.147460 70.182875 81.910500 57.533067 64.713054 58.891789 63.036946 87.037395 60.591925 74.550884 59.447641 33.969085 45.749440 85.528077 52.289199 54.565164 54.945118 35.835696 84.097657 30.266680 46.435473 36.558400 81.326905 45.423321 89.347926 85.584685 40.256762 48.233227 62.007087 40.589377 78.815446 72.308227 45.432453 45.022135 50.105662 75.074438 30.292787 80.459592 43.774415 30.797133 86.996244 83.962288 87.757452 32.580022 72.759757 33.065646 54.451926 58.541842 50.848103 32.443196 65.853972 45.607480 79.971351 87.629850 86.202454 43.759214 73.218786 75.388939 57.009236 50.338643 58.349523 89.159662 51.739889 44.979300 32.872101 77.383033 75.372611 69.448143 43.259931 69.450323 53.450547 85.155270 38.981129 61.223521 75.935312 44.890948 57.326638 71.987349 34.918168 60.736322 45.982908 67.320098 35.944211 38.434517 39.552214 86.640906 67.888595 44.747517 88.050315 62.218837 50.290866 70.300058 80.247955 74.031290 60.133249 55.408427 57.782370 87.182679 74.644357 57.311376 44.905019 73.193134 55.422132 74.555451 69.410896 70.951295 68.207454 44.914970 53.211686 76.198257 37.581862 47.198266 60.680351 41.913692 89.676382 68.157197 73.639140 54.738927 60.841414 37.871395 62.736158 80.702101 47.597061 76.698229 38.506726 69.262771 67.880894 53.637915 30.084693 63.951997 83.267521 57.148970 65.770720 68.570120 79.790304 78.622810 33.956090 34.888482 50.131081 59.438084 67.141781 67.659524 49.478646 77.592871 35.051213 45.259543 71.254091 55.213042 31.327868 43.019808 34.694744 63.830660 43.714512 52.054414 46.322693 84.017986 69.709803 56.707795 88.124674 70.056800 51.864004 30.397618 42.370517 76.947303 75.985876 78.494497 47.274569 66.596983 57.132236 43.590575 53.343383 44.811876 30.451707 39.208543 30.269358 84.552213 44.270455 49.635784 77.589397 55.817507 56.405683 35.976569 77.041241 61.324746 36.131273 80.897222 39.831506 63.924253 37.271847 46.521270 89.899985 85.757991 36.197321 42.652266 48.875408 58.345749 38.596014 52.528704 71.268234 77.274061 31.968952 35.783344 50.089104 61.386446 50.914803 40.816307 84.882085 52.086064 39.507316 35.461071 45.788888 53.669198 41.337285 75.351194 68.481814 49.120947 67.703805 46.706687 50.219105 64.648088 55.695308 36.629414 50.807955 52.859021 78.299583 82.982215 52.637614 40.790756 56.838191 41.362348 35.931167 39.878111 31.591481 69.070441 77.008353 41.025565 88.508103 73.874287 56.558714 45.075571 66.746205 31.347802 35.789563 60.004320 44.291488 46.860583 56.280473 84.928979 42.327759 72.174890 58.210737 86.500661 35.373467 76.155325 31.104242 35.635354 87.049699 33.793216 60.195849 77.864696 62.723197 54.284211 42.138749 49.090127 87.070346 77.472421 87.880191 39.961716 46.294618 68.979059 45.352264 66.987972 41.356636 87.625710 51.392260 85.848721 66.924223 70.531225 76.946501 33.949703 75.855792 73.447758 37.390315 78.531131 62.102476 62.731489 43.114188 33.034280 67.678347 67.136331 57.487981 58.338054 82.016166 64.022327 54.383353 66.481287 57.818281 81.268026 67.387835 71.685980 49.931412 71.506784 77.024583 77.329944 77.075445 81.120969 48.569374 31.627137 33.165799 40.688761 52.925584 76.423750 47.507590 52.385377 55.797431 42.969882 49.504881 60.076052 54.773975 47.284514 76.368578 44.509807 38.290473 64.468984 50.064940 33.308716 68.494549 40.693359 62.450561 69.189572 59.899877 45.072538 34.152532 69.325018 54.861087 45.559406 85.938510 31.607862 72.715768 80.822847 39.036663 38.879139 66.747049 74.327785 66.605600 81.635561 35.050297 54.904680 77.996851 42.676049 36.215546 42.757235 59.894206 39.763412 86.645136 79.213076 77.106904 44.683444 66.815424 45.252294 66.929088 42.259668 34.335750 61.525632 84.630501 57.933846 81.636466 77.013066 80.027797 83.877119 64.688381 47.653172 34.727905 88.655863 63.025169 55.452961 58.719568 33.409921 40.823381 44.446140 69.492067 54.991963 76.312491 37.752444 42.147545 36.070402 75.477239 78.600692 60.837903 41.145688 30.748862 62.417323 88.621362 77.717750 59.209158 62.445058 87.117580 74.064151 58.126444 49.511182 55.457268 62.437299 62.492486 68.313305 74.789349 50.960396 77.627963 41.136340 68.538661 47.865892 72.558559 80.034739 64.828250 68.653418 31.192495 84.841632 73.369569 69.376421 80.574124 35.959853 88.367865 60.824312 40.497042 82.527257 70.273527 30.696610 75.649474 54.147523 35.419952 54.099350 35.950279 34.027701 41.279618 57.847784 39.508222 46.941933 35.859788 85.564931 86.762741 66.631210 38.941454 31.383333 74.357481 34.393891 83.994288 61.921693 78.464751 73.251461 62.750866 45.040010 51.476716 83.471249 77.964209 45.992931 56.837471 37.700142 76.504735 49.119542 54.197350 63.955757 34.314077 47.392627 66.403365 79.575671 67.775263 39.672897 42.601835 73.567830 65.046227 85.441259 58.055762 74.477911 60.155810 47.014689 86.701319 48.522016 74.140103 52.206830 37.527993 30.460295 30.836066 58.220449 62.368933 75.023243 58.617978 81.647031 39.362491 62.498323 46.113939 86.863681 76.669817 74.245044 65.836608 34.397640 62.350720 71.183760 46.105871 32.479231 68.882708 54.782893 84.445391 34.388388 72.147599 40.278068 83.331854 84.382172 67.736249 67.858369 30.957441 43.378775 68.678231 61.060947 35.181799 44.506107 76.452236 73.709769 65.820373 45.181130 81.890667 60.405845 84.825408 81.279064 71.302824 48.299082 46.424026 75.057058 73.516562 49.734859 46.698796 85.075360 83.787725 78.768756 66.217002 87.762747 37.396038 66.542956 42.838323 84.560928 78.595214 75.380529 65.374905 47.205495 34.940954 39.638838 58.330237 43.931935 67.167542 72.263802 56.057928 57.321068 71.917799 84.408312 49.620241 65.952643 32.336045 37.851188 72.516139 84.318528 68.743453 81.706430 79.227625 55.145867 82.879245 78.299814 71.217277 71.341740 85.090385 30.335009 41.182362 82.633140 88.459057 62.084658 74.913115 63.666468 30.957221 38.070491 38.287268 35.303573 79.245739 88.860967 44.103725 30.954163 58.820904 41.402049 84.468799 44.373208 30.837266 52.992956 31.562336 60.400332 56.641494 44.648437 76.965340 67.687199 87.154442 54.700159 88.926957 44.650617 89.972545 38.412496 58.011658 61.004276 36.344640 62.599319 63.994257 31.266901 54.418803 87.008546 85.742281 77.849010 59.140929 47.428823 49.714637 72.073608 35.916534 37.338974 58.878668 36.053241 85.604646 50.799737 50.432756 40.286050 67.241213 61.160664 48.592931 53.991863 40.030092 43.080609 85.771476 68.433809 60.106766 66.163349 40.836979 77.185850 60.492895 66.342837 72.115009 73.445947 40.242073 37.068919 32.035201 40.133459 68.951556 64.981168 38.112765 48.502320 89.452995 60.021413 49.415249 56.477020 35.989943 83.583138 47.511786 33.943993 83.533806 37.160594 40.195640 78.001517 51.677882 65.448590 76.230543 57.040357 ```
andresmgot commented 4 years ago

I am going to cut a new release with the latest changes