search

stackabletech / stackablectl

Commandline tool to interact with a Stackable Data Platform
Other
8 stars 1 forks source link

Make sure no demo misuses external APIs #170

Closed lfrancke closed 1 year ago

lfrancke commented 1 year ago

We need to make sure that we are good netizens and every external API we call should be called with a really low rate limit.

sbernauer commented 1 year ago

This will be fixed/improved to approx 0.77 request/s (fetch the 700 stations every 15 mins) for the lakehouse demo in https://github.com/stackabletech/stackablectl/pull/175 The only other demo that needs to be modified is nifi-kafka-druid-water-level-data, which should be rather easy (copy/paste)

lfrancke commented 1 year ago

Has this been done for both demos? I see #175 is merged. Has the nifi-kafka-druid-water-level-data demo been fixed as well @snocke @sbernauer ?

sbernauer commented 1 year ago

Not yet, had no time so far. But after completing AuthenticationClass static and other stuff I should come to it pretty soon

lfrancke commented 1 year ago

Isn't that something @snocke can also do?

sbernauer commented 1 year ago

Technically anyone can do. The most time-consuming part is spinning everything up and test that it continues working

sbernauer commented 1 year ago

Last part: https://github.com/stackabletech/stackablectl/pull/188