We integrate with other services to get e.g. cites status.
When we are being crawled by services that actually render the client side code, then many of these called are triggered and will at times bring down our partner services.
We should try to prevent that. One approach could be to detect spiders that identify themself. And then just not contact external services in those cases.
We integrate with other services to get e.g. cites status. When we are being crawled by services that actually render the client side code, then many of these called are triggered and will at times bring down our partner services.
We should try to prevent that. One approach could be to detect spiders that identify themself. And then just not contact external services in those cases.