search

jitsucom / jitsu

Jitsu is an open-source Segment alternative. Fully-scriptable data ingestion engine for modern data teams. Set-up a real-time data pipeline in minutes, not days
https://jitsu.com
MIT License
4.13k stars 296 forks source link

Add support to replay failed Data Warehouse Events #1091

Closed gordonkjlee closed 9 months ago

gordonkjlee commented 9 months ago

I have a number of failed Data Warehouse Events due to an authentication issue which I've now addressed. However, Jitsu does not seem to be able to replay/resend events to the Data Warehouse.

It would be great if there was an ability in Jitsu to "replay" events to the Data Warehouse.

vklimontovich commented 9 months ago

We have a multiple replay mechanisms. If it didn't work for your case, could you be more specific? What data warehouse you used? Was it streaming or batch mode? How long authentification issue persisted? What is the version of Jitsu you've been using

gordonkjlee commented 9 months ago

@vklimontovich I'm using the jitsucom/ingest:latest image (so 2.4.6), and syncing data into Postgres in Batch mode. It failed on Friday, and I fixed the warehouse user credentials on Sunday. But I don't see any UI/button to trigger replay - or was this meant to be automatically done?

vklimontovich commented 9 months ago

Yes, Jitsu replays events automatically. That being said, there's a certain time window when Jitsu tries to do so. 2 days may be over this limit. @absorbb should know better, so I'd leave it for him