search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.96k stars 4.1k forks source link

Support heavy log file on the webapp (add pagination for logs) #9563

Closed AM-I-Human closed 2 years ago

AM-I-Human commented 2 years ago

Tell us about the problem you're trying to solve

I'm trying to debug a long sync job, and I need to see the error. The problem is that if the log is huge, there will be a lot of time spent on downloading the full log.

Describe the solution you’d like

The worked api support paginations starting from the tail of the log. It will enable you to have an hidden pagination that is called by scrolling upwards.

Describe the alternative you’ve considered or used

Attach to the docker worker and see all logs until you find the correct one.

Additional context

You don't know which log is it because

Are you willing to submit a PR?

Yes, but I don't think my employer let me do that

timroes commented 2 years ago

Duplicate of https://github.com/airbytehq/airbyte/issues/3501