search

ConduitIO / conduit-connector-file

Conduit connector for local files
Apache License 2.0
4 stars 3 forks source link

Source reading all lines on each change #27

Open hariso opened 1 year ago

hariso commented 1 year ago

Bug description

The source connector is reading all lines on each change.

Steps to reproduce

  1. Run a pipeline with the following config: 
    ---
version: 1.0
pipelines:
pipeline1:
status: running
name: pipeline1
description: 'file source, file destination'
connectors:
  file-source:
    type: source
    plugin: "builtin:file"
    name: file-source
    settings:
      path: /home/haris/projects/other/conduit-utils/file-source.txt
  file-destination:
    type: destination
    plugin: "builtin:file"
    name: file-destination
    settings:
      path: /home/haris/projects/other/conduit-utils/file-destination.txt
  2. Write a new line to the source file.
  3. Check the destination file

Expected behavior: Only the new line appears in the destination file.

Actual behavior: All the lines in the source file are added to the destination file.

Version

{ "version": "v0.4.0-nightly.20221201", "os": "linux", "arch": "amd64" }

hariso commented 1 year ago

I was using the Sublime text editor to edit the source file. When using echo, this issue is not reproduced. It looks like Sublime is always overwriting a file when saving it.