search

cube2222 / octosql

OctoSQL is a query tool that allows you to join, analyse and transform data from multiple databases and file formats using SQL.
Mozilla Public License 2.0
4.74k stars 201 forks source link

File line by line #318

Open gedw99 opened 1 year ago

gedw99 commented 1 year ago

In the readme it says it can read files line by line.

this seems to be the code:

https://github.com/cube2222/octosql/tree/main/datasources/lines

I wonder if this might match my use cases.

i need to query and subscribe to file changes of text files in a folder.

the text files are old format and not the point .

I don’t want to query against the contents of these text files but rather be notifies when a files changes and what line it was and what the old and new line is.

cube2222 commented 1 year ago

@gedw99 Hey!

octosql won't actually show you which line changes as it changes; it basically works like tail -f.

gedw99 commented 1 year ago

Ah thanks @cube2222

I have gotten something working that does tell me what kind changed , give me a diff and allows me to them patch .

It’s useful for being able to do mutations on json or csv.

If you want I can PR it in as it’s own cmd. It has basic tests.

cube2222 commented 1 year ago

I'm not sure if having a separate cmd makes too much sense here (unless you mean plugin) but I'd love to see a screenshot / code if it's open source @gedw99!

gedw99 commented 1 year ago

It’s open source bits ..

Let me pull it together and pt on my GitHub .

it uses unified diff that’s the code of git but it’s all golang.

That allows you to do a CQRS / cdc style pattern off a file such that a change event diff can be created and then used for patching on the other side .

The line number is part of the unified diff and created for you.

Kind of makes sense ? It’s a different way to skin this cat as they say