omnivore-app / omnivore

Omnivore is a complete, open source read-it-later solution for people who like reading.
https://omnivore.app
GNU Affero General Public License v3.0
13.61k stars 880 forks source link

docs: what is the expected csv format? #2094

Closed thiswillbeyourgithub closed 1 year ago

thiswillbeyourgithub commented 1 year ago

Hi!

I've been trying to import from wallabag by hand and don't find the reference for the csv import format. So far my api import commands have been silently accepted (using curl) but so far no new articles appeared :/

My current columns are "Title, URL Tags, Creation date"

I'm also wondering how I'm supposed to specify the label for my imports.

Thanks!

jacksonh commented 1 year ago

Hello, if using enum UploadImportFileType URL_LIST your CSV file should just have one column URL, right now there isn't a way to specify labels, but since we just added some labels during save API we could probably update the importer to allow this.

thiswillbeyourgithub commented 1 year ago

Great and thanks for the quick reply! I'll wait a bit for the update and can't wait to try it on!

thiswillbeyourgithub commented 1 year ago

Given that I'm migrating from wallabag, I would be really interested in trying yo import my passed articles and highlights. Would it be possible to have an extensive description of the csv that would fit omnivore's internal format? I would then gladly make a script for power users to fetch wallabag, process them in the right way and then export them as a csv.

Would that be doable? Most of the work would rely on my script for formatting instead on your shoulders (appart from the documemtation!)

Thanks!

Nezteb commented 1 year ago

So far my api import commands have been silently accepted but so far no new articles appeared

your CSV file should just have one column URL

I'm using a CSV I exported from Raindrop.io and deleted all columns except url. I used the code from the import demo, but after 30 minutes of waiting no new articles show up. Is that expected?

Nezteb commented 1 year ago

Just kidding, the import process just takes while (591 URLs). I started my import at 3:30pm, and it finished thirteen hours later at 4:30am.

The email I received did say that a single URL failed to import, but it gives no indication of which URL failed.

jacksonh commented 1 year ago

Thanks, yeah it can take a very long time as all those pages have to be fetched and indexed. Working on updating the docs and the demo app now.

thiswillbeyourgithub commented 1 year ago

Btw, I made a quick repo and script to make a transition from wallabag to omnivore via logseq.

The highlights and article content is imported to logseq The unread articles are imported to omnivore using the csv API.

https://github.com/thiswillbeyourgithub/wallabag_to_logseq_and_omnivore

It's very quick and dirty but hope it helps someone!

thiswillbeyourgithub commented 1 year ago

Reopening because although I receives an email saying the import process started i have yet to see imported documents in my library and it's been a few days.

An access to the progress status and logs in the settings panel would be very helpful IMO :)

Related to #2301 #2427 and #2149

matthieugd commented 1 year ago

The current documentation found at https://docs.omnivore.app/using/importing.html#importing-csv-files looks good to me, maybe we can close this issue ?

jacksonh commented 1 year ago

Yeah thanks!