omnivore-app / obsidian-omnivore

Obsidian plugin to fetch articles and highlights from Omnivore
MIT License
842 stars 82 forks source link

'Everything is up-to-date' but nothing synced #148

Open mrinc opened 1 year ago

mrinc commented 1 year ago

Obsidian v1.4.16
Omnivore v1.6.3

Config/settings : default (just api key added)

I had an oddity where the plugin would not sync my most recent articles - so I tried the re-installing and all basic stuff but a little stuck at why right now.

git obsidian message: Everything is up-to-date   
plugin:obsidian-omnivore:14284 obsidian-omnivore starting sync since: ''   
plugin:obsidian-git:44193 git obsidian message: Everything is up-to-date   
plugin:obsidian-omnivore:14284 obsidian-omnivore starting sync since: ''   
2plugin:obsidian-git:44193 git obsidian message: Everything is up-to-date   

I delete the omnivore folder so it can just re-sync from the start in case it was something weird with existing files - but nothing.

I am at the point now where it synced a single bookmark, and nothing else.
image

No errors or anything else occurs in the console that would allude to any errors.

I can revert back to a backup, so no issues for data - but I cannot sync new data and there are no errors... any idea or where I can look?

jonasgantner commented 1 year ago

Did you try to go to command prompt -> "Omnivore: Resync all articles"

mrinc commented 1 year ago

Did you try to go to command prompt -> "Omnivore: Resync all articles"

... yep, even deleted everything to start from fresh and nothing

sywhb commented 1 year ago

Hey @mrinc, sorry for the late reply.

I'm not sure if it is related but may I know the "filter" in your plugin settings? It defaults to sync highlights only.

Screenshot 2023-11-08 at 4 56 34 PM
mrinc commented 1 year ago

Welp... seems to work now - Thank you :D

It doesn't explain the initial issue at hand though, which was it just not syncing past a specific date even with that setting as all.
If it occurs again, I'll re-open this issue.

mrinc commented 1 year ago

Okay, so there is def somme up and I think I've at least got a direction to go to.... but no errors or anything
So no idea what is actually wrong :/

https://www.forbes.com/sites/abdoriani/2023/09/27/5-innovative-startup-opportunities-in-iot?sh=3c09738d2c75
This article and this config for the data/content: https://docs.omnivore.app/integrations/obsidian.html#controlling-the-layout-of-the-data-imported-to-obsidian

---
id: {{{id}}}
title: >
  {{{title}}}
{{#author}}
author: >
  {{{author}}}
{{/author}}
{{#labels.length}}
tags:
{{#labels}} - {{{name}}}
{{/labels}}
{{/labels.length}}
date_saved: {{{dateSaved}}}
{{#datePublished}}
date_published: {{{datePublished}}}
{{/datePublished}}
---

# {{{title}}}
#Omnivore

[Read on Omnivore]({{{omnivoreUrl}}})
[Read Original]({{{originalUrl}}})

{{{ content }}}

It just stops while trying to import, no error, no log - nothing.
From what I am able to work out, it seems the issue lies in the content variable.

What I think is happening is during the replacement of the content in the template. {{{content}}}

There is a video on the page, so it's possibly the video causing the problem.

If I do a full sync without content, and then update and do a full resync, it def gets stuck on that above article.

aaachen commented 11 months ago

similar: https://github.com/omnivore-app/omnivore/issues/2468

mrinc commented 11 months ago

similar: omnivore-app/omnivore#2468

It is not similar. this issue is unrelated to re-sync or syncing updates.

this issue seems to stem from the content generation itself.

sywhb commented 11 months ago

hey @mrinc , i cannot replicate this error on my local machine with the page.

i guess if you could move the front matter section in your template to the "Front Matter Template under advanced settings of the plugin because we have some issue with rendering the front matter template along with the article.

Screenshot 2023-12-21 at 5 18 02 PM
mrinc commented 11 months ago

Hey @sywhb - my front matter template was blank ... not sure if that is an issue?

I have updated it, and the template - all info below to see if you can replicate:

Template:

# {{{title}}}  

[Omnivore]({{{omnivoreUrl}}}) - [Original]({{{originalUrl}}})  

{{#highlights.length}}  
## Highlights  

{{#highlights}}
> {{{text}}} [⤴️]({{{highlightUrl}}}) {{#labels}} #{{name}} {{/labels}}
{{#note}}

{{{note}}}
{{/note}}

{{/highlights}}
{{/highlights.length}}  

{{{content}}}

Front Matter:

id: "{{{id}}}"
from: "omnivore"
status: "{{{state}}}" 
date_published: "{{{datePublished}}}"
date_saved: "{{{dateSaved}}}"
{{#author}}
author: >
  {{{author}}}
{{/author}}
tags:
{{#labels}}
- {{{name}}}
{{/labels}}

Settings:
image
image

Console for logs:

Obsidian Developer Console  
plugin:obsidian-git:42849 loading Obsidian Git plugin  
plugin:obsidian-omnivore:14281 obsidian-omnivore starting sync since: ''  
plugin:obsidian-git:44165 git obsidian message: Everything is up-to-date  

View of the folders/articles synced:
image
Ignore omnbivoreX - it's just the backup.

Articles links: synced: https://piotrminkowski.com/2023/09/29/manage-openshift-with-terraform/ synced: https://techcrunch.com/2023/10/03/cloaked-manages-your-logins-with-proxy-emails-phone-numbers-and-a-built-in-password-manager/
did not sync: https://github.com/warp-tech/warpgate
did not sync: https://www.qovery.com/blog/how-to-achieve-zero-downtime-application-with-kubernetes/

mrinc commented 11 months ago

I am now not sure what could be causing it either :/

Any way to enable/add more debug logging so we can troubleshoot?

Also, if it's easier - I would be happy to jump on a e-meet and then you can directly play... Just let me know.

sywhb commented 11 months ago

Thanks for the details. everything seems good and i still cannot replicate with all the configuration and links provided :/

it looks nothing caused any error from the console logs. let me look into the backend logs and see if i can find anything

mrinc commented 11 months ago

Mmmm balls.

I wonder if it's maybe timing out or something - or even a memory leak.
I have a LOT of links that are being imported - so maybe not the specific link, but the calumniation of the quantity.

If you want me to try something - lemme know and can do so.
I haven't done any dev for obsidian, so not 100% sure where to look for things, but if you can point me in a direction, I can find my way there.