error during initialization or sync causes pane to be empty or index files to be malformed

[DrAnomalocaris]

Versions Longform version: 2.0.3 Obsidian version: OS [e.g. macOS, Windows, iOS, Android]: Win11 Theme: Other plugins that you think might be relevant here:

Describe the bug Every now and then all the scenes get forgotten,

I just open obsidian and there are no scenes. the index with the scenes are there, and so are the files, but it does not appear in the longform scene tab

[b-camphart]

That's troubling 😅 Couple questions:

1. When you say they're forgotten, what does it look like visually?
2. Can you think of any patterns around when this occurs? Does it happen every time you open obsidian? Or just under certain circumstances?

[DrAnomalocaris]

Hi, it happens sometimes when I open Obsidian.It just looks like I have not started any longform project.   From: B. CamphartSent: 02 November 2023 14:36To: kevboh/longformCc: Albert Lahat; AuthorSubject: Re: [kevboh/longform] Forgetting all the scenes (Issue #212) That's troubling 😅 Couple questions:When you say they're forgotten, what does it look like visually?Can you think of any patterns around when this occurs? Does it happen every time you open obsidian? Or just under certain circumstances?—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***> 

[beryllium]

I may be experiencing the same bug.

Obsidian 1.4.16 on macOS Sonoma (which I just recently upgraded to). Longform 2.0.4.

I have one multiscene story in progress, and at least one shortstory.

When I click the Longform icon in the icon bar, none of them show up.

I tried disabling all of my other installed plugins & restarting Obsidian, but that did not fix it.

Curiously, when I tried to disable the Longform plugin, I got a message saying 'Failed to unload Longform plugin'.

I opened the Developer Tools Console and saw this message on a fresh startup:

Uncaught (in promise) TypeError: Cannot read properties of null (reading 'length')
    at ignoredPatternToRegex (plugin:longform:36334:41)
    at eval (plugin:longform:36281:64)
    at Array.map (<anonymous>)
    at StoreVaultSync.eval (plugin:longform:36281:53)
    at Generator.next (<anonymous>)
    at fulfilled (plugin:longform:28:58)

When I tried unloading the Longform plugin, the 'Failed to unload' message correlated with this console message:

app.js:1 Plugin failure: longform TypeError: this.unsubscribeDraftsStore is not a function
    at StoreVaultSync.destroy (plugin:longform:35973:14)
    at LongformPlugin.onunload (plugin:longform:37708:29)
    at e.unload (app.js:1:716024)
    at e.<anonymous> (app.js:1:2037547)
    at app.js:1:237027
    at Object.next (app.js:1:237132)
    at app.js:1:236048
    at new Promise (<anonymous>)
    at v (app.js:1:235793)
    at e.unloadPlugin (app.js:1:2037452)
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
s @ app.js:1
Promise.then (async)
l @ app.js:1
(anonymous) @ app.js:1
v @ app.js:1
e.disablePlugin @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
v @ app.js:1
e.disablePluginAndSave @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
v @ app.js:1
(anonymous) @ app.js:1
t.setValue @ app.js:1
t.onClick @ app.js:1

Hopefully this helps.

(edited to correct Obsidian version - 1.4.16, not 1.3.16)

[kevboh]

This does help, and seems like a bug. Do either of you have an ignoredFiles entry in your index files?

[beryllium]

I do, yes:

ignoredFiles:
    - META - Characters

(the middle hyphen is part of the filename, not a failed yaml array linewrap)

If I remove the ignoredFiles setting, activating the plugin shows this error:

Failed to open view TypeError: Cannot read properties of null (reading 'sessionGoal')
    at $$self.$$.update (plugin:longform:34131:75)
    at init (plugin:longform:2956:8)
    at new ProjectDetails (plugin:longform:34170:3)
    at Array.create_if_block_4 (plugin:longform:34784:19)
    at Array.create_else_block_2 (plugin:longform:34569:94)
    at Array.create_else_block$1 (plugin:longform:34433:94)
    at create_fragment$2 (plugin:longform:34939:94)
    at init (plugin:longform:2960:37)
    at new ExplorerView (plugin:longform:35023:3)
    at ExplorerPane.eval (plugin:longform:35563:33)
(anonymous) @ app.js:1
v @ app.js:1
t.setViewState @ app.js:1
eval @ plugin:obsidian-kanban:77
o @ plugin:obsidian-kanban:23
(anonymous) @ app.js:1
(anonymous) @ app.js:1
t.onClick @ app.js:1
plugin:longform:36334 Uncaught (in promise) TypeError: Cannot read properties of null (reading 'length')
    at ignoredPatternToRegex (plugin:longform:36334:41)
    at eval (plugin:longform:36281:64)
    at Array.map (<anonymous>)
    at StoreVaultSync.eval (plugin:longform:36281:53)
    at Generator.next (<anonymous>)
    at fulfilled (plugin:longform:28:58)
ignoredPatternToRegex @ plugin:longform:36334
eval @ plugin:longform:36281
eval @ plugin:longform:36281
fulfilled @ plugin:longform:28
Promise.then (async)
step @ plugin:longform:30
fulfilled @ plugin:longform:28
Promise.then (async)
step @ plugin:longform:30
eval @ plugin:longform:31
__awaiter @ plugin:longform:27
discoverDrafts @ plugin:longform:35976
watchProjects @ plugin:longform:37894
postLayoutInit @ plugin:longform:37779
t.onLayoutReady @ app.js:1
eval @ plugin:longform:37681
fulfilled @ plugin:longform:28
Promise.then (async)
step @ plugin:longform:30
eval @ plugin:longform:31
__awaiter @ plugin:longform:27
onload @ plugin:longform:37646
e.load @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1

[kevboh]

@beryllium can you email me (kb@kevinbarrett.org) your entire index file, if you're comfortable doing so? That's probably the fastest way to reproduce this.

[etobias-musiced]

I might be having a similar issue -- I opened Obsidian this morning and longform was no longer recognizing a project. All of my files/scenes are visible in the folder that is used for the project

and opening the the index file in Obsidian looks empty:

Wen I looked inside the Obsidian folder on my computer and opened the index file the file contained all of the information that should be there

I tried closing out and re-opening Obsidian - did not change anything restarted my computer and reopened Obsidian - did not change anything

I do have numerous ignored files that show up in the index file but none of them actually live in the project folder - I think they were all from file name changes that after getting updated Longform kept thinking the original file still existed.

I manually deleted all of the text under ignored files in the index.md file and that did not change anything about the index file opening in obsidian as if nothing was in it other than the metadata

I then went into the "view sync version history" of the index file -- with the view sync version history I could see all of the information that was in the index.md file in the project folder that I opened outside of Obsidian -

But when I attempted to restore that version Obsidian put up a message indicating that was the current version

When I used the "copy to clipboard" option and pasted all of the content into the index.md file in Obsidian the file now contained the information and when I went back to click on the longform plugin the project and all of the scenes were available again.

Any thoughts on how the index file opening within Obsidian and the same index file being opened outside of Obsidian were unsynced and if this could happen again? A little nervous about continuing to use the plugin for this large project.

Luckily the version history was helpful in this case. Sorry if this is in the wrong place in github - it seemed possibly related when I was looking for solutions this morning.

Thank you.

[bhrebec]

I'm having this issue as well. I do have a complicated sync setup, running mainly on windows 11 and syncing via icloud to an ipad. I've been thinking that's the underlying cause of it, but not 100% sure of that, or how it could possibly be happening, because everything else syncs just fine! The plugin seems to work perfectly while I'm using it on both devices, but it resets both scenes and ignoredFiles to empty arrays almost every time obsidian is restarted (but not every single time, sometimes it works correctly.)

Will attempt to reproduce a minimal test case without other plugins to see if I can give some actual useful information, will update if I find anything. I'm still kind of assuming that my goofy sync setup (folder junctions are involved) is my real issue here, but since others are having problems I thought I'd chime in.

In my case, the index file post-issue is literally just:

---
longform:
  format: scenes
  title: Test Note
  sceneFolder: /
  scenes: []
  ignoredFiles: []
---

If that helps in the slightest!

[etobias-musiced]

Just happened to me again - opened up my vault and Longform did not recognize that any longform project existed even though the folder still exists and the index file is there. The stability issue is a little scary - any ideas of what is happening?

[jgharding]

This is now happening to me regardless of sync, I can make a fresh test project, populate it with files etc, and as soon as I close and open Obsidian again all projects and scenes have vanished.

[kevboh]

Do you have errors in the developer console in Obsidian, and if so can you share them here?

It's not a stability issue, for what it's worth: you are not losing data (and Longform very purposefully does not store or alter notes in such a way that the changes can be easily lost). What happens when projects "disappear" is that an index file's longform property is invalid in some way, and Longform has lost track of it. It's still there on disk.

Fundamentally my hope is that the Obsidian team introduces more durable APIs for searching a vault by property. Right now it is error-prone, and very difficult to anticipate all the ways people's notes become malformed.

[kevboh]

@bhrebec it is almost certainly your sync setup; I'm not sure why, but it seems like your index file is being partially written (not by Longform, by icloud, maybe?) during sync, which causes the plugin to try and "fix" the index file. Not a lot I can do with sync issues, unfortunately, as I can't control what the file system does to files on disk.

[Nikko-A]

Hello! I seem to be having the same issue mentioned. Last thing that I could find that triggered this was that I closed Obsidian. No clue what happened. However, I did notice that my index file, upon creation, was different compared to what was shown here. My index file was legitimately empty aside from the property on top. Even when I add my scenes, it remained empty. Could that have been a part of the issue?

[jgharding]

Do you have errors in the developer console in Obsidian, and if so can you share them here?

It's not a stability issue, for what it's worth: you are not losing data (and Longform very purposefully does not store or alter notes in such a way that the changes can be easily lost). What happens when projects "disappear" is that an index file's longform property is invalid in some way, and Longform has lost track of it. It's still there on disk.

Yeah it's bizarre...I don't use the official Obsidian sync, it's just my files in Google Drive.

I've tried the three different methods of storing sessions too, including "as a file in your vault" as it seems conceptually the most robust. The issue seems to actually happen after the Android devices opens and closes Obsidian, but AFTER that, it is intermittent on a Windows devices alone. Sometimes I open it, and suddenly some projects are there and some aren't too...

These are the errors that are popping up for me, it seems something is getting broken in the frontmatter and "format"? Odd that it mentions Kanban AND Longform in the second error too...

plugin:longform:36212 Uncaught (in promise) TypeError: Cannot read properties of null (reading 'frontmatter') at StoreVaultSync.eval (plugin:longform:36212:26) at Generator.next () at eval (plugin:longform:31:71) at new Promise () at awaiter (plugin:longform:27:12) at StoreVaultSync.resolveIfLongformFile (plugin:longform:36210:16) at eval (plugin:longform:35979:75) at Array.map () at StoreVaultSync.eval (plugin:longform:35979:59) at Generator.next () at eval (plugin:longform:31:71) at new Promise () at awaiter (plugin:longform:27:12) at StoreVaultSync.discoverDrafts (plugin:longform:35976:16) at LongformPlugin.watchProjects (plugin:longform:37894:29) at LongformPlugin.postLayoutInit (plugin:longform:37779:14) at t. (app.js:1:1983837) at app.js:1:237258 at Object.next (app.js:1:237363) at a (app.js:1:236081)

app.js:1 TypeError: Cannot read properties of null (reading 'format') at Object.checkCallback (plugin:longform:36702:26) at app.js:1:1773636 at Array.filter () at e.listCommands (app.js:1:1773574) at e.getCommands (app.js:1:2372097) at t.getItems (app.js:1:2373140) at t.getSuggestions (app.js:1:1533151) at t.updateSuggestions (app.js:1:1532414) at t.onOpen (app.js:1:1532256) at t.onOpen (app.js:1:2372955) at e.open (app.js:1:1425669) at e.onOpen (app.js:1:2372027) at Oq (app.js:1:1772390) at e.executeCommand (app.js:1:1773923) at e.eval (plugin:obsidian-kanban:77:14406) at e.o [as executeCommand] (plugin:obsidian-kanban:23:77313) at e.onTrigger (app.js:1:1145316) at e.handleKey (app.js:1:743120) at e.handleKey (app.js:1:743233) at t.handleKey (app.js:1:743442) at e.onKeyEvent (app.js:1:744376)

[jgharding]

As a follow up, after I moved the projects file to a file in the vault it seems to be generally more stable.

on the occasions where all projects appear “gone”, closing and opening Obsidian fixes the error and they all return.

this seems to suggest it simply is not loading the sessions sometimes? They aren’t damaged, it’s just sort of skipping the file?

[kevboh]

Whoa, the kanban intersection is weird.

I'm going to push a small fix to handle obsidian cache misses, which I thought couldn't happen but I guess they can! I still don't think I can fix the actual root issue, however.

[kevboh]

see https://github.com/kevboh/longform/commit/617653515bdbe84d5c27a8d3b810302947e6883a

[kevboh]

I might close this (out of an abundance of hope?) in case this fixes it. Please comment if the next release with this in it (2.0.5) doesn't help!

[ProudBenzene]

@jgharding Hello, I wonder if your problem has been solved after the Longform updates? I've met just the same problem with you(ಥ_ಥ). Once I quit Obsidian and restart it, all my Longform projects were forgotten.

I use Syncthing to keep my Mac, iPad, Windows and Andriod devices synced, but I've added the longform profile into the ignored list:

.obsidian/plugins/longform/data.json

Yet, the problem still happens to me.

I've viewed my index.md in VSCode, and it seems that there were no problems with the format of yaml.

I really don't know how to solve the problem. Everytime I rewrited my projects into Longform and restarted Obsidian, my projects in Obsidian disappear again.

[kevboh]

The data.json file doesn't keep scene order and you should probably sync it.

How do you use syncthing to sync your iPad? I didn't think that was supported.

@ProudBenzene do you have an error in the console? Open developer tools and then restart Obsidian. If you see an error in red, share it here. That's the best way to dig into this sort of thing.

[kevboh]

@etobias-musiced what are you using to sync?

[kevboh]

To summarize, when this happens you should ideally collect and share with me, either here or privately:

1. The contents of the index file for any projects that seem off;
2. Any errors that might be present in dev tools;
3. If whatever you're using for sync has one, the sync log so I can look for index file conflicts;
4. Whatever you were doing right before this happened.

More broadly, there seem to be two issues at play here that end up looking similar but are very different. In the first, there's an error while loading the Longform plugin or parsing through index files, and that error causes projects to fail to load. They're all still there, the pane is just empty.

In the second, some combination of events causes Longform to think you have deleted scenes or moved them or edited the index file directly, and it attempts to correct the situation by updating the index file to reflect what it thinks is the reality of your vault—no scenes. Longform will always try to react to vault changes by keeping the index file up-to-date. Unfortunately, we have no way to know if sync is changing files around on disk vs. you making changes, and if sync is doing something weird like attempting to resolve conflicts by writing out an empty index file Longform can't really do anything about it. Ideally first-party sync would provide a hooks API to tell us when a file is about to updated via sync, or has a conflict, etc., but it does not—it all appears to be file system changes to Longform.

Anyway, if this happens again please try to provide the list above, either here or by email to kb@kevinbarrett.org. Thanks.

[kevboh]

It's also worth reiterating that your scenes themselves are not lost. I'm going to update the issue name to reflect this. Longform doesn't delete scene files automatically.

[kevboh]

Also @etobias-musiced I watched your video. It's a bit zoomed-in, so it's kind of hard to tell what's going on, but I think sync is breaking the index file's YAML frontmatter in some way. Also, when you copy-paste content into the index file it's not going to work how you think as those list items are not in the frontmatter. They have to be between the triple-dashes (---) in the longform.scenes value to be considered scenes. If you feel you must manually edit the index file it's best to do so in something like VSCode—Obsidian will struggle to handle the nesting. You sort of end up doing this at the end, but the YAML indentation is wrong. I can tell you're frustrated in the video, and appreciate it as a form of clarification, but the thing you end up creating in the index file isn't valid YAML and will break the plugin (even more than sync already appears to have broken it).

Are you editing the same Longform projects on two different machines simultaneously? Obsidian Sync is pretty naive: it's best to let it fully sync on one machine before touching any files.

[ProudBenzene]

The data.json file doesn't keep scene order and you should probably sync it.

How do you use syncthing to sync your iPad? I didn't think that was supported.

@ProudBenzene do you have an error in the console? Open developer tools and then restart Obsidian. If you see an error in red, share it here. That's the best way to dig into this sort of thing.

Hello, sorry to get you back so late.

I use an app called Mobius Sync to help me sync things on iPad. It is almost perfect and costs 38 CNY.

Could you please tell me your email? I've recorded a screen of what happened and would like to send it to you. What's more, forgive me, I don't know which information in the console is what you need🥲 If you'd be happy to, could you tell me in detail exactly which part?

[kevboh]

@ProudBenzene kb@kevinbarrett.org. I also use syncthing (and mobius), and wasn't aware you could open obsidian vaults from it. Can you?

[ProudBenzene]

@ProudBenzene kb@kevinbarrett.org. I also use syncthing (and mobius), and wasn't aware you could open obsidian vaults from it. Can you?

Yes, I can use mobius to sync the notes file to the Obsidian folder, very handy!

[wwjCMP]

After syncing, I encountered the same issue.

[drewlyton]

Hey friend, not to pile onto this but I'm also experiencing this issue when I close and reopen Obsidian. My index file loses most of my nested scenes and even if I restore a version of Index.md for my project that I know worked, it just overrides it again with the malformed and missing scenes.

[kevboh]

@drewlyton How do you sync?

[drewlyton]

@drewlyton How do you sync?

I sync with Git

[drewlyton]

Hey @kevboh, I did a reproduction in my personal vault to help give you some more to work off of.

Quitting and relaunching Obsidian is actually not what causes the malformation for me. Instead, it's when longform reinitializes on another machine. Here's how it went using my personal and work computer.

I start out with the malformed Longform project on my personal laptop.

At this point, my data.json file in the longform folder looks like this:

{
  "version": 3,
  "projects": {},
  "selectedDraftVaultPath": "The Enlightened Entrepreneur/The Enlightened Entrepreneur/Index.md",
  "userScriptFolder": null,
  "sessionStorage": "data",
  "sessions": [
    {
      "start": "2024-02-12T20:15:24.953Z",
      "total": 0,
      "drafts": {
        "The Enlightened Entrepreneur/The Enlightened Entrepreneur/Index.md": {
          "total": 0,
          "scenes": {
            "TLDR; (Too Long Didn't Read)": 0,
            "This Might Not Be For You If...": 0,
            "Tell The Truth": 0,
            "Choose Sustainability Over Speed": 0,
            "Ignore the gamblers": 0,
            "Do Things That Don't Scale": 0,
            "Focus on service": 0,
            "Build bridges, not moats": 0,
            "Give It All Away": 0,
            "People Are The Point": 0,
            "Cultivate Legacy": 0,
            "In Practice": 0
          }
        }
      }
    }
  ],
  "showWordCountInStatusBar": false,
  "startNewSessionEachDay": false,
  "sessionGoal": 500,
  "applyGoalTo": "all",
  "notifyOnGoal": false,
  "countDeletionsForGoal": false,
  "keepSessionCount": 30,
  "sessionFile": "longform-sessions.json",
  "numberScenes": true,
  "sceneTemplate": null,
  "workflows": {
    "Default Workflow": {
      "name": "Default Workflow",
      "description": "",
      "steps": [
        {
          "id": "strip-frontmatter",
          "optionValues": {}
        },
        {
          "id": "remove-links",
          "optionValues": {
            "remove-wikilinks": true,
            "remove-external-links": true
          }
        },
        {
          "id": "prepend-title",
          "optionValues": {
            "format": "$3{#} $1",
            "separator": "\n\n"
          }
        },
        {
          "id": "concatenate-text",
          "optionValues": {
            "separator": "\\n\\n---\\n\\n"
          }
        },
        {
          "id": "write-to-note",
          "optionValues": {
            "target": "manuscript.md",
            "open-after": true
          }
        }
      ]
    }
  }
}

And my Index.md for the project looks like this:

---
longform:
  format: scenes
  title: The Enlightened Entrepreneur - Deprogram Yourself From Startup Culture and Build A Better Business
  workflow: Default Workflow
  sceneFolder: /
  scenes:
    - TLDR; (Too Long Didn't Read)
    - This Might Not Be For You If...
    - Tell The Truth
    - Choose Sustainability Over Speed
    - - Ignore the gamblers
    - Do Things That Don't Scale
    - - Focus on service
      - Build bridges, not moats
    - Give It All Away
    - People Are The Point
    - Cultivate Legacy
    - In Practice
  ignoredFiles: []
---

Using the drag and drop window, I re-add and re-order the documents.

My data.json file gets updated to:

{
  "version": 3,
  "projects": {},
  "selectedDraftVaultPath": "The Enlightened Entrepreneur/The Enlightened Entrepreneur/Index.md",
  "userScriptFolder": null,
  "sessionStorage": "data",
  "sessions": [
    {
      "start": "2024-02-12T20:15:24.953Z",
      "total": 563,
      "drafts": {
        "The Enlightened Entrepreneur/The Enlightened Entrepreneur/Index.md": {
          "total": 563,
          "scenes": {
            "TLDR; (Too Long Didn't Read)": 0,
            "Learning things the hard way": 0,
            "Making amends": 0,
            "Keeping a promise": 0,
            "This Might Not Be For You If...": 0,
            "You want to be your own boss": 0,
            "You want to get rich": 0,
            "You want to do what you love": 0,
            "You want to work less": 0,
            "You want to help people": 0,
            "You want to be seen": 563,
            "Tell The Truth": 0,
            "People love liars": 0,
            "Present - don't pitch": 0,
            "Publish your metrics": 0,
            "Be a dreamer, not a visionary": 0,
            "Choose Sustainability Over Speed": 0,
            "Moving fast breaks things": 0,
            "Stay part-time until you have none": 0,
            "Ignore the gamblers": 0,
            "Practice patience": 0,
            "Do Things That Don't Scale": 0,
            "The problem with solutions": 0,
            "Focus on service": 0,
            "Promote democracy": 0,
            "Build bridges, not moats": 0,
            "Give It All Away": 0,
            "Open up": 0,
            "Patents are for trolls": 0,
            "Gift ideas": 0,
            "People Are The Point": 0,
            "Diversify your identity": 0,
            "Show gratitude": 0,
            "Stay friends": 0,
            "Share the struggle": 0,
            "Cultivate Legacy": 0,
            "Teach everything you know": 0,
            "Spiral upwards": 0,
            "In Practice": 0
          }
        }
      }
    }
  ],
  "showWordCountInStatusBar": false,
  "startNewSessionEachDay": false,
  "sessionGoal": 500,
  "applyGoalTo": "all",
  "notifyOnGoal": false,
  "countDeletionsForGoal": false,
  "keepSessionCount": 30,
  "sessionFile": "longform-sessions.json",
  "numberScenes": true,
  "sceneTemplate": null,
  "workflows": {
    "Default Workflow": {
      "name": "Default Workflow",
      "description": "",
      "steps": [
        {
          "id": "strip-frontmatter",
          "optionValues": {}
        },
        {
          "id": "remove-links",
          "optionValues": {
            "remove-wikilinks": true,
            "remove-external-links": true
          }
        },
        {
          "id": "prepend-title",
          "optionValues": {
            "format": "$3{#} $1",
            "separator": "\n\n"
          }
        },
        {
          "id": "concatenate-text",
          "optionValues": {
            "separator": "\\n\\n---\\n\\n"
          }
        },
        {
          "id": "write-to-note",
          "optionValues": {
            "target": "manuscript.md",
            "open-after": true
          }
        }
      ]
    }
  }
}

And my Index.md now reflects the correct structure:

---
longform:
  format: scenes
  title: The Enlightened Entrepreneur - Deprogram Yourself From Startup Culture and Build A Better Business
  workflow: Default Workflow
  sceneFolder: /
  scenes:
    - TLDR; (Too Long Didn't Read)
    - - Learning things the hard way
      - Making amends
      - Keeping a promise
    - This Might Not Be For You If...
    - - You want to be your own boss
      - You want to get rich
      - You want to do what you love
      - You want to work less
      - You want to help people
      - You want to be seen
    - Tell The Truth
    - - People love liars
      - Present - don't pitch
      - Publish your metrics
      - Be a dreamer, not a visionary
    - Choose Sustainability Over Speed
    - - Moving fast breaks things
      - Stay part-time until you have none
      - Ignore the gamblers
      - Practice patience
    - Do Things That Don't Scale
    - - The problem with solutions
      - Focus on service
      - Promote democracy
      - Build bridges, not moats
    - Give It All Away
    - - Open up
      - Patents are for trolls
      - Gift ideas
    - People Are The Point
    - - Diversify your identity
      - Show gratitude
      - Stay friends
      - Share the struggle
    - Cultivate Legacy
    - - Teach everything you know
      - Spiral upwards
    - In Practice
  ignoredFiles: []
---

Now, I commit and push the changes to git and pull them on my work computer using the terminal.

But when I launch Obsidian on my work computer and longform initializes, it has lost the ordering of my multi-file project and prompts me to go through the process again.

My data.json looks like this:

{
  "version": 3,
  "projects": {},
  "selectedDraftVaultPath": "The Enlightened Entrepreneur/The Enlightened Entrepreneur/Index.md",
  "userScriptFolder": null,
  "sessionStorage": "data",
  "sessions": [
    {
      "start": "2024-02-12T20:15:24.953Z",
      "total": 0,
      "drafts": {
        "The Enlightened Entrepreneur/The Enlightened Entrepreneur/Index.md": {
          "total": 0,
          "scenes": {
            "TLDR; (Too Long Didn't Read)": 0,
            "This Might Not Be For You If...": 0,
            "Tell The Truth": 0,
            "Choose Sustainability Over Speed": 0,
            "Ignore the gamblers": 0,
            "Do Things That Don't Scale": 0,
            "Focus on service": 0,
            "Build bridges, not moats": 0,
            "Give It All Away": 0,
            "Open up": 0,
            "People Are The Point": 0,
            "Cultivate Legacy": 0,
            "In Practice": 0
          }
        }
      }
    }
  ],
  "showWordCountInStatusBar": false,
  "startNewSessionEachDay": false,
  "sessionGoal": 500,
  "applyGoalTo": "all",
  "notifyOnGoal": false,
  "countDeletionsForGoal": false,
  "keepSessionCount": 30,
  "sessionFile": "longform-sessions.json",
  "numberScenes": true,
  "sceneTemplate": null,
  "workflows": {
    "Default Workflow": {
      "name": "Default Workflow",
      "description": "",
      "steps": [
        {
          "id": "strip-frontmatter",
          "optionValues": {}
        },
        {
          "id": "remove-links",
          "optionValues": {
            "remove-wikilinks": true,
            "remove-external-links": true
          }
        },
        {
          "id": "prepend-title",
          "optionValues": {
            "format": "$3{#} $1",
            "separator": "\n\n"
          }
        },
        {
          "id": "concatenate-text",
          "optionValues": {
            "separator": "\\n\\n---\\n\\n"
          }
        },
        {
          "id": "write-to-note",
          "optionValues": {
            "target": "manuscript.md",
            "open-after": true
          }
        }
      ]
    }
  }
}

And my Index.md now looks like this:

---
longform:
  format: scenes
  title: The Enlightened Entrepreneur - Deprogram Yourself From Startup Culture and Build A Better Business
  workflow: Default Workflow
  sceneFolder: /
  scenes:
    - TLDR; (Too Long Didn't Read)
    - This Might Not Be For You If...
    - Tell The Truth
    - Choose Sustainability Over Speed
    - - Ignore the gamblers
    - Do Things That Don't Scale
    - - Focus on service
      - Build bridges, not moats
    - Give It All Away
    - - Open up
    - People Are The Point
    - Cultivate Legacy
    - In Practice
  ignoredFiles: []
---

I can confirm that these are the only changes made. The good thing is it seems mostly idempotent? The files lost seem to be similar and the data.json seems to be similar as well.

This makes me wonder if it is a session issue? Like there is state on the other machine that doesn't match and is overriding the state loaded from the data.json and the Index.md? Just a thought.

Hope this helps! I'm actively writing a book and would love for this plugin to work correctly across my machines 🙏

[drewlyton]

Okay, I was actually able to reproduce the same behavior by duplicating my vault on the same machine. Which makes me feel even more there's an internal state issue somewhere.

UPDATE: Sorry to send you on a wild goose chase. This issue was actually cause by the case sensitivity setting in my git repo. The file names in Index.md were lower case, but the files themselves were still upper case because I'm on a case insensitive filesystem.

I set my git.config.ignoreCase to false, removed the duplicate files, and everything worked with Longform!

[kevboh]

This was still fascinating! Glad you found a fix. Your issue does lend some cred to my suspicion that this ends up being a result of the underlying sync system...

[harry-meadows]

I am having this issue as well. I'll attempt to walk through the documentation steps this weekend, but in short: 3 clients, Windows 10, OS X, and IOS using obsidian sync with every client set to sync all possible files, settings and plugins. Longform setup worked fine on windows machine, opened vault on mac and scene listings vanished on both machines. Recreating and importing scenes only works on one machine while obsidian is open. When it's restarted scene listings vanish again.

[harry-meadows]

1. Made new vault
2. installed longform, enabled
3. sync is off
4. created project folder and scenes folder
5. copy and pasted scenes from other vault into scenes.
6. restarted obsidian.
7. behavior so far is as expected.
8. output of index from mac:

longform: format: scenes title: test1 workflow: Default Workflow sceneFolder: scenes scenes:

• 1 The Party - Foundation
• 1.1 Getting Ready
• 1.2 Meeting Adam
• 1.3 After the party Adam's view
• Adams Christmas ignoredFiles: []

9. Setup obsidian sync and left vault open on mac.
10. restarted obsidian on mac, no issues.
11. restarted obsidian on pc, no scenes listed. Output of scenes from PC:

longform: format: scenes title: test1 workflow: Default Workflow sceneFolder: scenes scenes:

• 1 The Party - Foundation ignoredFiles: []

12. Mac now shows same as PC, no scenes. Log from mac: `app.js:1 Obsidian Developer Console

plugin:longform:37909 [Longform] Starting Longform 2.0.7…

app.js:1 Total plugin setup: 22ms

longform: 22ms

plugin:longform:36267 [Longform] Loaded and watching projects. Found 1 drafts in 0.333s.

app.js:1 Measure loop restarted more than 5 times

e.measure @ app.js:1`

[themadder]

Hiya, I don't move my vault around any which way, so I do not Sync in any which way, it stays just where it's at, just how it is, whenever I open obsidian and close obsidian.
However I wanted to try LongForm, so I installed it and went to make a project and my Index files are consistently malformed, they appear with the strange property as others have shared above. The file in the folder reveals it does have yaml formating, my question is, how do I fix this by making the yaml that is written in the Index.MD into Properties in my obsidian. So my issue, the Index is never Indexing, and the sidebar routinely goes blank or I have duplicate links all to the same scene (ie, the list will appear 4 times, so there are 4 links to Scene2, that all lead to the same note).

I know very little, but my working theory is that Obsidian wants to read the properties, the fact that it can't read this wonky big one is the issue for me, so if I could split it into smaller properties, perhaps that would snatch together into a beautiful woven thread rather than a tangled knot. I am thinking Format, Title, Workflow, would be Text or List Property, and sceneFolder would be text string? Scenes and Ignored files would be List? Because as I'm tryin to understand this right now, it's reading Longform as one singular property, whereas it has multiple properties within it. Now I noticed my calendar doesn't do this with its properties, it has a lot of individualistic property settings, which has led to this theory. I can use it with Aprils Automatic Timelines and we're dandy, they've all got their own properties, with prefixes that I can associate with each of them. So, my fantasy calendar has fc-date property, fc-end property, and that can be read and included into the aat-timeline property, am I making sense? But Longform seems to only want to read the current Longform property (which then contains these smaller parts).

My solution theory is that there needs to be lf-format, lf-title, lf-workflow, lf-scenefolder, lf-scenes, lf-ignoredfiles, and these individualistic properties would need to be read by the sidebar, but this theory firmly puts me outside of my depth.

And to repeat, I am not syncing my vault any which way, it stays basically locked onto this desktop home PC. I have removed the synchronization issues from the problem and I still have the problem.

[kevboh]

It's not the shape of the frontmatter that is the issue. I suspect the content is malformed in some way. You can email me (kb@kevinbarrett.org) the index file or post it here. Usually it's a null or empty property.

[AleksVersus]

Hello, and sorry my bad english.

I see this problem, but I don't screened error in console.

My Index.md at time when truble is visibled^

---
longform:
  format: scenes
  title: 09_folder_name
  workflow: Default Workflow
  sceneFolder: source
  scenes:
    - 00_head
  ignoredFiles:
    - 
---

I look other index files and saw that IgnoredFiles was written differently. I fixed IgnoredFiles like this:

---
longform:
  format: scenes
  title: 09_folder_name
  workflow: Default Workflow
  sceneFolder: source
  scenes:
    - 00_head
  ignoredFiles:
    - aaa
---

Problem has gone.

[kevboh]

Yeah, just removed ignoredFiles entirely. The empty list there breaks parsing.

[robotfactory]

I had similar issues using the official Obsidian Sync where my projects would just not load in to the Longform pane. Removing the (empty) ignoredFiles list completely from my frontmatter resolved the issue. I was getting the same "can't read properties of null" mentioned here

[KunckelEDV]

I have the same issue. Obsidian Sync, edited an my phone, went to my Mac - empty. I followed the discussion on GitHub, checked my Index.md, and it contains all of my scenes in the korrekt order - in Longform (on the left) there are only some of them, and in no order. I started with "New scene", kopied a missing scene-name from my Index, pasted, return. Scene is there, complet with contend, but at the bottom of the Longform-list.

[markmit1908]

I am encountering this problem. Using up to date Obsidian [1.7.4] and Longform [2.0.7]. Running on MacOS, IOS (iPad and iPhone) and windows. Using Obsidian sync.

My observation is that the error is more common when i work on longford for a while. I suspect that something happens in the background and longform fails to save the new index file. Index gets out of sync and the long form folder is lost.

I cannot find a way to reconnect longform to the folder. A single broken folder is enough to stop the adding from working with any of the folders in the ./Longform/ folder.