Task plugin induce a longer first note opening or switching on obsidian startup android

mathisgauthey commented 1 year ago

Please check that this issue hasn't been reported before.

[X] I searched previous Bug Reports didn't find any similar reports.

Expected Behavior

When I launch my vault without any community plugin, or just with the task plugin disabled, I can open a note directly without loading time.

Current behaviour

Currently, when I launch my vault and open a note or want to switch from the first opened note to another, only the title of the note is changing. The note content takes a good 5s or more to load, even if there's not a single task or task query inside.

Steps to reproduce

To be fair, there just seems to need to have the obsidian task plugin installed as I used the divide and conquer plugin and identified that this behavior is linked to the task plugin on its own.

I do have a vault with many queries on my periodic notes to be honest, but when I asked, the task plugin should only impact queries loading time in note opening, it shouldn't induce a vault first note opening or switching freeze.

Which Operating Systems are you using?

[X] Android
[ ] iPhone/iPad
[ ] Linux
[ ] macOS
[ ] Windows

Obsidian Version

1.4.9 (108)

Tasks Plugin Version

1.11.1

Checks

[X] I have tried it with all other plugins disabled and the error still occurs

Possible solution

The only solution I see is to disable the task plugin. I don't know if this is an expected behavior on obsidian tasks with mobile, but I don't remember having it before a few weeks back.

claremacrae commented 1 year ago

It's not an expected behaviour.

There have been a few similar reports that would be worth your reading, especially the first one.

I don't have an Android device, but many people are using Tasks on Android and reports of slow startups are rare, so it is likely going to be something particular to your vault.

I believe on Android it's possible to check the developer console for warnings, so that would be something to try too.

mathisgauthey commented 1 year ago

It's not an expected behaviour.

There have been a few similar reports that would be worth your reading, especially the first one.

https://github.com/obsidian-tasks-group/obsidian-tasks/issues/2042

https://github.com/obsidian-tasks-group/obsidian-tasks/issues/602

I don't have an Android device, but many people are using Tasks on Android and reports of slow startups are rare, so it is likely going to be something particular to your vault.

I believe on Android it's possible to check the developer console for warnings, so that would be something to try too.

The second link you provided sounds like it has the same issue as I do. I don't have any opened notes with queries while launching and don't know how to get some logging on Android yet but will try.

Do you know how I can send you a 33mb screen recording of 50s so that you could see how it looks like ? Edit : found it, here is the recording

ezgif.com-optimize(1).gif

claremacrae commented 1 year ago

Thanks for the video.

Now please could you do the same thing in a new vault on your phone, and just the Tasks plugin enabled?

mathisgauthey commented 1 year ago

Thanks for the video.

Now please could you do the same thing in a new vault on your phone, and just the Tasks plugin enabled?

Yup, here it is :

ezgif.com-optimize(2).gif

No issues. Just like when I open my own vault without the task plugin, except for the basic obsidian loading that is longer due to the vault size and the other plugins but anyway !

I don't have anything other than images in a specified attachment folder, there's some pdf in here as well but small one's and just a few.

claremacrae commented 1 year ago

Thank you.

So I think that suggests it's some interaction between:

The tasks plugin and...
One or more of the files in your vault....

mathisgauthey commented 1 year ago

I can't seem to find a way to show console logs on Android yet. I don't have the issue on desktop, but maybe some warnings could be useful, I'll check that for now and see where it goes !

claremacrae commented 1 year ago

From Koala on Discord

How to test plugins on Android

I think you need to have ADB on your computer installed (I always had that installed; if the following directions don't work without it, install it and activate it on your phone); on your android phone enable ADB (it's under developer settings [tap the build number three times to get these settings]); then connect the phone to your computer over USB; open chrome (or chromium based browser should also work) and type chrome://inspect in the browser. Open obsidian on your phone, then it should appear in the browser.

You will have the console and the inspector like you have for desktop Obsidian.

And for sending versions of the plugin to android, you can do that over USB or you use obsidian sync to sync community plugins.

mathisgauthey commented 1 year ago

Update here ! I haven't had the chance to connect my android device yet, but I saw that the dataview indexing was pretty long on desktop, and that this could be due to the huge number of icons I had using the icon-folder plugin.

Mayyyybeeee, it could be related, in a way that something tries so hard to access all these small svg files. I will try to delete all these icon pack and use default emojis.

Edit : It helped.

Obsidian Developer Console
plugin:obsidian-auto-link-title:302 loading obsidian-auto-link-title
plugin:oz-clear-unused-images:1 Clear Unused Images plugin loaded...
plugin:dataview:19532 Dataview: version 0.5.56 (requires obsidian 0.13.11)
plugin:file-explorer-note-count:933 loading FileExplorerNoteCount
plugin:find-unlinked-files:703 loading Find orphaned files and broken links plugin
plugin:obsidian-mkdocs-publisher:58 [GITHUB PUBLISHER] v.6.3.2 (lang: en) loaded
plugin:obsidian-mkdocs-publisher:50 [GITHUB PUBLISHER] migrating workflow
plugin:obsidian-mkdocs-publisher:53 migrating token
plugin:obsidian-mkdocs-publisher:50 [GITHUB PUBLISHER] Configuring other repositories
plugin:heatmap-calendar:340 heyoh null
plugin:obsidian-icon-folder:4360 loading obsidian-icon-folder
plugin:nldates-obsidian:9043 Loading natural language date parser plugin
plugin:note-refactor-obsidian:6488 Loading Note Refactor plugin
plugin:obsidian-paste-image-rename:465 Plugin loading: obsidian-paste-image-rename 1.6.1 BUILD_ENV=production
plugin:url-into-selection:690 loading url-into-selection
plugin:shortcuts-extender:83 e
plugin:quickadd:19480 Loading QuickAdd
plugin:quickadd:17480 QuickAdd: (LOG) No migrations to run.
plugin:symbols-prettifier:102 loading symbols prettifier
plugin:obsidian-tasks-plugin:268 loading plugin "tasks"
app.js:1 Error: Plugin "todoist-sync-plugin" is not passing Component in renderMarkdown. This is needed to avoid memory leaks when embedded contents register global event handlers.
    at t.render (app.js:1:1236555)
    at t.renderMarkdown (app.js:1:1236336)
    at eval (plugin:todoist-sync-plugin:20201:35)
    at run (plugin:todoist-sync-plugin:14:12)
    at Array.map (<anonymous>)
    at eval (plugin:todoist-sync-plugin:827:58)
    at flush (plugin:todoist-sync-plugin:494:17)
t.render @ app.js:1
plugin:dataview:12575 Dataview: all 2994 files have been indexed in 3.807s (2994 cached, 0 skipped).

Here you can see my log. Before deleting all the icons folder and using emojis instead, the dataview was loading for 25s approximately and then there was each icon pack loading on their own, this could take more than a minute or two in total, maybe more but a really long time. No performance impact on desktop but it was doing it in the background so maybe on mobile.

Now it doesn't solve the issue completely, the task plugin still seems to induce a little extra time in switching file, but it went from 5-10s to 3s max which is correct with a now reduced to less than 3s loading and a huge vault with almost 3000 files.

TLDR : If you have the icon folder plugin, use emoji, and delete every icon pack from your plugin config and then from your vault directory using your file explorer.

mathisgauthey commented 1 year ago

Alright, today I tried using Obsidian-plugin-group to filter out task plugin and make him start like 5s after loading. Well, it works, but 5s after loading I still get a 5s waiting on switching pages.

I can't escape the issue like that it seems, I'll have to find out why tasks is taking that much time loading.

claremacrae commented 1 year ago

@mathisgauthey Thanks for your experiments. I would really like to help you, but I am somewhat stumped in how to proceed....

Do you wanna set up a session to experiment together and see what we can figure out? If so, send me an email - address is in my GitHub profile...

mathisgauthey commented 1 year ago

I'm still on a road trip right now, so I have not tried to connect my phone yet, I'm sure I'll find some information here.

I can't plan anything just yet, but I'll contact you for sure in the following days if I can't find anything by myself 👍

mathisgauthey commented 1 year ago

It would seem like the file switch can only happen just after the task finished loading the tasks queries when on a note using queries.

My guess would be that when on a file without any query, or when you don't have any open file, the tasks plugins still search for something. But what ? I will try adb in the following days to figure out something 👍

claremacrae commented 1 year ago

Thanks for the update @mathisgauthey, and for your efforts. If you become free to work together on it, I look forward to hearing from you.

claremacrae commented 11 months ago

Hi @mathisgauthey, good to e-meet you last week...

I've a new build of Tasks with lots more logging.

You can download it from here: https://github.com/obsidian-tasks-group/obsidian-tasks/actions/runs/6764254041

The initial comment in this pull request describes how to turn on logging:

2397

So basically if you turn all the info to debug in the new settings and then start Obsidian again, I am hoping there will be enough detail to see where the time is going. Perhaps the differences in time-stamps may help too.

See you Tuesday.

mathisgauthey commented 10 months ago

Here's an update based on what we currently found out :

Start-up file switching that takes ages (5-10s) might be caused by some exclusive obsidian function going on and blocking other obsidian processes before Cache.loadVault(): state = Warm on big vaults, which is hard to reproduce on a test vault. Maybe there's some test vault with many pre-made files existing somewhere, I'm sur it's been done.
File with lots of queries reloading might be caused by obsidian sync that makes tasks redraw every single time. So for now, not having a file with queries opened during the sync process is the option to improve task-plugin experience. But this is not the issue depicted in this thread.

In addition, I found a really old commit explaining the way the Cache.ts was changed, leading to the state in which we are now.

Edit : I managed to get the issue on my computer now, despite being faster we have the slowing down on file switching with only the task plugin activated.

Strangely enough, I notice the same behavior as on mobile, there seems to be a timing on start-up.

What I mean is that if you switch file right after launch, it'll have a delay.
If you wait a little bit, it'll have none.
If you wait more, there is going to be a delay.
If you wait sufficiently, as everything is loaded, there is no delay.

I can't entirely reproduce with certainty, this is just an observation from what I experience during my testings. Speaking of which, I'm starting to dig through some flame charts as well to figure things out.

claremacrae commented 10 months ago

If you want to create a large number of tasks for testing purposes, there is: https://github.com/obsidian-tasks-group/obsidian-tasks/blob/main/scripts/stress_test_obsidian_tasks.py

mathisgauthey commented 10 months ago

I'm doing it on my full vault using a backup for now. It does work well. Learned to use the performance api, and I can now sort of confirm our observations. I'll add that adding some timer on sensitive sections of the code might help with optimizing the way Obsidian behave, using some debug argument on the data.json to activate the timer for example.

private loadVault(): Promise<void> {
        this.logger.debug('Cache.loadVault()');
        performance.mark('Cache.loadVault.start');
        return this.tasksMutex.runExclusive(async () => {
            this.state = State.Initializing;
            this.logger.debug('Cache.loadVault(): state = Initializing'); // ADD START TIMER

            await Promise.all(
                this.vault.getMarkdownFiles().map((file: TFile) => {
                    return this.indexFile(file);
                }),
            );
            this.state = State.Warm;
            // TODO Why is this displayed twice:
            this.logger.debug('Cache.loadVault(): state = Warm');
            performance.mark('Cache.loadVault.end'); // ADD END TIMER
            const measure = performance.measure('Cache.loadVault', 'Cache.loadVault.start', 'Cache.loadVault.end'); // MEASURE ELAPSED TIME

            // Notify that the cache is now warm:
            this.notifySubscribers();
        });
    }

[2023-12-02-10:32:01.174][debug][tasks.Cache] Cache.loadVault(): state = Initializing 
[2023-12-02-10:32:06.088][debug][tasks.Cache] Cache.loadVault(): state = Warm

The markdown page loading is waiting for the obsidian-tasks functionloadVault() to end, at least from what I understand.

mathisgauthey commented 10 months ago

Alright.

As for my progresses, I'd say that the issue is that Obisidian tasks want to be the only one to be able to use getMarkdownFiles(). The issue is that it is required to load a file on the vault.

And as to why it sometimes seems to work, well it's probably these times when obsidian had the time to load the vault before Obsidian :

// Resolved fires on every change.
// We only want to initialize if we haven't already.
if (!this.loadedAfterFirstResolve) {
    this.loadedAfterFirstResolve = true;
    this.loadVault();
}

Hence when switching file, if obsidian tasks already started the load vault, there is no way to switch file. That's at least my observation based on the loadVault() and the flamechart :

To be honest, I don't know how to circumvent that, maybe trying to make sure that obsidian first has the time to lead its things before Obsidian starting to cache things up ? I don't know.

Edit : I managed to reproduce a scenario with no wait time. On a working scenario where there is no issue, the big tasks that load the future file you just switched on is capable of doing it before tasks "takes the control of this function exclusively" :

Maybe some relevant informations can be found in this section of the obsidian dev documentation.

mathisgauthey commented 10 months ago

Just figured that Dataview isn't causing issues for the loading of its cache.

Not saying he's doing the same job, but I guess there's some similarities.

We could maybe get inspired here. If it doesn't have any issue with the way he handles things, we might as well try.

claremacrae commented 10 months ago

Just figured that Dataview isn't causing issues for the loading of its cache.

Thanks. I have a dim recollection that Dataview may use a Web Worker to prevent some of its tasks blocking Obsidian...

obsidian-tasks-group / obsidian-tasks