Avoid backend request to lindylearn.io if social features are disabled

[delvinj]

I disabled Private Notes and Social Highlights, but I still see network requests to the lindylearn.io backend every time the extension reformats a page (see screenshot).

Ideally the extension would not make any backend calls if if those features are disabled.

Thank you, Dan J

[phgn0]

Yes I agree there should be no API requests for features that are not enabled. I thought this was the case already but maybe there's a bug when updating the settings. Thank you for reporting this.

Can you please take a screenshot of the toolbar icons in the top right of the unclutted page? Do you still see API requests after enabling & disabling the social highlights, and then reloading the page?

[delvinj]

Toolbar:

I still see the request after enabling+disabling and reloading. These are my settings:

This is the spot where webtools indicates that call originates:

Thank you, Dan J

[phgn0]

Thank you for the info!

To clarify, do you still see the API request after clicking on the social highlights toolbar icon? Meaning does the request happen when there's a line through the "people" icon?

[delvinj]

Sorry for the late response.

No, I do not see any requests to lindylearn.io when the social highlights toolbar icon is disabled and the extension is activated.

[delvinj]

The extension appears to be making a request to api.lindylearn.io for certain top level navigation events. For example, opening a new tab and navigating to craigslist.org produces two requests to lindylearn.io. These requests happen even if the extension has not been activated.

I missed this in the original report because this code runs in a service worker context. I only noticed after seeing more lindylearn.io requests in Fiddler.

This is the code that is responsible:

// The extension fetches this remote configuration text record once to display the number
// of shown social comments per (hashed) URL the user visits.
// lindylearn.io is the official publisher domain for this browser extension.
const staticFileUrl = "https://s3.lindylearn.io/unclutter-url-counts-v2.csv";
// Load the URL counts map to memory for faster lookup
const annotationCounts = {};
async function loadAnnotationCountsToMemory() {
    try {
        const response = await fetch(staticFileUrl);
        const text = await response.text();
        let start = performance.now();
        const lines = text.split("\r\n");
        lines.map((line) => {
            const [hash, count] = line.split(",");
            annotationCounts[hash] = count;
        });
        let duration = Math.round(performance.now() - start);
        console.log(`Loaded ${lines.length} annotation counts to memory in ${duration}ms`);
    }
    catch (err) {
        console.error(`Failed to load URL counts:`, err);
    }
}

[phgn0]

So the file download from s3.lindylearn.io is the only network request you see if the social highlights are disabled? Toggling the toolbar icons carries over to all pages, so there should be no more requests after you disable the feature.

The code you posted should run in the web extension service worker (independently from visited pages), and only when Unclutter updates or is reloaded. It shouldn't run too often also for performance reasons. Do you see this service worker re-starting multiple times?

[delvinj]

@phgn0, yes, I do see the service worker request happen quite often. I am unsure exactly how chrome triggers extension service workers, but it appears to happen for these events:

1. New Tab
2. Close Tab
3. Navigate somewhere via address bar

And again, it doesn't always happen. I am using Fiddler to record the HTTP requests. Would a quick video or screen share help?

Thanks, Dan J

[phgn0]

@delvinj Arg, sorry for forgetting about this. I did see the issue too, and it seems to be directly related to how Manifest v3 handles background service workers: https://bugs.chromium.org/p/chromium/issues/detail?id=1152255

But I just had another idea -- what if we saved the downloaded annotation counts in a persisted IndexedDB database instead of loading them to memory all the time? Then the .csv download would only need to run when the extension updates (about once every week).

Do you want to take a try implementing this? It should only require changes to apps/unclutter/source/background/annotationCounts.ts and we could use the idb-keyval library already used elsewhere in this repo.

[delvinj]

@phgn0 no worries. I agree with your determination of cause and proposed solution.

And I would love to implement this, BUT I can't commit any time until mid-December.

In any event, thank you for the invitation.

Dan J