Completed: Use a plugin to override API endpoint

[rickdgeerling]

I wonder if we could use a regular plugin to override the api.obsidian.md endpoint, instead of patching the Obsidian package. A library like https://www.npmjs.com/package/@mswjs/interceptors could handle the plumbing.

If PRs are welcome I might give it a try in the next few weeks

[acheong08]

Hello,

Thanks so much for the idea. I was having a lot of trouble patching & signing the IOS binary.

If PRs are welcome I might give it a try in the next few weeks

I haven't tried my hand at plugins yet so any help is much appreciated. PRs are welcome!

[acheong08]

I'm not a JavaScript dev so there might be a mistake but I don't think mswjs/interceptors can intercept requests made from other JS files. I copied over the example code into the sample plugin and made a few requests. Nothing was console.logged

const interceptor = new ClientRequestInterceptor()

// Enable the interception of requests.
interceptor.apply()

// Listen to any "http.ClientRequest" being dispatched,
// and log its method and full URL.
interceptor.on('request', ({ request, requestId }) => {
  console.log(request.method, request.url)
})

// Listen to any responses sent to "http.ClientRequest".
// Note that this listener is read-only and cannot affect responses.
interceptor.on(
  'response',
  ({ response, isMockedResponse, request, requestId }) => {
    console.log('response to %s %s was:', request.method, request.url, response)
  }
)

[acheong08]

let obsidian_url = localStorage.getItem("obsidian-dev-url");
if (!obsidian_url) {
    window.smalltalk
    .prompt("Obsidian Sync URL", "", "https://api.obsidian.md")
    .then((value) => {
      localStorage.setItem("obsidian-dev-url", value);
    });
}

...

var Ut=localStorage.getItem("obsidian-dev-url")

This patch makes it a lot easier to define & save the endpoint. I'll make an automated workflow that takes the official releases and applies the patch.

I still can't figure out how to do it via plugins though. It seems the plugins have no access to the app.js file

[acheong08]

It's probably possible. I'm just not skilled enough at JavaScript to figure out how

[acheong08]

Edit:

So I got confused reading the JS code.

In app.js

 return (
                  (o = e.url),
                  (a = e.method),
                  (s = e.contentType),
                  (l = e.body),
                  (c = null),
                  s && (c = { "Content-Type": s }),
                  [4, fetch(o, { method: a, headers: c, body: l })]

Which lead me to use FetchInterceptor. However, after using the debugger in Electron, I found that fetch was actually aliased to p.send which uses XMLRequest. XMLHttpRequestInterceptor thus worked

[acheong08]

Hit a snag. Seems to be an upstream issue:

• https://github.com/mswjs/interceptors/issues/405

[acheong08]

interceptor.on("request", ({ request, requestId }) => {
    console.log(request.method, request.url);
    // Replace api url with sync api url
    let url = request.url.replace(
        "https://api.obsidian.md",
        this.settings.SyncAPI
    );
    fetch(url, {
        method: request.method,
        headers: request.headers,
        body: request.body,
    }).then((response) => {
        request.respondWith(response);
    });
});

[acheong08]

import { App, Plugin, PluginSettingTab, Setting, requestUrl } from "obsidian";
import { XMLHttpRequestInterceptor } from "@mswjs/interceptors/XMLHttpRequest";

const interceptor = new XMLHttpRequestInterceptor();

// Enable the interception of requests.
interceptor.apply();

// Remember to rename these classes and interfaces!

interface MyPluginSettings {
    SyncAPI: string;
}

const DEFAULT_SETTINGS: MyPluginSettings = {
    SyncAPI: "https://api.obsidian.md",
};

export default class MyPlugin extends Plugin {
    settings: MyPluginSettings;
    async onload() {
        await this.loadSettings();

        interceptor.on("request", ({ request, requestId }) => {
            console.log(request.method, request.url);
            // Replace api url with sync api url
            let url = request.url.replace(
                "https://api.obsidian.md",
                this.settings.SyncAPI
            );
            // The body is a stream. Finish reading it first
            let reader = request.body?.getReader();
            if (reader) {
                reader.read().then((result) => {
                    let body = result.value;
                    fetch(url, {
                        method: request.method,
                        headers: request.headers,
                        body: body,
                    }).then((response) => {
                        // Remove headers
                        response.headers.forEach((_, key) => {
                            if (key != "content-type") {
                                request.headers.delete(key);
                            }
                        });
                        request.respondWith(response);
                    });
                });
            }
        });

        // This adds a settings tab so the user can configure various aspects of the plugin
        this.addSettingTab(new SampleSettingTab(this.app, this));
    }

    onunload() {}

    async loadSettings() {
        this.settings = Object.assign(
            {},
            DEFAULT_SETTINGS,
            await this.loadData()
        );
    }

    async saveSettings() {
        await this.saveData(this.settings);
    }
}

class SampleSettingTab extends PluginSettingTab {
    plugin: MyPlugin;

    constructor(app: App, plugin: MyPlugin) {
        super(app, plugin);
        this.plugin = plugin;
    }

    display(): void {
        const { containerEl } = this;

        containerEl.empty();

        new Setting(containerEl).setName("Obsidian Sync URL").addText((text) =>
            text
                .setPlaceholder("https://api.obsidian.md")
                .setValue(this.plugin.settings.SyncAPI)
                .onChange(async (value) => {
                    this.plugin.settings.SyncAPI = value;
                    await this.plugin.saveSettings();
                })
        );
    }
}

There was a streaming issue & header issue

[acheong08]

I got it working

[acheong08]

There is a bug where you can't reload the extension or else

• https://github.com/mswjs/interceptors/issues/405 will trigger

[acheong08]

I made a very rough draft of it: https://github.com/acheong08/rev-obsidian-sync-plugin

A lot of buggy/unexpected behavior. A more polished PR is still welcome.

(Sorry, I couldn't sleep so I spent some time poking around)

[acheong08]

• https://github.com/obsidianmd/obsidian-releases/pull/2353

Obsidian Sync is a service we intend to keep first-party only for the foreseeable future.

[rickdgeerling]

That's awesome 😄 I'll check it out next week!

I'm not surprised they denied it though and I don't blame them either. Obsidian is a great note app and they've got every right to monetize it.

[CzBiX]

Hi, I saw your project mentioned by someone in my group. I'm glad there are others out there doing the same thing as me, I'm not alone. Beside the sync server, I happen to have a similar idea to replace the API server with a plugin. My implementation is here. So we can refer to each other's code if needed, haha😀.

[acheong08]

@CzBiX Nice work. I'll refer to it if there is something I'm unable to understand.

Have you had any luck getting your plugin into the community plugin list?

[CzBiX]

No, I didn't even try to submit it. Because this plugin is not necessary for other normal users. It will simply just affect the official revenue.