search

coddingtonbear / obsidian-local-rest-api

Unlock your automation needs by interacting with your notes in Obsidian over a secure REST API.
MIT License
607 stars 56 forks source link

TypeError: argument entity must be string, Buffer, or fs.Stats at etag (plugin:obsidian-local-rest-api:39419:15) #79

Open calmwaves111 opened 11 months ago

calmwaves111 commented 11 months ago

obsidian-web js a brower-plugin：https://github.com/coddingtonbear/obsidian-web now it cannot work well，In the console i see this message:

TypeError: argument entity must be string, Buffer, or fs.Stats
    at etag (plugin:obsidian-local-rest-api:39419:15)
    at generateETag (plugin:obsidian-local-rest-api:40374:16)
    at ServerResponse.send2 [as send] (plugin:obsidian-local-rest-api:43551:20)
    at ServerResponse.json (plugin:obsidian-local-rest-api:43579:19)
    at RequestHandler.returnCannedResponse (plugin:obsidian-local-rest-api:45578:63)
    at RequestHandler.eval (plugin:obsidian-local-rest-api:46188:12)
    at Generator.next (<anonymous>)
    at eval (plugin:obsidian-local-rest-api:79:61)
    at new Promise (<anonymous>)
    at __async (plugin:obsidian-local-rest-api:63:10)

there is more information in this issue：https://github.com/coddingtonbear/obsidian-local-rest-api/issues/80

in fact. I also meet this error in other case：when I click the 【 download this certificate】It will jump to the edge browser and report this error image

coddingtonbear commented 11 months ago

I'm hoping to centralize all of the conversation around this issue here from the various places the conversation has been ongoing including, at least:

So far: I haven't been able to reproduce this on my end, so I don't have a very good idea about where to look. I'm running this plugin in the following environment:

& have tried probably a dozen operations including, at least:

esodesod commented 11 months ago

I can verify that at least Local REST API (1.6.0) is working normally in a "test vault", with the same version of Obsidian (v1.4.15 Insider and installer v1.4.14), and in the same OS/packages, so I guess there is a combination of x and y, causing issues. When time permits, I'll try to narrow it down by "dissecting" plugins, and then maybe notes (which contain metadata).

As mentioned in the discussion https://github.com/coddingtonbear/obsidian-local-rest-api/discussions/78, if I put a line with console.log to output "entity" (like the snippet below, at line 39419), I get the results like in the screenshot below. To me, it seems to be able to get the note (via note recall), but it also throws the error.

if (!isStats && typeof entity !== "string" && !Buffer.isBuffer(entity)) {
    console.log("Entity: " + (entity));
    throw new TypeError("argument entity must be string, Buffer, or fs.Stats");
  }

.. And as mentioned, if I comment out the line throwing the error (39419), note recall at least works as expected again (though it's not a solution)

Screenshot from Obsidian > Developer Tools > Console (cleared first), after a (single) reload on a URL-matching webpage (this GitHub issue), only contained in the note below. Note that it also repeats 3 more times. Maybe this is the error-handling just re-trying.

image

In Developer Tools > Sources, the scope for "etag" seems to be "buffer". I have no idea how to "inspect it further" (tried looking at the memory inspector for the buffer, but I don't see any obvious issues here)

image

I've also tried disabling "Note Recall" (which is the main future I really love and use the most). I then found that just by entering the API key in Obsidian Web, the TypeError is thrown.

I've also been running "insecure" (since only used locally here), but as mentioned in this issue, I also get the TypeError, just by trying to download the certificate:

image

A quick check using openssl, shows some more details. At least the certificate is valid (but not trusted in my browser, since I don't use anymore).

❯ openssl s_client -connect 127.0.0.1:27124
CONNECTED(00000003)
Can't use SSL_get_servername
depth=0 CN = Obsidian Local REST API
verify error:num=18:self-signed certificate
verify return:1
depth=0 CN = Obsidian Local REST API
verify return:1
---
Certificate chain
 0 s:CN = Obsidian Local REST API
   i:CN = Obsidian Local REST API
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Mar 22 13:10:18 2023 GMT; NotAfter: Mar 21 13:10:18 2024 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDPTCCAiWgAwIBAgIBATANBgkqhkiG9w0BAQsFADAiMSAwHgYDVQQDExdPYnNp
ZGlhbiBMb2NhbCBSRVNUIEFQSTAeFw0yMzAzMjIxMzEwMThaFw0yNDAzMjExMzEw
MThaMCIxIDAeBgNVBAMTF09ic2lkaWFuIExvY2FsIFJFU1QgQVBJMIIBIjANBgkq
hkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvQfOBnTm3wcrnO9MsigN4IH+y/SB4vGv
lE5NKdPX0fe1Pr9ThV9rIyX6QYFyDV8DrVrYxusknVlCn4+nSL1gyzdY+aa3rT84
ernFPGT51f1OVu9+ECe4npUnZowo5+zw+8dDVjhmfAavfCPtrLchjuiKFrl7h590
X9qmmGZrpUOA4ibrusWH82QJLXqv5PnSNKXt3FNVzVcNCLjCfuVjpWQfD6MPl1aP
E/Gball8MgH4Bbls7X2xOf7hKDk+UNq/7/TwLWv9IQaQMem1YpiIT6f44d7se8Sv
cPU1fun0Sa116TUfmsS58eGPNzzDwEV0J+PpkGCDqiFk/7Wu8F4XFQIDAQABo34w
fDAMBgNVHRMEBTADAQH/MAsGA1UdDwQEAwIC9DA7BgNVHSUENDAyBggrBgEFBQcD
AQYIKwYBBQUHAwIGCCsGAQUFBwMDBggrBgEFBQcDBAYIKwYBBQUHAwgwEQYJYIZI
AYb4QgEBBAQDAgD3MA8GA1UdEQQIMAaHBH8AAAEwDQYJKoZIhvcNAQELBQADggEB
AAnt6UpMvcyO18ddYn+bvGslExIDOieilgR7M8oxibTOORO7u9DaN9KLasiFEPBB
Ne+7wrYZrax+NQwNFboc27HkIcTbZ0Axe3aI3eMuJh+/oUnEDUER1VVbVSEQoRXx
gi3MWA6aBpuWC5mfnjB9HqaZT31WNKPFqb45KFSEoxX8FpftBGhOoObE1y0SsI+6
XAD5yvnV2RsHfdVMH9X7df4KrDI//QZoaUG09NaKpzmBha6oIyuXo4W+KxoyjLlk
j1CmEydoEjFSst1waXceZ2iQnQe2T5fZvhluQWNq0CisSY3xDlNOr5lN65H8fZDK
KU/lRDu62GLRl1BfuQ//wqQ=
-----END CERTIFICATE-----
subject=CN = Obsidian Local REST API
issuer=CN = Obsidian Local REST API
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 1323 bytes and written 373 bytes
Verification error: self-signed certificate
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
This TLS version forbids renegotiation.
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 18 (self-signed certificate)
---

In my "test vault", I get directly to the "download certificate" (without the error).

PS: I also tried to reset crypto in Local REST API, and reloaded the plugin, but the same issue still occurs.

BTW: I'm no developer, but if you know any "tricks" on how to increase verbosity/logging, etc., I guess I'll manage to implement it.

Since everything is working as expected in my test vault, it will be interesting to see results after trying to dissect a bit more. I'll update when time permits.

calmwaves111 commented 11 months ago

I find https://github.com/Sergei-Korneev/obsidian-local-images-plus , the newest version may break local -rest-api however,Even after I uninstalled local-images-plus, there are still some issues about obsidian-web

esodesod commented 11 months ago

Thanks @calmwaves111 👍 .

I also recently updated local images plus to version 0.15.8 (https://github.com/Sergei-Korneev/obsidian-local-images-plus/releases/tag/0.15.8), so it's something here...

I (also) got it working (fully) after disabling local images plus (0.15.8), and using Obsidian Web in secure/encrypted mode (disabled "Enable Non-encrypted (HTTP) Server). Verified traffic using tcpdump/wireshark (port 27124 being used).

Trying to use non-encrypted, I also got an "ERR" in Obsidian Web. Based on tcpdump, it still tries to use port 27124. Removing the certificates from my "keychain" (not trusted anymore), reloading Local REST API plugin in Obsidian, and restarting the browser, then going into Obsidian Web, removing the API key, then "pressing anywhere" outside the API key field (to save???), and then re-entering the key seems to solve that as well (it now connects non-encrypted, using port 27123). After the steps outlined, I also get the "icon" for non-encrypted.

image

So after disabling local images plus v0.15.8, I got it working again, both in secure and non-secure mode.

esodesod commented 11 months ago

Regarding the ERR in Obsidian Web, I noticed it's a bit "tricky" to change around, e.g. from non-encrypted to encrypted, and vice versa (resulting in "ERR" in Obsidian Web). This might be more related to the Obsidian Web behavior, but leaving my notes here, if it helps:

After successfully being set up using non-encrypted (certificate not imported/trusted), I can verify traffic and connections, using port 27123.

image

If I then import the certificate (into the trusted keychain, for MacOS), restart the browser, and reload the Local REST API plugin, I immediately see the "encrypted icon" from Obsidian Web > Options (without changing anything.. yet).

image

BUT Obsidian Web (in Vivaldi here) still connects to non-encrypted, using port 27123 (see below)

❯ grc sudo lsof -iTCP -n -P|egrep -e '(27123|27124)'
Obsidian  20949 esod   34u  IPv4 0xe161315d72222c05      0t0  TCP 127.0.0.1:27124 (LISTEN)
Obsidian  20949 esod   42u  IPv4 0xe161315d71db0295      0t0  TCP 127.0.0.1:27123 (LISTEN)
Obsidian  20949 esod   48u  IPv4 0xe161315d724defb5      0t0  TCP 127.0.0.1:27123->127.0.0.1:57433 (ESTABLISHED)
Vivaldi   29317 esod   34u  IPv4 0xe161315d7222374d      0t0  TCP 127.0.0.1:57433->127.0.0.1:27123 (ESTABLISHED)

If I then disable non-encrypted by de-selecting "Enable Non-encrypted (HTTP) Server", reload the Local REST API plugin, and restart the browser, I actually get the ERR again (note that certificate is now fully trusted, and should be working)

Checking connections, it seems to establish "something" on port 27124 (maybe the plugin settings, itself?)

❯ grc sudo lsof -iTCP -n -P|egrep -e '(27123|27124)'
Obsidian  20949 esod   34u  IPv4 0xe161315d72e6f74d      0t0  TCP 127.0.0.1:27124 (LISTEN)
Obsidian  20949 esod   42u  IPv4 0xe161315d72842fb5      0t0  TCP 127.0.0.1:27124->127.0.0.1:57649 (ESTABLISHED)
Vivaldi   30447 esod   30u  IPv4 0xe161315d72502fb5      0t0  TCP 127.0.0.1:57649->127.0.0.1:27124 (ESTABLISHED)

BUT the client (Obsidian Web) still also seems to try to connect to port 27123, based on traffic, e.g. this snippet from tcpdump (viewed in wireshark), while using note recall.

image

To "solve it", I have to:

Obsidian Web now connects successfully using encrypted mode, without any ERR.

image

calmwaves111 commented 11 months ago

My problem is that if I open 【Search for previous notes about this page in the background】 in obsidian-web, obsidian-web will show err, But when I click on the icon of obsidian-web, other functions can still be implemented

win 10, obsidian 1.4.14, loacal rest api 1.6.0, edge 117.0.2045.43

This seems to be due to edge, I have everything working fine in chrome after I uninstalled local-images-plus

I don't know what to do because it was working fine a few days ago My guess is that the recent update of edge caused obsidian-web's err.

esodesod commented 11 months ago

My problem is that if I open 【Search for previous notes about this page in the background】 in obsidian-web, obsidian-web will show err, But when I click on the icon of obsidian-web, other functions can still be implemented

Basically what I noticed as well (but MacOS and Vivaldi). Did you try my "solution" mentioned above (re-entering the API Key, after reloading Local REST API plugin AND reloading your browser)? At least that worked for me, both going from encrypted > non-encrypted, and from non-encrypted to encrypted. If not, I'd spin up a tcpdump, just to verify that (all) traffic was going as expected (based on your settings, that is).

Anyways, I guess this is "out of scope" for this issue, and more related to how Obsidian Web (re)connects to the Local REST API.

calmwaves111 commented 11 months ago

My problem is that if I open 【Search for previous notes about this page in the background】 in obsidian-web, obsidian-web will show err,我的问题是，如果我在obsidian-web中打开【在后台搜索有关此页面的以前的笔记】，obsidian-web会显示err， But when I click on the icon of obsidian-web, other functions can still be implemented 但是当我点击obsidian-web的图标时，其他功能仍然可以实现

Basically what I noticed as well (but MacOS and Vivaldi). Did you try my "solution" mentioned above (re-entering the API Key, after reloading Local REST API plugin AND reloading your browser)? At least that worked for me, both going from encrypted > non-encrypted, and from non-encrypted to encrypted. If not, I'd spin up a tcpdump, just to verify that (all) traffic was going as expected (based on your settings, that is).基本上我也注意到了（除了 MacOS 和 Vivaldi）。您是否尝试过我上面提到的“解决方案”（在重新加载本地 REST API 插件并重新加载浏览器后重新输入 API 密钥）？至少这对我有用，无论是从加密>非加密，还是从非加密到加密。如果没有，我会启动 tcpdump，只是为了验证（所有）流量是否按预期进行（即根据您的设置）。

Anyways, I guess this is "out of scope" for this issue, and more related to how Obsidian Web (re)connects to the Local REST API.无论如何，我想这“超出了这个问题的范围”，并且与 Obsidian Web 如何（重新）连接到本地 REST API 更相关。

yes, I create a new ob vault, then installed the local rest api plugin, and then connected the key of this new vault. Then I changed the key back to the old valut, and something magical happened. I was able to connect to the old vault again.

The problem of this issue should only be related to the update of local image plus. maybe I have to propose an issue with local image plus.

esodesod commented 11 months ago

something magical happened. I was able to connect to the old vault again.

Great! As mentioned, it seems a bit tricky to "save changes" in Obsidian Web sometimes, hence my little "remove API key, click somewhere, and re-apply the very same API key"-trick. I know the extension is mainly for Chrome, so could be Vivaldi-related (in my case, at least).

maybe I have to propose an issue with local image plus.

Please do! I'll follow, and assist (more) if possible 👍 . For now, I've just pinned version Local Images Plus on 0.15.7 using Obsidian BRAT, as the issues with Local REST API only seems to occur in combination with the latest version (0.15.8).

x012 commented 11 months ago

Disabling 'Local Images Plus' and restarting Obsidian also did the trick for me! Thanks guys!

Sergei-Korneev commented 11 months ago

It is something related to the http etags, I guess. You are very welcomed to investigate and propose a patch (to this software or mine ).

codecleric commented 11 months ago

My setup is very similar to @coddingtonbear (thank you for this plugin, BTW), Ubuntu 23.04, Local Rest API 1.6.0, but my Obsidian appimage is 1.4.16. This same problem started for me and it's not only this plugin, but the Telegram sync plugin can no longer function, and it has a similar error: Telegram Sync: Error with user connection -> TypeError: Argument must be a Buffer.

I'll try the suggestions with Local Images Plus and report back.

Yes, in fact Local Rest API and Telegram Sync are once again working, so that plugin seems to be the culprit.

esodesod commented 11 months ago

We're still discussing possible changes/compatibility issues at https://github.com/Sergei-Korneev/obsidian-local-images-plus/issues/47

The issue is possibly related to the Buffer.isBuffer(entity) part on line 39418 (as mentioned earlier, part of the etag function), and something introduced related to buffer/entity in 0.15.8 version of Local Images Plus, that does not play well with Local REST API (well, it throws the error), and possibly other plugins as well (like Telegram, mentioned by @codecleric ).

Again, I'm no developer (which I guess should be a "prerequisite" for troubleshooting the code here 🙈), but I love both plugins, hence wanting to help, as much as I can.

I don't know if we should keep the issue open here (until fixed), but I guess @coddingtonbear can decide.

coddingtonbear commented 11 months ago

I don't mind keeping the issue open so folks can find it, but from what I can read on https://github.com/Sergei-Korneev/obsidian-local-images-plus/issues/47, it does look like this is, unfortunately, a problem that would need to be fixed there rather than here.

For those looking for a TLDR: if I understand the conversation there -- and, please, correct me if I don't -- the problem is rooted in a dependency used by obsidian-local-images-plus. That plugin is dependent on a package named jimp, and that package includes a hack that overwrites a method (Buffer.isBuffer) that is used by other libraries, including libraries used by this one.

@57r31 has posted a pull request https://github.com/Sergei-Korneev/obsidian-local-images-plus/pull/48/files#diff-4fab5baaca5c14d2de62d8d2fceef376ddddcc8e9509d86cfa5643f51b89ce3dR1024 that corrects the changes made by jimp such that the implementation of Buffer.isBuffer is closer to what it was before (thanks for your effort, @57r31!), but unfortunately, there's some skepticism about whether that's the right approach given that, although the solution works, it is layering a new hack on top of the problematic one. Speaking from experience, although I think all of us would like to get to a solution as quickly as possible, hacks of this sort are one of the things that make open-source projects so hard to maintain in the long-term, and I can understand why it wasn't merged without further conversation.

Unfortunately @Sergei-Korneev is too busy right now to help with this further, and, honestly, I can relate to that! Please try to keep in mind that those of us who work on open-source projects like this one are doing so as volunteers in our free time because we enjoy the creative pursuit and could just as easily decide to be spending our time drinking margaritas on a beach or playing video games. How you engage with folks who work on open-source projects impacts how much those folks continue to enjoy it... and you're dependent on them enjoying the work for them to do the work.

If you don't think you have software skills -- you probably have more than you think! You're on github right now, and this stuff isn't as complicated as you might imagine; give it a shot! To help out, I think there look to be a few different things you might be able to do:

Good luck and, hopefully, have fun!