image with attachment link does not work

hermanhel commented 3 months ago

When a page with attachment link is rendered (from graph or link preview), the page would crash, showing Application error: a client-side exception has occurred (see the browser console for more information). In console it says TypeError: Arguments to path.join must be strings

example: https://hermanhel.codeberg.page/braindump-ui/#655c0a49-1b15-431f-ba4d-aebc87b90bd2

context:

It works perfectly in org-roam-ui locally
I think that the probelmatic path.join should be in https://github.com/rasendubi/uniorg/tree/master/packages/uniorg-attach, as in there it is possible to have only number as a node in path /65/5c....

thargosu commented 3 months ago

Hi, Do you still have the issue if you remove the image?

hermanhel commented 3 months ago

remove the image

What do you mean by removing the image? The link or the image file?

My setting is: I have all images under org-roam-directory/img, and I have property dir set to ../img (I have my notes under org-roam-directory/main/xxx.org). And on the errornous page I have an attachment link linking with the filename of the image.

If I delete the attachment link, the error will disappear.

ikoamu commented 3 months ago

@hermanhel Thank you for raising this issue. We hadn't considered attachments, so we'll make the necessary modifications to support them!

My setting is: I have all images under org-roam-directory/img, and I have property dir set to ../img (I have my notes under org-roam-directory/main/xxx.org). And on the errornous page I have an attachment link linking with the filename of the image.

To help with the implementation, could you provide an example of how you're writing the attachment link in your note (org-roam-directory/main/xxx.org)?

hermanhel commented 2 months ago

Sure, I'm using the DIR file property to set a relative path for org-attach attachment files.

This following file is located under [org-roam-directory]/reference/hongCuriositydrivenRedteamingLarge2024.org:

:PROPERTIES:
:ID:       db55e3c7-5de6-44ce-9622-f36f4f79cf67
:mtime:    20240705195014 20240704171107 20240704153414 20240701133407 20240629123438 20240627073912 20240626105746 20240625203758 20240625183547 20240625150959
:ctime:    20240625150959
:ROAM_REFS: @hongCuriositydrivenRedteamingLarge2024
:DIR:      ../img
:END:
#+title: Hong, Zhang-Wei et al::: Curiosity-Driven Red-teaming for Large Language Models
#+filetags: :draft:
The did [[id:b37b6f67-dc9d-4cee-9875-ecbcc7505ceb][reward shaping]] optimized for diversity of learned behaviour.

Specifically, they took a PPO algoirhtm, and replaced the original reward function with their own, adding 2 terms

* Thought
+ They want to see if entropy bonus and novelty score can make it perform better

* reward :ATTACH:
[[attachment:_20240625_203750screenshot.png]]
** entropy bonus
[cite:@zhangEfficientEntropyPolicy2018]
+ Entropy - bits required to encode all possible configurations. The larger entropy, the more diverse possible outcome.
* experiment
Diversity Metric:
+ [[id:3a54a57d-1e65-4555-9ccc-ccbf689f6774][SelfBLEU]] - diversity in the form of text
+ [[roam:BERT-sentence embedding distances]] - diversity in the semantics of text
** results :ATTACH:
[[attachment:_20240629_123654screenshot.png]]

[[attachment:_20240629_123713screenshot.png]]
** my comments on the results
+ they did quite well maintaining the quality while incresing diversity.
+ I don't understand why instruction following is that good. Their explaination is that red LLM suck in instruction-following, so the extra exploring brings it on. i.e. diversity boosted quality.
  + on the contrary, text continuation may be easier for Red LLM, so they don't need much diversity to land on good policy.
* implementation
+ [[id:165ae590-fcb7-46df-9195-f610c187709e][Curiosity-Driven Red-teaming for LLM code]]
* notes
+ [ ] add the math equations
+ [ ] I think I can make the diversity optimization a seperate organization? it is important.

It is with 3 attachment links

[[attachment:_20240625_203750screenshot.png]]
[[attachment:_20240629_123654screenshot.png]]
[[attachment:_20240629_123713screenshot.png]]

As specifed in the property DIR, the directory storing these 3 files is ../img with regard to [org-roam-directory]/reference/hongCuriositydrivenRedteamingLarge2024.org, which is [org-roam-directory]/img, so namely the attachment files are at:

[org-roam-directory]/img/_20240625_203750screenshot.png
[org-roam-directory]/img/_20240629_123654screenshot.png
[org-roam-directory]/img/_20240629_123713screenshot.png

ps: my file structure under org-roam-direcotory looks like this:

[org-roam-directory]/
- main
   - LLM.org
   - red_teaming_LLM.org 
- img
   - a_picture.png 
- reference
   - someReference24.org

ikoamu commented 2 months ago

@hermanhel I tried it in my repository and the attached image was displayed correctly without any errors. I cannot reproduce the issue. My directory structure and files are as follows:

[org-roam-directory]/
- img
  - test.png
- test.org

:PROPERTIES:
:ID:       481E1424-6124-4B93-B60C-C2F5CB54DF10
:DIR:      ./img
:END:
#+title: test
[[attachment:test.png]]

Comparing with your configuration, there are two differences:

Your DIR property for the img directory is ../img
You manage org files in two subdirectories: main and reference

If you don't mind, let me ask a few questions as material for making org-roam-ui work properly in this directory configuration:

Have you set the org-roam-directory input? If so, what have you set it to? (See: https://github.com/ikoamu/publish-org-roam-ui?tab=readme-ov-file#store-org-files-in-subdirectory)
Do you get the same error when using attached images in org files located in the main directory?
Could you explain how to manage org-roam-directory with two or more directories? (I've only used org-roam by storing all org files in a single directory.)

Thank you for your contribution to this project.

hermanhel commented 2 months ago

Sure,

Have you set the org-roam-directory input? If so, what have you set it to? (See: https://github.com/ikoamu/publish-org-roam-ui?tab=readme-ov-file#store-org-files-in-subdirectory) I changed the head of local.sh to the following
ROAM_PATH=/home/linfeng/NoDropbox/Notes/OrgNotes/RoamNotes
ROAM_DB_FILE=org-roam.db
corresponding to my org-roam-directory /home/linfeng/NoDropbox/Notes/OrgNotes/RoamNotes and db file /home/linfeng/NoDropbox/Notes/OrgNotes/RoamNotes/org-roam.db

Do you get the same error when using attached images in org files located in the main directory?

I have just tested this, and I think I have made an mistake here. Only the main/skill.org crashes the client, and some other attachment links just doesn't work, and gives "WOW, an image" text.

Could you explain how to manage org-roam-directory with two or more directories? (I've only used org-roam by storing all org files in a single directory.)

I use 1 org-roam-directory with several subdirectories. The org-roam-directory variable is set to parent directory of main and reference. This usage works out of box(you can have arbitrary directory structures under org-roam-directory, and all org-files will be correctly linked), I snatched my org-roam-capture-templates from Jethro's config(Listing 2).

hermanhel commented 2 months ago

After I tested that the reference/curiosityxxxxxx.org did not crash the client, I went on to inspect further, and here's what I have find out:

the client crash is likely corresponds to the note not being in the database. During the first attempt to add DIR property to every note I have accidentally added empty line to some property drawers, and that causes the property drawer to not be recognized, making the note not in the database. This will give this in org-roam-ui and this in publishe-org-roam-ui, with local.sh.
the attachment link not working is likely because I failed to populate <org-roam-directory>/img. I just checked codeberg repo (commit c264cd5be6) I used to hold it, and it is indeed quite empty. It should be also because of the first attempt to add DIR, in which I did not use (org-attach-set-directory), rather only did textual replacement to property drawer.
relative file:../img/xxx.png, upon set with (org-insert-link), will be automatically converted to absolute path I tried use file:../img/xx.png and it did not work for a while. Also doesn't work is [[../img/xx.png][label]] . The only non-attachment link that works under my test condition is [[../img/xx.png]]

hermanhel commented 2 months ago

After fixing the property drawer issue, the client crash is gone, and after fixing the file path and populating the img/ directory correctly, the attachment links now shows images correctly.

There is however the client crash problem where in org-roam-ui a similar "bad/non-existent-id" node only gives 500 error in node text. It is confusing, I'll open another issue for it.

Thanks @ikoamu and @thargosu for the help! I would have assumed it was some upstream bug if it's only myself :)

hermanhel commented 2 months ago

I just tested with the latest commit and the client crash problem is gone! Instead you have with what seems to be the last accessible version of it?(I removed the property drawer, edited the note text in org, and the edit did not appear in <publish-org-roam-ui-root>/out/notes nor in the web ui.)

I think this is a better behaviour than what's in org-roam-ui (the 500 error), Nice works guys!

ikoamu commented 2 months ago

@hermanhel Thank you for looking into this in detail! I'm very glad to hear that the problem has been resolved. If you have any other problems or feature suggestions in the future, please feel free to open an issue! I appreciate your contributions and feedback to help improve the project.

ikoamu / publish-org-roam-ui

image with attachment link does not work #22