slnsys / obsidian-canvas2document

Plugin for Obsidian to convert a complete Canvas to a long form document
https://obsidian.md/plugins?id=canvas2document
MIT License
29 stars 0 forks source link

Thank you very much for your plugin. I was wondering if it would be possible to add a third step, which is to import the md document into the canvas to replace the original card. This plugin can provide links for cards in the canvas: https://github.com/Quorafind/Obsidian-Link-Nodes-In-Canva #5

Open CamWam opened 6 months ago

CamWam commented 6 months ago

Thank you very much for your plugin. I was wondering if it would be possible to add a third step, which is to import the md document into the canvas to replace the original card. This plugin can provide links for cards in the canvas: https://github.com/Quorafind/Obsidian-Link-Nodes-In-Canvas

slnsys commented 6 months ago

Hello, to further evaluate your idea, could you please specify a bit more in detail? Which md document should get imported? The generated md documents (s. step 1 and step 2) are put next to the original canvas.

slnsys commented 6 months ago

oh, i got it now. You mean, referencing back from the new document to the originating canvas and its nodes? Like in this feature request (if it's not your's :-) https://forum.obsidian.md/t/feature-request-for-link-to-a-block-in-a-note-within-canvas/81457

But it seams, when checking and testing the linking-syntax "[[...#...", "[[...^...", there is not yet support for the elements of canvases.

CamWam commented 6 months ago

oh, i got it now. You mean, referencing back from the new document to the originating canvas and its nodes? Like in this feature request (if it's not your's :-) https://forum.obsidian.md/t/feature-request-for-link-to-a-block-in-a-note-within-canvas/81457

But it seams, when checking and testing the linking-syntax "[[...#...", "[[...^...", there is not yet support for the elements of canvases.

Thank you very much for your reply! Yes. Sorry, my previous statement was too vague, to the point of misstating the steps. My idea is the first step is to extract all the cards from Canvas and import them into a folder, with each card corresponding to an md file. The second step is to change the content of the corresponding cards in Canvas (without changing their positions) to the md text from the first step (i.e., turn the card into markdown). The third part is an update feature, for example, if you create new cards and update the original card content in the Canvas document from the second step, then update the corresponding markdown files (and create new markdown files). However, generally speaking, documents that reference .md should be able to be updated directly. Why do this? Mainly because of the problem I raised in this post https://forum.obsidian.md/t/i-have-tested-the-13900k-3090-7950x-4090-and-7950x3d-4090-and-there-is-a-serious-issue-with-canvas-performance-no-configuration-can-smoothly-run-a-canvas-card-with-26-000-characters-of-content/81855. The performance of Canvas is too far behind, and I feel that your plugin has great potential to solve this problem

Your plugin provides another way of generating inspiration. The previous method was to generate separate markdown files and then integrate them on a canvas. However, you have provided content generation on the canvas (the 2D browsing method on the canvas makes it easier for people to get inspired) and then to markdown files (offering two modes of editing: interactive information editing and focused information editing). Your current plugin just lacks a synchronization feature, which you have already accomplished in the first step. If you implement the second step…”

CamWam commented 6 months ago

Hello, I’ve added an example and found that it doesn’t need to be so complicated. For instance, we have a Canvas file, canvas1, and then we use your plugin “Convert canvas to a longform document,” which is the first step function you’ve already implemented. This way, we get the md file for each card.

In the second step, we copy the canvas1 file, and we find that its content is:

{
    "nodes": [
        {"id": "b8bd6e6d50f32b7d", "x": -340, "y": -340, "width": 1000, "height": 640, "type": "text", "text": "# test1\nfjaskldjasda\nadkajda\nasda"},
        {"id": "4c8676d53fe2d5c5", "x": -340, "y": 400, "width": 1000, "height": 1160, "type": "text", "text": "# test 2\nsfkasjldkf \nsdfklafj \nadklfjla "}
    ],
    "edges": []
}

Then we only need to replace"type": "text", "text": "# test1\nfjaskldjasda\nadkajda\nasda" with the corresponding md file. "type": "file", "file": "Card Replacement/canvas 2_canvas2doc-data/newdoc-node_b8bd6e6d50f32b7d_fromCanvas.md"

Like this

{
    "nodes":[
        {"id":"b8bd6e6d50f32b7d","x":-340,"y":-340,"width":1000,"height":640,"type":"file","file":"卡片更换/canvas 2_canvas2doc-data/newdoc-node_b8bd6e6d50f32b7d_fromCanvas.md"},
        {"id":"4c8676d53fe2d5c5","x":-340,"y":400,"width":1000,"height":1160,"type":"file","file":"卡片更换/canvas 2_canvas2doc-data/newdoc-node_4c8676d53fe2d5c5_fromCanvas.md"}
    ],
    "edges":[]
}

This way, we can replace all the card contents in the canvas with md files without changing their positional relationships. However, if a third step update function is added, we can no longer create new files. Instead, we should determine if the type is “text,” then we need to convert it into an md file using the first step, and then replace it in the original canvas file.

slnsys commented 6 months ago

yes, for its first conversion step canvas2document itself needs to convert all card nodes to embedded files. It does this for capsulating the structural content in these cards and have card level navigational headers to rearrange the doc. So it already fulfills your requirements?

CamWam commented 5 months ago

yes, for its first conversion step canvas2document itself needs to convert all card nodes to embedded files. It does this for capsulating the structural content in these cards and have card level navigational headers to rearrange the doc. So it already fulfills your requirements?

Hello, I’m sorry for not responding promptly as I have been ill these past few days. I think an additional step could be added. Previously, it was only about converting each card in the canvas into a .md document, and then converting the entire canvas (cards) into a single .md document, turning it from two-dimensional into a linear document. My idea is Canvas(cards) → Canvas(.md), which means it’s still within the Canvas, but each card becomes an embedded form of a .md document, rather than the original card. This can solve the problem of Canvas not being able to use the ^abc tag. It can also alleviate the performance of Canvas. For example, for longer cards, we can edit directly in the .md file.

PS:At the same time, there is another issue with the future updates of the cards. If new cards are added to the canvas, we would need to be able to convert again, turning the unconverted cards into .md format. However, we should not create a new canvas but update on the original canvas. So, the overall steps are roughly as follows:

  1. for the original canvas, action 1 is to convert each card into a .md file (action 1 includes integrating them into a single .md document).
  2. Action 2 is to create a new canvas, select the information of each card from the original canvas, and replace the content after “type”: with the corresponding .md document.
  3. Then, in this new canvas, update new cards with action 3 (which is different from action 1+action 2, not creating a new canvas, but directly replacing cards with .md).
slnsys commented 5 months ago

Can look at it next week, as i'm on a journey. Canvas2document is able to convert everything in a canvas and it could get some extra functions for that.

CamWam commented 5 months ago

Referen

Thank you very much for your selfless contribution. Your plugin is really great and even solves many design flaws in the Obsidian canvas. Wishing you a happy life.

CamWam commented 5 months ago

Hello, I have written a Python script to implement the replacement from ‘card’ to ‘md’, but I have not yet added the update functionality. I hope it will be useful to you.

import json
import os

def find_obsidian_root(current_dir):
    # Search upwards from the current directory until finding the root directory containing .obsidian folder
    while True:
        if os.path.exists(os.path.join(current_dir, '.obsidian')):
            return current_dir
        # Get the parent directory
        current_dir = os.path.dirname(current_dir)
        # Stop searching if reached the root directory
        if current_dir == os.path.dirname(current_dir):
            raise FileNotFoundError("Root directory containing .obsidian folder not found")

def replace_canvas_content():
    # Get the directory where the script is located
    script_dir = os.path.dirname(os.path.abspath(__file__))
    print(f"Script directory: {script_dir}")

    # Find the root directory containing .obsidian folder
    obsidian_root = find_obsidian_root(script_dir)
    print(f"Found .obsidian root directory: {obsidian_root}")

    # Get all .canvas files in the script directory
    canvas_files = [f for f in os.listdir(script_dir) if f.endswith('.canvas')]
    print(f"Found .canvas files: {canvas_files}")

    for canvas_file in canvas_files:
        print(f"\nProcessing file: {canvas_file}")

        canvas_file_path = os.path.join(script_dir, canvas_file).replace("\\", "/")
        base_name = os.path.splitext(canvas_file)[0]
        base_path = os.path.join(script_dir, f"{base_name}_canvas2doc-data").replace("\\", "/")

        # Calculate relative path
        relative_base_path = os.path.relpath(base_path, obsidian_root).replace("\\", "/")

        print(f"Relative directory path: {relative_base_path}")

        # Read Canvas file content
        with open(canvas_file_path, 'r', encoding='utf-8') as file:
            canvas_data = json.load(file)
        print(f"Read file content: {canvas_data}")

        # Iterate through each node in the Canvas file, find and replace content
        for node in canvas_data.get("nodes", []):
            print(f"Processing node: {node}")

            if node.get("type") == "text":
                node_id = node["id"]
                new_file_path = os.path.join(relative_base_path, f"newdoc-node_{node_id}_fromCanvas.md").replace("\\", "/")
                node["type"] = "file"
                node["file"] = new_file_path
                # Delete the original "text" field
                del node["text"]

                print(f"Modified node: {node}")

        # Write the modified content back to a new Canvas file
        new_canvas_file_path = os.path.join(script_dir, f"{base_name}_modified.canvas").replace("\\", "/")
        with open(new_canvas_file_path, 'w', encoding='utf-8') as file:
            json.dump(canvas_data, file, ensure_ascii=False, indent=4)

        print(f"Content successfully replaced and saved to: {new_canvas_file_path}")

# Run the script
replace_canvas_content()
CamWam commented 5 months ago

This is much better now, with the added update feature, all text in the original canvas file or the new canvas file can be replaced with file. Then, the updated content will only be on the modified version.

import json
import os

def find_obsidian_root(current_dir):
    # Search upwards from the current directory until the root directory containing the .obsidian folder is found
    while True:
        if os.path.exists(os.path.join(current_dir, '.obsidian')):
            return current_dir
        # Get the parent directory of the current directory
        current_dir = os.path.dirname(current_dir)
        # Stop searching if the root directory is reached
        if current_dir == os.path.dirname(current_dir):
            raise FileNotFoundError("Could not find the root directory containing the .obsidian folder")

def load_canvas_data(file_path):
    with open(file_path, 'r', encoding='utf-8') as file:
        return json.load(file)

def save_canvas_data(file_path, data):
    with open(file_path, 'w', encoding='utf-8') as file:
        json.dump(data, file, ensure_ascii=False, indent=4)

def update_modified_canvas_with_new_nodes(original_canvas, modified_canvas, base_path, relative_base_path, obsidian_root):
    modified_node_ids = {node["id"] for node in modified_canvas.get("nodes", [])}
    for node in original_canvas.get("nodes", []):
        if node["id"] not in modified_node_ids:
            print(f"Syncing new node: {node}")
            if node.get("type") == "text":
                node_id = node["id"]
                new_file_path = os.path.join(base_path, f"newdoc-node_{node_id}_fromCanvas.md").replace("\\", "/")

                # Create the markdown file and copy content if it does not exist
                if not os.path.exists(new_file_path):
                    with open(new_file_path, 'w', encoding='utf-8') as md_file:
                        md_file.write(node["text"])

                node["type"] = "file"
                node["file"] = os.path.relpath(new_file_path, obsidian_root).replace("\\", "/")
                del node["text"]

                print(f"Synced node: {node}")

            modified_canvas["nodes"].append(node)

def replace_canvas_content():
    # Get the directory of the script
    script_dir = os.path.dirname(os.path.abspath(__file__))
    print(f"Script directory: {script_dir}")

    # Find the root directory containing the .obsidian folder
    obsidian_root = find_obsidian_root(script_dir)
    print(f"Found .obsidian root directory: {obsidian_root}")

    # Get all .canvas files in the script directory
    canvas_files = [f for f in os.listdir(script_dir) if f.endswith('.canvas') and not f.endswith('_modified.canvas')]
    print(f"Found .canvas files: {canvas_files}")

    for canvas_file in canvas_files:
        modified_canvas_file = canvas_file.replace('.canvas', '_modified.canvas')

        # Skip processing if the modified file exists
        if os.path.exists(os.path.join(script_dir, modified_canvas_file)):
            print(f"Found modified file: {modified_canvas_file}, skipping {canvas_file}")
            continue

        print(f"\nProcessing file: {canvas_file}")

        canvas_file_path = os.path.join(script_dir, canvas_file).replace("\\", "/")
        base_name = os.path.splitext(canvas_file)[0]
        base_path = os.path.join(script_dir, f"{base_name}_canvas2doc-data").replace("\\", "/")

        # Skip processing if the corresponding folder does not exist
        if not os.path.exists(base_path):
            print(f"Warning: Folder {base_path} does not exist, skipping {canvas_file}")
            continue

        # Read the contents of the canvas file
        canvas_data = load_canvas_data(canvas_file_path)
        print(f"Read file content: {canvas_data}")

        # Calculate the relative path
        relative_base_path = os.path.relpath(base_path, obsidian_root).replace("\\", "/")
        print(f"Relative folder path: {relative_base_path}")

        # Iterate through each node in the canvas file and replace content
        for node in canvas_data.get("nodes", []):
            print(f"Processing node: {node}")

            if node.get("type") == "text":
                node_id = node["id"]
                new_file_path = os.path.join(base_path, f"newdoc-node_{node_id}_fromCanvas.md").replace("\\", "/")

                # Create the markdown file and copy content if it does not exist
                if not os.path.exists(new_file_path):
                    with open(new_file_path, 'w', encoding='utf-8') as md_file:
                        md_file.write(node["text"])

                node["type"] = "file"
                node["file"] = os.path.relpath(new_file_path, obsidian_root).replace("\\", "/")
                # Delete the original "text" field
                del node["text"]

                print(f"Modified node: {node}")

        # Save the modified content back to a new canvas file
        modified_canvas_file_path = os.path.join(script_dir, modified_canvas_file).replace("\\", "/")
        save_canvas_data(modified_canvas_file_path, canvas_data)

        print(f"Content successfully replaced and saved to: {modified_canvas_file_path}")

    # Process all _modified.canvas files
    modified_canvas_files = [f for f in os.listdir(script_dir) if f.endswith('_modified.canvas')]
    for modified_canvas_file in modified_canvas_files:
        print(f"\nProcessing modified file: {modified_canvas_file}")

        modified_canvas_file_path = os.path.join(script_dir, modified_canvas_file).replace("\\", "/")
        base_name = modified_canvas_file.replace('_modified.canvas', '')
        base_path = os.path.join(script_dir, f"{base_name}_canvas2doc-data").replace("\\", "/")
        original_canvas_file_path = os.path.join(script_dir, f"{base_name}.canvas").replace("\\", "/")

        # Read the contents of the modified canvas file
        modified_canvas_data = load_canvas_data(modified_canvas_file_path)
        original_canvas_data = load_canvas_data(original_canvas_file_path)
        print(f"Read file content: {modified_canvas_data}")

        # Calculate the relative path
        relative_base_path = os.path.relpath(base_path, obsidian_root).replace("\\", "/")
        print(f"Relative folder path: {relative_base_path}")

        # Sync new nodes
        update_modified_canvas_with_new_nodes(original_canvas_data, modified_canvas_data, base_path, relative_base_path, obsidian_root)

        # Iterate through each node in the modified canvas file and replace content
        for node in modified_canvas_data.get("nodes", []):
            print(f"Processing node: {node}")

            if node.get("type") == "text":
                node_id = node["id"]
                new_file_path = os.path.join(base_path, f"newdoc-node_{node_id}_fromCanvas.md").replace("\\", "/")

                # Write the text content to a new markdown file
                if not os.path.exists(new_file_path):
                    with open(new_file_path, 'w', encoding='utf-8') as md_file:
                        md_file.write(node["text"])

                node["type"] = "file"
                node["file"] = os.path.relpath(new_file_path, obsidian_root).replace("\\", "/")
                # Delete the original "text" field
                del node["text"]

                print(f"Modified node: {node}")

        # Save the modified content back to the modified canvas file
        save_canvas_data(modified_canvas_file_path, modified_canvas_data)

        print(f"Content successfully replaced and saved to: {modified_canvas_file_path}")

# Run the script
replace_canvas_content()