jfcostello / AnyBlock-To-Markdown

Used LLM to Parse the verbose JSON files that make up AnyType exports and created a script to turn those files into simple Markdown with all of your relation data
23 stars 4 forks source link

Anyblock Exporter

Anyblock Exporter is an open-source tool designed to convert Anytype exports into Markdown format. This tool allows users to easily transform their Anytype content into a more universal and portable format, facilitating content migration, backup, or integration with other Markdown-compatible systems. Note, this tool was orignally built with a goal of exporting my own data, so had not been tested thoroughly and is liable to break or be missing edge cases I didn't encounter. As such, please either send feedback by opening an issue on Github, or, if you want, submit a pull request! Thanks to some great community members it has continued to be iterated upon, so always check back for updates and open an issue here if something isn't working

Table of Contents

Features

Requirements

How to download and use the tool

Step 1: Downloading the files

Option 1: Git clone

Open 'Terminal' Navigate to the folder you want to put the tool in, and then type the following:

   git clone https://github.com/yourusername/anyblock-exporter.git

to open the folder in terminal, type:

   cd anyblock-exporter

Option 2: Download zip

If you're not familiar with Git commands or using the terminal, you may find it easier to just download the zip file Goto https://github.com/yourusername/anyblock-exporter, click the green 'Code' button, then click 'Download ZIP' Extract the zip file to a folder and remember where you put it for later

Step 2: Install Pyton

Install Python: If you don't have Python installed on your computer, follow these steps to install it:

  1. Optional - verify the installation:
    1. Open a terminal or command prompt.
    2. Type python --version and press Enter.
    3. You should see the installed Python version displayed. If you see a version number, Python is installed correctly.

Step 3: Export your Anytype data

  1. In Anytype, export your data using the Anyblock option, selecting the JSON format. You can export your entire space, parts of it, or just one page.
  2. Place all the exported JSON files in the anyblock_files directory of this project (Note, even if you export just one page, you'll end up with hundreds of JSON files and you'll need them all). Ensure all files you want to convert are in the folder rather than in any subfolders.

Step 4: Optional - Customize the config.yaml file

In the /anyblock_exporter/ directory, you'll see a file called config.yaml. Open this in a text editor, one like notepad will do, and you can customize certain aspects of the conversion process. You don't need to use terminal to access this, you can open it in windows explorer or finder or whatever you use, find config.yaml and open it with any old text editor.

To see what each setting does, look below at the Configuration section

Note, this is optional, and you can just run the script with the default settings

Step 5: Run the script

Open a terminal and navigate to the project directory. On windows, to make it easy, you can right click the folder in explorer, select 'open in terminal'. For Mac, you can open the folder in finder, right click and select 'new terminal at folder'.

Once open, just run the script. To do so, type:

python anyblock_exporter.py

If you get an error saying 'command not found: python' some instances, you may need to instead type

python3 anyblock_exporter.py

If you get an error about missing modules, you can install them by typing

pip install -r requirements.txt

OR, if you get an error about pip not being found, you can install the modules by typing

pip3 install -r requirements.txt

And then try running the script again.

The converted Markdown files will be created in the markdown_files directory.

Configuration

config.yaml

The config.yaml file in the /anyblock_exporter/ directory allows you to customize various aspects of the conversion process:

decode_timestamps:

Can be set to yes or no, defaults to yes. If set, it'll try convert timestamps into EST, if no, it'll leave them as unix timestamps

ignored_properties:

This is the most important setting. Put the list of all properties (relations) you want to ignore here. There's alot of metadata, and it can be verbose without this, so default sets a reasonable amount of ignores. But if something is not showing up for you, come here and see if removing some of these will reveal it. Or, if you see Anytype adding weird things to the metadata, you can add them here to ignore them.

input_folder: anyblock_files output_folder: markdown_files

Define the input and output folder for the script. Change it if the mood so takes you.

log_level: INFO log_file: anytype_conversion.log

Sets the log level and name of the file the logs are dumped in, mainly used for troubleshooting. If nothing is going wrong, no need to worry

The exporter supports the following standard Python logging levels, from least to most severe:

turn_relations_into_obsidian_links:

If set to 'none' (or in fact, blank or anything that isn't some or all), all relations are just in plain text - relation: value If set to some, the script will attempt to identify fields that are time/date stamps or free form text. For those fields, it renders in plain text, relation: value. But for the rest, it'll render relation: "[[value]]" which is useful in Obsidian creating links. If you have selects or multi selects, probably use this especially if you may use Obsidian If set to all, it does the same as 'some' except it'll also wrap plain text relations in that link style - but still no dates

Conversion Process

File Naming and Truncation

Relation Handling

Block Conversion

Project Structure

Contributing

Contributions to the Anyblock Exporter are welcome! Please feel free to submit pull requests, create issues or spread the word.

License

None, do whatever you want with this code, so long as you're helping get yours or anyone elses data into a universal format. Submitting PRs here is the best way to do it, but fork this code, modify it, do whatever you want with it. I've basically gotten what I needed from it, so I'd just be happy if it continues to help others,.