[C4GT Community]: Markdown Enrichment Service using frontend JS and Neo4j

[KDwevedi]

Description:

This project aims to build a front-end application using HTML/CSS + JS. The application will enrich markdown documents by automatically inserting links to entities within a knowledge graph that share the same name as the keywords within the markdown. Additionally, the app will provide a visualization of the knowledge graph. The knowledge graph is built on the Neo4j platform and interacted with using Cypher queries. This functionality mirrors how Wikipedia links keywords to corresponding articles, thereby enhancing the informational value and navigability of markdown documents.

Goals:

• [x] Develop a front-end application (HTML/CSS + Vanilla JS is sufficient).
• [x] Integrate markdown processing to dynamically insert links matching keywords with knowledge graph entities.
• [x] Implement knowledge graph visualization within the application.
• [x] Implement Markdown rendering within the application.
• [x] Implement interaction between the graph and Markdown visualisations (Clicking on linking in markdown should zoom in on relevant graph node).
• [ ] Include comprehensive unit and integration tests to ensure reliability.
• [ ] Provide detailed documentation on setup and usage for the application.

Background:

A knowledge graph organizes and integrates information into an interconnected graph structure, facilitating better data querying and visualization. Renowned examples include Wikidata and Google's Knowledge Graph in search results. These graphs power enriched user experiences by dynamically linking related information, as seen in Wikipedia's hyperlinked content.

Milestones:

Milestone 1: Project Setup

Set up the project repository including any relevant frameworks utilised.

Milestone 2: Neo4j Database Interface Setup

Implement the interface for interacting with the Neo4j database using its JavaScript driver. Design and document the Cypher query templates for searching and retrieving node data.

Milestone 3: Setting Up Dummy Database and Markdown Data for Testing

Create a dummy Neo4j database with a variety of nodes and relationships for testing. Prepare sample markdown documents containing potential keywords for linking.

Mile Milestone 4: Markdown Parsing and Formatting

Develop functionality to parse markdown documents. Identify potential keywords within the markdown that could match entities in the knowledge graph.

Milestone 5: Enriching Markdown with Links

Implement the logic to enrich identified keywords in markdown with dynamic links pointing to entities in the knowledge graph.

Milestone 6: Integration of Neo4j Visualization Tools

Integrate Neo4j graph visualization tools into the application. Custom coloring of nodes based on node labels.

Milestone 7: Dynamically Updating Visualization

Develop functionality to dynamically update the visualization when new data is added or existing data is updated in the graph database.

Milestone 8: Interactivity Between Markdown and Graph Visualization

Implement interactivity where clicking on linked text in the markdown updates the graph visualization to highlight or bring up relevant nodes.

Milestone 9: Basic Search Functionality on Markdown Side

Add a basic search feature to find keywords within the markdown documents quickly.

Milestone 10: Comprehensive Testing and Documentation

Develop comprehensive unit and integration tests for each component. Document the setup, usage, and any relevant configurations for the application.

Expected Outcome:

The outcome will be an interactive front-end application capable of enhancing markdown documents with dynamic links to a Neo4j-based knowledge graph and visually representing the graph.

Clicking on hyperlinked text in the markdown view should bring the relevant node in graph view into focus.

This app aims to make markdown content more navigable and informative, akin to the hyperlinking features seen in Wikipedia, leveraging the depth and connectivity of knowledge graphs.

Acceptance Criteria:

• Accurate identification and linking of keywords in markdown files to corresponding entities in the knowledge graph.
• Embedded links and graph visualization are seamlessly integrated within the markdown documents.
• The application maintains high performance, efficiently processing and enhancing large markdown files.
• Robust error handling for missing links, graph query failures, or visualization issues.

Implementation Details:

• The front-end application should be developed using HTML/CSS and vanilla JS or a framework of preference.
• Integration with Neo4j will employ the official Neo4j JavaScript drivers for querying the knowledge graph.
• Markdown enhancement will utilize libraries designed for parsing and dynamically modifying markdown content.
• Graph visualization will leverage tools listed in the Neo4j graph visualization tools directory.

Mockups/Wireframes:

Product Name:

Bharat Sahaiyak

Organisation Name:

SamagraX

Domain:

Documentation and Knowledge Graph Visualization

Tech Skills Needed:

Markdown, HTML, CSS, Javascript

Mentor(s):

@KDwevedi

Complexity:

High

Category:

Frontend, Visualization, Documentation, Data Integration

Relevant Links:

• Marked: A Markdown to HTML parser
• Marked Demo
• Neo4j Graph Visualisations
• Neo4j Intro
• Cypher Query Language

Additional Notes:

This application seeks to revolutionize how markdown-based documentation is experienced by integrating dynamic linking and visualization of knowledge graphs. It presents a unique opportunity to blend narrative content with data-driven insights, similar to the interactivity found in Wikipedia but with the added dimension of graphical data representation.

[c4gt-community-support[bot]]

Hi! Important Details - These following details are helpful for contributors to effectively identify and contribute to tickets.

• Sub-Category - Please mention the sub-category if any for the ticket

Please update the ticket

[ThisisMS11]

Hi @KDwevedi, Can I work on this project? I have worked with Neo4j before and I find this project idea very interesting, can I contribute?

[KDwevedi]

For sure! I'll assign this ticket to you.

Can you setup an initial repo and raise a PR?

[ThisisMS11]

ok I will do that by today.

[ThisisMS11]

I am not able to make a pull request to your git repo as it is not yet initialized and If I try to add origin as https://github.com/BharatSahAIyak/kg-markdown-enhancer and then try to push to it then permissions issue comes into picture . Is there any other way how it should be done ?

[vsvishalsharma]

Hi @KDwevedi, I would also love to work on this project as i have a hands on experience in working with Neo4j which is one of the best graph database but as i see you have already assigned it to @ThisisMS11 but i suggest we both work in parallel so that it will create a healthy competition among us and produce better results looking forward to contribute

[KDwevedi]

Hey @ThisisMS11 I've initialised the repository. Can you take a clone and try to raise a PR again? LMK if you face more issues.

@vsvishalsharma The point of open source is more collaboration than competition :) How about we let @ThisisMS11 initialise the repo, and see if we can breakdown tasks/milestones between the both of you? I would be more than happy to break down this ticket into more C4GT Tickets, depending on the tasks you want to take up.

[vsvishalsharma]

@KDwevedi sir I am really sorry if in any way I sounded rude I would be happy to contribute to the project ,also sir i am comfortable working in nextjs app router so shall i proceed with that

[ThisisMS11]

I have made a pull request , can you please check and let me know if that is fine or not for a starting point .

[KDwevedi]

Hey @ThisisMS11 I've merged your PR as well

@vsvishalsharma you weren't rude, don't worry

If you're both part of the C4GT discord community, I'm there under KanavDwevedi, can you guys send me a DM? We can establish a quicker rate of comms.

[PRANJALRANA11]

Hy @KDwevedi I wanted to contribute on this issue could you tell me the initial steps to move with

[KDwevedi]

Hey @PRANJALRANA11, Please hit me up on discord, and we can decide where you would best fit to contribute

[KDwevedi]

• [ ] #4
• [ ] #5
• [ ] Integration between Markdown and Neo4J based Knowledge graph in common visualisation.

[Harpreet2004]

Hi @KDwevedi I want to contribute on this issue, need your assistance. Looking forward to contribute

[Singh-Supreet]

@KDwevedi I am very confident that I will be able to do this efficiently. I have a lot of experience in making responsive websites. Looking forward to discussing with you

[prajak002]

Hey @KDwevedi ! i have gone through with the goal and problem statement , and i have created a structured workflow of this. I would like to work on this issue!So, i am writing my approach below .

Objective: The objective of this project is to optimize Bharat Sahaiyak, a front-end application that enriches markdown documents with dynamic links to a Neo4j-based knowledge graph and provides visualization of the graph. The optimization efforts will focus on enhancing performance, usability, and maintainability across various aspects of the application.

Milestones:

1. Project Setup :

   • Set up the project repository and relevant frameworks.
   • Define project structure and development environment.

2. Neo4j Database Interface Setup (Deadline: March 22, 2024):

   • Implement the interface for interacting with the Neo4j database using its JavaScript driver.
   • Design and document Cypher query templates for data retrieval.

3. Markdown Parsing and Formatting (Deadline: March 25, 2024):

   • Develop functionality to parse markdown documents.
   • Identify keywords within markdown for linking with knowledge graph entities.

4. Enriching Markdown with Links (Deadline: March 28, 2024):

   • Implement logic to dynamically insert links to knowledge graph entities matching keywords in markdown.
   • Ensure accurate identification and linking of keywords.

5. Integration of Neo4j Visualization Tools (Deadline: April 5, 2024):

   • Integrate Neo4j graph visualization tools into the application.
   • Customize node coloring based on labels.

6. Interactivity Between Markdown and Graph Visualization (Deadline: April 12, 2024):

   • Implement interactivity where clicking on linked text in markdown updates the graph visualization.
   • Ensure seamless interaction between markdown and graph views.

7. Basic Search Functionality on Markdown Side (Deadline: May 1, 2024):

   • Add basic search feature to quickly find keywords within markdown documents.
   • Enhance user navigability and searchability.

8. Comprehensive Testing and Documentation (Deadline: May 5, 2024):

   • Develop comprehensive unit and integration tests for each component.
   • Document setup, usage, and configurations for the application.

Tech Stack:

• Front-end: HTML/CSS, JavaScript (ES6+), Vanilla JS
• Neo4j Database: Neo4j JavaScript Driver, Cypher Query Language
• Markdown Parsing: Marked.js
• Graph Visualization: Cytoscape.js
• Testing: Jest, Chrome DevTools
• Documentation: Markdown, Docusaurus

Conclusion: By adhering to the outlined milestones and deadlines, this project aims to optimize Bharat Sahaiyak, enhancing its performance, usability, and overall quality. With a well-defined timeline and a carefully selected tech stack, I anticipate delivering a refined and efficient application by the project end date.

Signed: Prajak Sen email:- prajaksen.s@gmail.com Discord:- prajak.2002 [17 march 2024]

[rising0raj]

Hey @KDwevedi I would like to work on this . I have gone through the code base can you assign this to me if it is open??

[vednp]

Hey @KDwevedi I'll like to work on this if it's open

[Yash-Sajwan24]

@KDwevedi I understand the problem, and I believe I have the necessary skills. Could I have the opportunity to work on this?

[aryangandhi05]

Hare Krishna @KDwevedi Sir I want to solve this issue, as i understand it, and i will give my 100% for solving this issue So please, can you assign me this?

[abhinow01]

hi @KDwevedi , is this issue open ? can i work on this

[AKSHATkumar22]

Hello Sir, my name is Akshat Kumar as I have worked and built many projects as a fresher and learner wanted to work on this issue and wanted to resolve the issue. As, I think with my tech skills knowledge I have can add some value to this project and complete the project. So, Sir please let me give a chance to showcase and perform under your guidance.

[Sanika-k-1317]

Hello @KDwevedi I have experience working with HTML, CSS, JS , nodejs. I am really interested in contributing to the project. Could you please let me know how to connect with you for further discussion?

[SAdhikary2]

Hello @KDwevedi , i am a working professional on fullstack web developer domain , i have enough confidence to solve this issue , please allow me to solve this issue and showcase my skills

[heyhimansh]

Hey @KDwevedi , I hope you're doing well! I'm having trouble joining the Discord channel. Could you please send me the new link or an invitation? Thanks a lot.