apache / age

Graph database optimized for fast analysis and real-time data processing. It is provided as an extension to PostgreSQL.
https://age.apache.org
Apache License 2.0
3.03k stars 406 forks source link

Ruby driver #370

Open chughes87 opened 1 year ago

chughes87 commented 1 year ago

Is your feature request related to a problem? Please describe. My company would like to utilize Apache AGE. However, we use Ruby as our primary programming language. Consequently, we will need a driver for Ruby to be implemented.

Describe the solution you'd like A Ruby driver.

Describe alternatives you've considered Without this, we may implement our projects with regular relational database tables.

Additional context None.

andycamp commented 1 year ago

I'm in the same boat. Would be interested in working on the ruby driver.

23tux commented 1 year ago

I just stumbled upon Apache AGE while researching graph databases for Ruby. The first hit was Neo4J, but having some graph functionality on top of an existing Postgres database (and therefore not having to build new infrastructure) sounds very promising to me! So 👍 from my side for the Ruby driver

M4rcxs commented 1 year ago

sounds a good Idea, I started reading more about Ruby PG gem

it works for PostgreSQL 9.3.x or later

Type Casts

Pg can optionally type cast result values and query parameters in Ruby or native C code.

eyab commented 1 year ago

@chughes87 Any updates on this issue?

CoralineAda commented 10 months ago

I'm very interested as well. Years ago I worked with neo4j and it had a Ruby gem, but it looks like all the ORMS and Ruby drivers haven't been updated in years.

btihen commented 8 months ago

noe4j ruby driver is now called activegraph found at: https://github.com/neo4jrb/activegraph

btihen commented 8 months ago

I'm in the same boat. Would be interested in working on the ruby driver.

I'd find it cool to colaborate on this with some one

btihen commented 4 months ago

Greetings - I am exploring ApacheAge Rails integration. So far I am just exploring - no real directions no thoughtful design. I am starting by just trying it out and seeing what's needed. you can see my experiments here https://btihen.dev/posts/ruby/rails_7_1_explore_rails_graphdb_age_integration/

All in the rails console so far - so no rails app yet. Hopefully coming soon. If anyone wants to collaborate I would enjoy that

btihen commented 4 months ago

I am curious - I have started playing with a structure and am using ActiveModel::Base and ActiveModel::Attributes (which generates a Rails Schema) and provides the attributes method but I realized the whole structure then needs to be done with those 2 things in mind. Does that make sense or should it use Plan Old Ruby Objects (POROs)? Less rails like, but more generic. I will also look at what they have done in Neo4j and see what the their core does and how that supports the rails version. In any case, some ideas and feedback from those interested in a gem would be useful to get the API correct.

Until I get feedback, I will probably keep it close to the Rails API - since that is how I would most likely use it and if there is a case for a core (non-rails) code then WE can work on that too. So far:

A simple Edge looks like:

module AgeSchema
  module Edges
    class WorksAt
      include ApacheAge::Edge

      attribute :employee_role, :string
    end
  end
end

A simple Node looks like:

module AgeSchema
  module Nodes
    class Company
      include ApacheAge::Vertex

      attribute :company_name, :string
    end
  end
end

A node with override values:

module AgeSchema
  module Nodes
    class Person
      include ApacheAge::Vertex

      attribute :first_name, :string
      attribute :last_name, :string
      attribute :given_name, :string
      attribute :nick_name, :string
      attribute :gender, :string

      def initialize(**attributes)
        super
        self.nick_name ||= first_name
        self.given_name ||= last_name
      end
    end
  end
end

Usage (so far:

# persisted node
quarry = AgeSchema::Nodes::Company.create(company_name: 'Bedrock Quarry')

# not-persisted node
fred = AgeSchema::Nodes::Person.new(first_name: 'Fred', last_name: 'Flintstone', gender: 'male')

# create edge handles both persisted and not-persisted nodes
works_at = AgeSchema::Edges::WorksAt.create(employee_role: 'Crane Operator', start_node: fred, end_node: quarry)

For all the experimental code see: https://github.com/btihen-dev/rails_graphdb_age_app

btihen commented 4 months ago

Most basic CRUD operations are functional, but I have a problem with the test environment (running tests). I also have documented the issue in the rails repo: https://github.com/rails/rails/issues/51843 - if anyone here knows rails, db config, migrations and the PG driver well - I welcome help debugging / fixing the rails test env.

btihen commented 4 months ago

Solution to tests - rails generates a flawed schema.rb - replace the AGE part with:

# db/schema.rb
ActiveRecord::Schema[7.1].define(version: 2024_05_05_183043) do
  execute('CREATE EXTENSION IF NOT EXISTS age;')
  execute <<-SQL
    DO $$
    BEGIN
      IF NOT EXISTS (
        SELECT 1
        FROM pg_namespace
        WHERE nspname = 'ag_catalog'
      ) THEN
        CREATE SCHEMA ag_catalog;
      END IF;
    END $$;
  SQL

  # These are extensions that must be enabled in order to support this database
  enable_extension 'age'
  enable_extension 'plpgsql'

  # Load the age code
  execute("LOAD 'age';")

  # Load the ag_catalog into the search path
  execute('SET search_path = ag_catalog, "$user", public;')

  execute <<-SQL
    DO $$
    BEGIN
      IF NOT EXISTS (
        SELECT 1
        FROM pg_constraint
        WHERE conname = 'fk_graph_oid'
      ) THEN
        ALTER TABLE ag_label ADD CONSTRAINT fk_graph_oid FOREIGN KEY (graph) REFERENCES ag_graph (graphid);
      END IF;
    END $$;
  SQL

  # create_schema 'age_schema'
  execute <<-SQL
    DO $$
    BEGIN
      IF NOT EXISTS (
        SELECT 1
        FROM ag_catalog.ag_graph
        WHERE name = 'age_schema'
      ) THEN
        PERFORM create_graph('age_schema');
      END IF;
    END $$;
  SQL
end

be sure to commit this and restore it with each subsequent migration (just keeping the new stuff) - as rails (for now) will repeatedly reset the schema to wht it thinks is correct, but is actually incorrect.

btihen commented 4 months ago

An early first version 0.1.0 of a rails plugin / gem can be found at: https://rubygems.org/gems/rails_age

It does not yet support all features, but Edges and Nodes are workable. You can try it out and give feedback.

I'll post a sample app soon:

btihen commented 4 months ago

I now have a sample app at: https://github.com/marpori/rails_age_demo_app - demonstrates nodes and edges in use at all levels of a standard rails app.

The gem (now at 0.3.0) also now has an installer task bin/rails apache_age:install to simplify installation and configuration.

I next plan to create node and edge generators (something like):

feel free to comment on the generator API if desired

PS - Rails still mangles the db/schema.rb file after each migration, but you can use the installer to fix the schema or do a careful git commit to the schema and discard the unwanted changes not directly related to the newest migration. The rails people will not be addressing the interaction with this PG extension any time soon, but the git commit or the installer make it easy to reset the schema file as needed.

btihen commented 3 months ago

v0.4.0 - released breaking change: edges now require custom type :vertex (or a specific node type)

Next

Planned (builds the entity and the associated controller and views (based on the given attributes)

btihen commented 3 months ago

v0.6.0 is released

now you can use generators to create a fully functional rails app based on Apache AGE data nodes and edges (& you can mix in normal rails models and behaviors too).

This includes:

see the README for a simple step-by-step guide. Advanced features such as custom cypher queries, etc will be worked on soon

Aubermean commented 1 month ago

Really nice. Maybe you should get featured in some Ruby/Rails newsletters to help gain interest/traction/support, when you feel the project is 'ready' of course.

Things that might help new interest:

...I think you can get a lot of help/contributors once the 'stars' start rolling in!

btihen commented 1 month ago

Really nice. Maybe you should get featured in some Ruby/Rails newsletters to help gain interest/traction/support, when you feel the project is 'ready' of course.

Things that might help new interest:

  • a 'state of the project' or a checklist of features implemented and features missing...
  • a brief comparison to activegraph (ruby neo4jrb, which for one doesn't work in Rails 7.1)
  • abstracting the 'ruby only' stuff to another gem, for those that don't use rails
  • getting featured in the apache/age README as a community driver!

...I think you can get a lot of help/contributors once the 'stars' start rolling in!

Thanks for the feedback and encouragement. I had a family emergency that took up all my time, so I put this aside for a bit.

I haven't really this to activegraph - I know it has a ruby gem and a rails engine, but the code looked a bit hard to get into and I didn't see a little tutorial explaining how to create a rails app. So I decided this would start with a focus on rails and making it easy to generate and use within the rails ecosystem. Along with what I hope is an easy to use getting started and creating a nice quick rails app.

So far I rely heavily on ActiveModel - so I if I do a ruby version at some point it might be done in 2 stages first with activemodel included and then maybe without.

At the moment if you are doing standard models and simple relations this is complete. I figure to be useable for complex - real-world apps I need to allow cypher queries to be built similar to active record queries. I think I have a nice approach. Just testing the query aspects.

I also want to find a way to build simple path queries a lot like a node or an edge query and return a path object. Just playing with that as mind experiments.

I figure when those are done (and hopefully tested by others) that can be a 1.0 release.

Any help would be appreciated, however, I am not sure how many people are interested.

Maybe it is just visibility, but I am not sure how many are interested in graph databases.

I was planning to introduce this to my local ruby community this fall and see what the response is like.

btihen commented 1 day ago

rails_age - v0.6.1 is released

This release allows generic queries on nodes and edges, ie:

flintstone_family =
  Person
    .where(last_name: 'Flintstone')
    .order(:first_name)
    .limit(4).all
    .map(&:to_h)

# generates the query
SELECT *
FROM cypher('age_schema', $$
    MATCH (find:Person)
    WHERE find.last_name = 'Flintstone'
    RETURN find
    ORDER BY find.first_name
    LIMIT 4
$$) as (Person agtype);

# and returns:
[{:id=>844424930131974, :last_name=>"Flintstone", :first_name=>"Ed", :gender=>"male"},
 {:id=>844424930131976, :last_name=>"Flintstone", :first_name=>"Edna", :gender=>"female"},
 {:id=>844424930131986, :last_name=>"Flintstone", :first_name=>"Fred", :gender=>"male"},
 {:id=>844424930131975, :last_name=>"Flintstone", :first_name=>"Giggles", :gender=>"male"}]

and
```ruby

fred =
  Person
    .where(last_name: 'Flintstone')
    .where(first_name: 'Fred')
    .order(:first_name)
    .limit(4).all
    .map(&:to_h)

# generates the query
SELECT *
FROM cypher('age_schema', $$
    MATCH (find:Person)
    WHERE find.last_name = 'Flintstone' AND find.first_name = 'Fred'
    RETURN find
    ORDER BY find.first_name
    LIMIT 4
$$) as (Person agtype);

# and returns:
[
 {:id=>844424930131986, :last_name=>"Flintstone", :first_name=>"Fred", :gender=>"male"}
]

NOTE: Input variables are not yet sanitzed -- so do not yet use any untrusted inputs (especially user input params in generic queries).

Sanitizing Query inputs is the goal of v0.6.2 (using the rails sanitizer).

btihen commented 25 minutes ago

rails_age - v0.6.2 is released

Now Queries using hash params are sanitized, string inputs are not yet.

This query is sanitized:

Person
  .where(last_name: 'Flintstone')
  .order(:first_name)
  .limit(4).all

NOTE: attributes are not yet checked (a goal for v0.6.3)

This is not (and probably can't easily be done:

Person
  .where("last_name = 'Flintstone'")
  .order('first_name')
  .limit(4).all

The goal for v0.6.3 is to allow (& sanitize):

Person
  .where("last_name = ?", 'Flintstone')
  .order('first_name desc')
  .limit(4).all

PS - an independent security review of this work would be appreciated if someone has the time and inclination.