search

DotNet4Neo4j / Neo4jClient

.NET client binding for Neo4j
https://www.nuget.org/packages/Neo4jClient
Microsoft Public License
429 stars 146 forks source link

Status of this project #90

Closed cheerfulstoic closed 9 years ago

cheerfulstoic commented 9 years ago

Hello! I'm a developer advocate for Neo4j and I wanted to check on the status of this driver. Looking around I haven't seen much activity. Feel free to let me know if you need anything from Neo to help support the .NET community!

RaphHaddad commented 9 years ago

Hi @cheerfulstoic. This is being used.

http://stackoverflow.com/questions/tagged/neo4jclient

Thank you :)

ekkis commented 9 years ago

@cheerfulstoic, one thing it's missing is Powershell support. I've done the preliminary work to figure out how to use it, as documented here: http://stackoverflow.com/questions/30314696/return-overload-fails and mean to document it into the Neo wiki. in fact, what would make the whole thing considerably easier is if Neo4jClient would offer a non-generic version of the .Return method.

anyhow, if you're looking for something to do, perhaps you could document my efforts into the wiki. it is possible to use it from Powershell but it's tricky and has taken me some effort to figure out

cheerfulstoic commented 9 years ago

Sorry, I wasn't clear ;) I'm not a maintainer of the library, I was asking the maintainer(s) (the Readify team, I think) if this project is being maintained (responding to issues/fixing code/adding code/releasing new versions). I've not used .NET yet, but I wanted to check in on it.

neutmute commented 9 years ago

+1. A few outstanding pull requests and not much activity on them. Where is the love Readify? :)

tathamoddie commented 9 years ago

Hi. I'm the current maintainer.

@jexp has been asking me the same question via email, so time to write it all out…

Looking back

This library started back in early 2011. We (@Readify) were kicking off a project that we decided to use Neo4j for. That project is still running to this day: it’s the one @RaphHaddad is on. The majority of the team are client-devs, with some Readify leadership and augmentation. At the time, there were no compelling .NET drivers, and Cypher didn’t really exist yet (it was just an early proposal).

Rather than building our own library in isolation, with the risk of it bleeding into our app, I decided to put it in a separate repository, open source it, and pull it into the project via NuGet. To support a fairly rapid development loop, I bolted the CI build straight on to NuGet upload. We’ve seen 662 builds released since.

Within the project context, I remained the primary driver of the library, with some support from @mavadat (now Readify alumni), @rajeev-n (project team), and @moh-abed (now Readify alumni).

As the library gained popularity, we received contributions from a number of people outside the original project: @aranm, @cskardon (who also answered a lot of StackOverflow questions), @hoopoos, @arturosevilla, @gpierrick (who uses it behind his startup’s Neo4j-based web app), @scoarescoare, @danielcor, @sbarski, @wis3guy, @jpotts2, @bholdt (Readify), @pjvds, @brentonmcs, @simonpinn, @mcliment, @sprucely, @tendersoft-mjb, @b2berry, @juwujaren, @egarrington and @michaelbarnes. I am immensely grateful for all of these contributions.

The Neo Technology team supported me personally a lot, including inviting me over to San Francisco for a week to hang out with other driver authors from around the world.

In late 2012, I personally moved into a different role at Readify which saw me focussed more on software process, and not touching Neo4j at all. At this point Neo4jClient moved into more of a personal side-project for me.

In mid-2014, I moved into a different role again, in Readify’s national management team. This has two direct impacts: a) I’m a long way from dogfooding the library myself, and b) my personal priorities are quite different.

Current state

The functionality that is in place is well tested and robust.

There is both Gremlin and Cypher support.

The Cypher support is slowly ageing out, as new keywords are added to the language, but not supported in the library.

The Cypher response parser is a complete mess of complexity, that needs to be revisited entirely. There are a good number of tests around it, so it’s just a matter of re-writing the implementation. The main goal here is to support streaming, whereas the current parser implementation doesn’t at all.

There are number of prototypes and pull requests for transactions support, but none of these have landed yet.

The Gremlin support is creating a tax on the entire library, slowing down the ability to make other changes. I think it needs to go.

Project management, and my capacity

There’s a good release structure in place: pull requests get automatically validated; once they’re merged, they are then automatically released within a few minutes. If it’s a small and targeted PR, with a clear diff, and includes appropriate tests, then I click a single button on github.com and it’s all done.

The challenge comes with pull requests which require work. The second that the diff gets bit and ugly, the tests are incomplete, or there’s something up for architectural debate, it effectively gets parked. As demonstrated over the last 12+ months, I just won’t get to it.

Going forward

I’ll continue to try and accept ‘good’ pull requests: short, targeted, clean diff, tested.

I cannot even pretend to commit to personally taking on anything more complex. Even if I had the time, I don’t think I’m in a good position to drive the library at the moment, considering I’m not actively working with Neo4j at all.

Readify as a company only has minimal engagement in one project that uses Neo4j. While we put a lot of corporate resources towards open source, and have fairly liberal IP policies to support this, we’d need more people actively using Neo4j to see any further efforts here.

I’d fully support somebody wanting to take over the management aspects. As a first step, they could take over some of the core issues mentioned above, and pending PRs. I’d still be around as a final review on the ‘Merge’ button for the first few, but with an intention to hand that permission over pretty quick. In this scenario, we can work out what to do about the branding of it being under the @Readify org here on GitHub, versus helping somebody else build their own brand from their efforts.

If you’re a passionate .NET developer, doing some work with Neo4j, and would like to be able to use company-sponsored time towards the cause, take a look at https://github.com/Readify/madskillz/blob/master/Consulting.md then https://join.readify.net :)

How does that sit with everyone? What other ideas are out there?

tathamoddie commented 9 years ago

Ooops ... one more option I forgot to mention: does this library have enough baggage that it's time to cut our losses and start again, with something that is more modern and Cypher-focused? Transactions, new remoting protocol, and pure-Cypher support from day 0?

cskardon commented 9 years ago

Hi @tathamoddie

First and foremost - I think a big thanks is in order, Neo4jClient has been the defacto standard for .NET development for a while now and without it, rolling our own would have taken up a lot more time, and there would be even less .NET usage (yes - even less)

In terms of options - I think there are a few things to get into Neo4jClient (transaction support, and to handle all keywords) up to Neo4j 2.x.

Cutting and running to a new library entirely has it's plus points (who doesn't like starting from scratch), but there will be people who (for whatever reason) will need to stay on 2.x and we can't just say 'sorry, no dice' to them. No reason not to go for a new driver for 3.x+ though :)

I think we need a list of developers still using Neo4jClient and willing to put some time in, as for similar reasons to yourself, people will have moved on from using it, or are on different projects, or "reason".

So - I'm willing and able to put time in, and I'll put my hat into the ring to help with the management.

simonpinn commented 9 years ago

Hi @tathamoddie ,

Also a huge thank you for providing a library to allow us to unlock Neo4j in the .Net space without going through the pain of managing our own libraries, I certainly wouldn't have picked up Neo4j as a replacement datastore so quickly had it not been available. And, I for one still regularly use the libraries having also produced the entity extension library to augment Neo4jClient (https://github.com/simonpinn/Neo4jClient.Extension) which is usually how I use Neo4j and definitely the quickest way to do a demo for new graph developers and make their eyes light up!

I'd very much like to see the project continue, adding full support for 2.x and transaction support would make it all much better, agree with the sentiments around the response parser since having gotten involved with supporting camel casing.

I would also like to do what I can to help it continue. Probably not enough time to manage it, but definitely contribute to moving parts forward.

ekkis commented 9 years ago

I wish to second the sentiment in that we owe you a debt of gratitude. a .Net client opens the world of Neo for a good many of us. I’ve only recently discovered Neo and see tremendous potential within the corporation. without a .Net client writing apps with it would be nearly out of reach for me.

to that I would like to add that I wish I had the time as I would love to contribute, but frankly I’m always just buried with work. I have too much on my hands already but I do hope this project finds a new sponson and doesn’t get abandoned. it would be a real pity.

thanks,

On Jul 8, 2015, at 00:18, Chris Skardon notifications@github.com wrote:

Hi @tathamoddie https://github.com/tathamoddie First and foremost - I think a big thanks is in order, Neo4jClient has been the defacto standard for .NET development for a while now and without it, rolling our own would have taken up a lot more time, and there would be even less .NET usage (yes - even less)

In terms of options - I think there are a few things to get into Neo4jClient (transaction support, and to handle all keywords) up to Neo4j 2.x.

Cutting and running to a new library entirely has it's plus points (who doesn't like starting from scratch), but there will be people who (for whatever reason) will need to stay on 2.x and we can't just say 'sorry, no dice' to them. No reason not to go for a new driver for 3.x+ though :)

I think we need a list of developers still using Neo4jClient and willing to put some time in, as for similar reasons to yourself, people will have moved on from using it, or are on different projects, or "reason".

So - I'm willing and able to put time in, and I'll put my hat into the ring to help with the management.

— Reply to this email directly or view it on GitHub https://github.com/Readify/Neo4jClient/issues/90#issuecomment-119466217.

arturosevilla commented 9 years ago

Hi,

Where I work we keep using Neo4jClient for a big product that we have, but we use it using my transactions implementation as we do require it.

Thanks for the great work.

RaphHaddad commented 9 years ago

I would like to see a new client with a 'Cypher-first' architecture.

I would be more than happy to contribute to this as well. However we shouldn't just drop support for the current client.

ekkis commented 9 years ago

something which just takes a cypher string. that would be my preference too. leave the client agnostic of the language the back-end talks

On Jul 8, 2015, at 16:15, Raphael Haddad notifications@github.com wrote:

I would like to see a new client with a 'Cypher-first' architecture.

I would be more than happy to contribute to this as well. However we shouldn't just drop support for the current client.

— Reply to this email directly or view it on GitHub https://github.com/Readify/Neo4jClient/issues/90#issuecomment-119756482.

neutmute commented 9 years ago

@arturosevilla which branch of your fork do you use for your production transactions implementation? We are starting a new project with neo4j and since the master of this repo does not support transactions, we will start with yours. Is your version published as a nuget package?

btw @tathamoddie thank you for the frank response on your status. With Readify only having one neo4j customer, does this hint at your migration to an alternate graph database? We would consider Orient but for the client docs saying it is not production ready.

RaphHaddad commented 9 years ago

@neutmute I am at this customer at the moment.

There is no migration to an 'alternate graph database' in our minds nor in the customer's mind at the moment (for this customer). Readify has been engaged (and has an excellent relationship) with this customer for several years and we've seen the project grow, scale and adapt. Neo4j has also been part of this growing and adapting. I don't think we will be dropping Neo4j anytime soon.

It seems from this thread that there are several people who are interested in contributing. With @cskardon putting his hand up for managing. I too would be happy to help in the management of this project.

The question still remains though on the way forward.

I would suggest:

  1. To keep and maintain this client as @tathamoddie has been for the last few months. That is, no pull requests with major architecture fixes and no major feature additions.
  2. To create a new client with a 'cypher-first' architecture in mind.
arturosevilla commented 9 years ago

@neutmute inside of my repository you can find the branch aggregatewithoutgenericnodewithnullinmemberexpressionfix which I recommend because is more up to date with the changes that also have been applied as well as other changes that I made. If you see a problem you can also find the transactions branch. But no, I don't have them published as a nuget package.

I do agree with @tathamoddie as I had to basically rewrite the request and parser part of the library to implement transactions.

cskardon commented 9 years ago

Personally, I think a more 'overhaul' approach might be better, yes - it's work as @tathamoddie and @arturosevilla have mentioned a rewrite of the request / parser area is significant and time consuming. But Neo4jClient has been downloaded 65K+ - now I know I'm responsible for downloading it a lot more than once :), but regardless it's a significant number of people, and it's a large number of people who would benefit from Transactional support (for example).

First port of call in my view is to visit the existing pull requests and yay/nay them. Then triage the issues here. Get a proper view of where we actually are before we move away from the past few years of tests and general stability.

technige commented 9 years ago

You ran an excellent project, @tathamoddie. You'll be sorely missed.

neutmute commented 9 years ago

@cskardon agree with your overhaul approach. Given @arturosevilla's transaction support has been in PR status for 18 months, there is an immediate risk that this project fragments between the dormant readify:master branch and @arturosevilla's more active fork. Starting a rewrite now would only dilute potential contributor resources even further IMHO.

Perhaps something like this?

  1. First pass on simple PR's. Accept/Reject/Amend with the submitters
  2. Review and work with @arturosevilla on merging his transaction supported branch(s). Noting that his request / parser code has already been massively refactored in order to support transactions. I note he has extensive 61 unit tests dedicated to transactions that are passing.
  3. Consider bumping the version to 2.0 and drop all Gremlin support. I'm a newcomer to Neo4j and was only running through the movie tutorial last night. Everything is Cipher and I haven't seen a mention of Gremlin anywhere. Existing Gremlin consumers can remain on the 1.* nuget package line and it has the advantage of simplifying further refactoring (I'm hoping).

Looks like we are two weeks too late. This discussion would have made a good use case for the last alt.net topic :)

jexp commented 9 years ago

Thanks a ton @tathamoddie for your amazing work, and I'm really happy that so many of you chime in with their suggestions and offers to help. We should probably have done this earlier :)

I totally agree with @neutmute's points.

I'd love if the work from @arturosevilla was merged in and we started to address a number of topics (which we should create issues for).

I suggest that @cskardon and someone else could step up for the management aspects and everyone who's able to would start contributing with docs, code, tests, example projects, articles, talks :)

I'm very excited about this and willing to support you all with anything that I can do.

Things I'd love to see in Neo4jClient:

(to be converted into gh-issues if not already fixed)

Feedback from customers:

Ping @peter-wiles @IngvarKofoed @GlennSarti

arturosevilla commented 9 years ago

@neutmute @jexp I'm open to any changes or branches you'd want to join into my transactions implementation. I can tell you that where I work we are serving critical infrastructure using such build, but I do think there are some improvements that could be done regarding the ITransaction interface that I did with the System.Transactions in .NET. If you want to with my fork, please review the Neo4jClient.Execution namespace as well as that's where I basically rewrote the way Neo4jClient did its requests. There are some small changes in Serialization as well, and I'm working on actually implementing the new auth mechanism on 2.2.x.

However, right now I'm trying to implement the auth mechanism and backport the whole new "Execution"/Request namespace into a separate package so I can write the auth on top of it. However if you decide to join it with my transactions fork, I think it's going to be easier for me to implement such new feature.

cskardon commented 9 years ago

So we have 2 core branches here, the master on Neo4jClient that contains pulls and fixes including F# support etc and @arturosevilla 's aggregate.... branch with transaction support.

Ideally, we want to pull @arturosevilla's into the master and get the benefits of

a) the current tested master codebase of @tathamoddie b) the current tested aggregate... codebase of @arturosevilla

Pulling into master means we get the nuget updated, and everyone can get the update, without having to switch to a new package. Also - other projects (@simonpinn's, the AspNet.Neo4j.Identity project for example) aren't rendered into needing big overhauls.

Soo.... In my view the easiest route would seem to be to merge the current master into aggregate... then once that's done, pull request the new aggregate+master branch back into the master branch. The two branches have gone their separate ways quite significantly, but I think the merge from master to agg will be simpler to manage (I could be wrong).

What are people's thoughts on this?

Moving on, I think this plus the existing valid pull requests and fixes would be the last v.1 release, aside from fixes etc, and a v.2 where we can pull out things like Gremlin, 'RootNode' bits etc

CharlieDigital commented 9 years ago

@cskardon I think it's about time :)

I asked @arturosevilla to spend a month to write and unit test the transactional support at the very beginning of our project as we wanted to have unit tests for our core queries.

For us, it has been working flawlessly over thousands of hours of use and the transactions and are an absolute must if you want to unit test your core queries or multi-statement execution blocks.

We are already planning on having @arturosevilla add the authentication support as well due to our customer demand for it so it is great news to see that his work will be merged in to master.

cskardon commented 9 years ago

@CharlieDigital I think so too :)

My current branch (cskardon/neo4jclient) currently contains the merge of @arturosevilla's and this repo's master. Obviously this needs some testing, but as it stands all tests are passing (though one does fail in NCrunch for some reason - which I need to investigate - I think it's a quick one).

So this includes the CamelCase fixes, F#, Transactions etc, and should be an easy merge back into this repo when we're ready. By all means, test it out and lets look at getting it all out there.

neutmute commented 9 years ago

@cskardon if you can get your branch published as a nuget prerelease package I'm happy to dogfood it with the caveat that our project is just kicking off so it won't exactly be a huge amount of coverage. I'll be using it in combination with fluent config from Neo4jClient.Extension

jexp commented 9 years ago

Hello Everyone, just a quick question, my colleagues are looking for example use-cases how people use Neo4j from .Net, so if you want to share what you're doing, please drop me an email to michael at neo4j.org

Thanks a ton!

cskardon commented 9 years ago

@neutmute I can publish one to a myget feed for now? Obviously it'll take a bit of time before getting to the proper Nuget feed - but it would work in the same way....

neutmute commented 9 years ago

@cskardon a myget feed would do fine!

cskardon commented 9 years ago

@neutmute - okey dokey - give https://www.myget.org/F/neo4jclient-tx/api/v2 a whirl - it's in there as pre-release. There is a breaking change, and that's the return of the 'collectas' methods - no longer Node but just T

arturosevilla commented 9 years ago

@cskardon That's awesome, I'll be happy to provide input on implementation if needed.

cskardon commented 9 years ago

@arturosevilla That'd be great, obviously your unit tests provide some documentation, but more is always good :)

cskardon commented 9 years ago

Guys (n' maybe gals?) - there is a pre-release version containing @arturosevilla's changes in the official nuget (https://www.nuget.org/packages/Neo4jClient/) location now.

Depending on feedback, we'll look at pushing to the proper version asap, please note there is a breaking change if you use CollectAs in your code, in 1.0.0.662 this returned IEnumerable<Node<T>>, in 1.1.0-Tx# this returns just IEnumerable<T>.

Once we've ascertained that the Tx stuff is all good - my plan is to go through the other pull requests and merge them in (Oh hey! there's 2 by @me, they'll be good :) ).

So erm, let the awesomeness continue!

gpierrick commented 9 years ago

Bit late to the discussion but willing to help as well!

Although these days I spend more time in node.js with Neo4j and .net and SQL but I still maintain my first startup timeblend.com which uses Neo4jClient. (Neo4j 1.9)

Thanks

cskardon commented 9 years ago

Ace @gpierrick, there's still your Pull to get in :)

neutmute commented 9 years ago

Great work @cskardon! Just last night I branched @simonpinn's extensions and referenced your myget package. Will update to the official prerelease and pull that into my solution over the coming days. Been busy shaving vNext yaks.

tathamoddie commented 9 years ago

To keep everyone in the loop, ~18 hours ago @cskardon and I finally caught up for a long overdue Skype chat.

It's evident from Chris' long term StackOverflow efforts, prior PRs, and engagement in this thread, that he's a key contributor to the project, and a safe set of hands.

I've gone ahead and given him write access to this repo, and owner rights to https://nuget.org/packages/Neo4jClient. This means he now has all the permissions required to publish official releases, as evidenced by https://www.nuget.org/packages/Neo4jClient/1.1.0-Tx00008 coming out.

The build infrastructure to date has been on my own TeamCity instance, which is reasonably old now. I've paused all these builds, and he already has new ones up and running over at MyGet.

We might need to shift the repo out from under github.org/Readify to a different org at some point, to make user management easier, and reflect the project becoming more of its own self-standing thing. This is probably a good opportunity to come up with a better name than "Neo4jClient". First reaction was Neo4net, but that's already been used of course, and it misses the part about being a client driver. Ideas welcome.

With all those points out of the road, I think we've answered the original question of "what's the status of this project?", so I'm going to close this issue, and we can use new issues for further, more-specific threads.

mavadat commented 9 years ago

Great stuff @tathamoddie :+1:

jexp commented 9 years ago

Thanks so much Tatham, Chris and everyone who chimed in, I'm soo happy and grateful that Neo4jClient will be woken from its beauty sleep.

As mentioned before, I'm happy to give the project a new home at github.com/neo4j-contrib if that would be interesting.

I'm not sure we need a name, change, it is very much established, has high google results and stackoverflow tags.

I think what makes more sense is to break out smaller modules for certain aspects, like a fast, thin plain cypher transport or the binary transport. And perhaps some higher level ones like EntityFramework integration or a LinQ module. Which then can either get very descriptive names or just a suffix. (but that's just my 2 cents).

Otherwise a new name might make sense for a rewrite or a sufficient revamp for a Neo4j 3.x compatible version? But I wouldn't focus on that right now.

Am 16.07.2015 um 04:22 schrieb Tatham Oddie notifications@github.com:

To keep everyone in the loop, ~18 hours ago @cskardon https://github.com/cskardon and I finally caught up for a long overdue Skype chat.

It's evident from Chris' long term StackOverflow efforts http://stackoverflow.com/search?q=user:2266+%5Bneo4jclient%5D, prior PRs, and engagement in this thread, that he's a key contributor to the project, and a safe set of hands.

I've gone ahead and given him write access to this repo, and owner rights to https://nuget.org/packages/Neo4jClient https://nuget.org/packages/Neo4jClient. This means he now has all the permissions required to publish official releases, as evidenced by https://www.nuget.org/packages/Neo4jClient/1.1.0-Tx00008 https://www.nuget.org/packages/Neo4jClient/1.1.0-Tx00008 coming out.

The build infrastructure to date has been on my own TeamCity instance, which is reasonably old now. I've paused all these builds, and he already has new ones up and running over at MyGet.

We might need to shift the repo out from under github.org/Readify to a different org at some point, to make user management easier, and reflect the project becoming more of its own self-standing thing. This is probably a good opportunity to come up with a better name than "Neo4jClient". First reaction was Neo4net, but that's already been used http://neo4net.codeplex.com/ of course, and it misses the part about being a client driver. Ideas welcome.

With all those points out of the road, I think we've answered the original question of "what's the status of this project?", so I'm going to close this issue, and we can use new issues for further, more-specific threads.

— Reply to this email directly or view it on GitHub https://github.com/Readify/Neo4jClient/issues/90#issuecomment-121804511.

neutmute commented 9 years ago

+1 name shouldn't change due to SEO