If this project is dead, what's the alternative?

[fkleedorfer]

I suppose other rdfstore-js users are wondering about this as I am.

I'm working on a project that requires an RDF store in the browser, I chose rdfstore-js as it seemed superior to the other options I checked (and there weren't many). So, above all: Thanks @antoniogarrote for building this! It's great, and, of course, it has its problems and bugs, but the most important problem is: it's abandoned.

• @antoniogarrote - you are working on the 1.1 branch, but I don't see any documentation to help me figure out if I can use that code. Where is it going? Will it eventually replace the abandoned version in the master branch?
• To those currently working on forks and waiting on pull requests (@trueg, @juniperchicago, @bensinober, @RubenVerborgh, @jandrieu... ) - Is any of you planning to continue working on rdfstore-js in the near future? Could we find some way of collaborating to get this code to work the way we need it? Maybe you want to share what your modifications were about? Maybe we could find a way to combine the forks into one we continue to maintain? It seems to me that the potential for continuing is there - or am I missing something?
• If we have to give up on rdfstore-js: which RDF store would you recommend for use in the browser?

[RubenVerborgh]

Since rdfstore-js uses my N3.js parser, I created pull requests to incorporate the latest version of this library. Maintaining a library of my own, I have however no intention of working on rdf-store.js. My personal plans are as follows:

• Maintain the N3.js library for parsing, writing, and storing Turtle, TriG, N-Triples, and N-Quads in a streaming way.
• Maintain the SPARQL.js SPARQL parser.
• Maintain the Linked Data Fragments client, which executes SPARQL queries against a Triple Pattern Fragments server (using the above two libraries). Eventually, this project might or might not contain a SPARQL query executor for other back-ends.

In general, while I applaud @antoniogarrote's efforts to make this feature-complete rdfstore-js library, the source code is not in optimal shape. It is my personal opinion that it's not the best way to continue; but no other library is as feature-complete as this one (but recent efforts, such as TriG, are missing).

Depending on what features you need, I highly recommend N3.js for the browser. We build cool applications with it, including this one.

[kidehen]

On 2/26/15 4:48 PM, Ruben Verborgh wrote:

Since rdfstore-js uses my N3.js https://github.com/RubenVerborgh/N3.js parser, I created pull requests so incorporate the last version of this library. Maintaining a library of my own, I have however no intention of working on rdf-store.js. My personal plans are as follows:

• Maintain the N3.js https://github.com/RubenVerborgh/N3.js library for parsing, writing, and storing Turtle, TriG, N-Triples and N-Quads in a streaming way.
• Maintain the SPARQL.js https://github.com/RubenVerborgh/SPARQL.js SPARQL parser.
• Maintain the Linked Data Fragments client https://github.com/LinkedDataFragments/Client.js, which executes SPARQL queries against a Triple Pattern Fragments server (and use the above two libraries). Eventually, this project might or might not contain a SPARQL query executor for other back-ends.

In general, while I applaud @antoniogarrote https://github.com/antoniogarrote's efforts to make this feature-complete rdfstore-js library, the source code is not in optimal shape. It is my personal opinion that it's not the best way to continue; but no other library is as complete as this one.

Depending on what features you need, I highly recommend N3.js https://github.com/RubenVerborgh/N3.js for the browser. We build cool applications with it, including this one http://client.linkeddatafragments.org/.

— Reply to this email directly or view it on GitHub https://github.com/antoniogarrote/rdfstore-js/issues/86#issuecomment-76281379.

We have worked with rdf-store.js and to date, there's no other alternative that we are aware of.

We will continue to use this effort. Naturally, any bug fixes or new features we introduce will certainly be contributed back to this project.

Our soon too be released RDF Editor does make use of rdf-store.js, for instance.

Regards,

Kingsley Idehen Founder & CEO OpenLink Software Company Web: http://www.openlinksw.com Personal Weblog 1: http://kidehen.blogspot.com Personal Weblog 2: http://www.openlinksw.com/blog/~kidehen Twitter Profile: https://twitter.com/kidehen Google+ Profile: https://plus.google.com/+KingsleyIdehen/about LinkedIn Profile: http://www.linkedin.com/in/kidehen Personal WebID: http://kingsley.idehen.net/dataspace/person/kidehen#this

[melvincarvalho]

Might not be everyone's cup of tea, but I am currently using :

https://github.com/linkeddata/rdflib.js

It's an active project with 20 commits in the last month from 5 different authors (judge that as you will!)

I would say it requires quite a bit of web literacy to dive into, as the documentation relatively light. But I've been very happy with the results so far.

[fkleedorfer]

@melvincarvalho Thanks for the pointer - as I extract from open issues, there is no support for JSON-LD, and I don't see any named graph handling (I might have overlooked something), which I need, too. So there seems to be an arms race in terms of features, which rdflib.js will be winning in the long run, given developer activity, but if you need these (and maybe other) features right now, you're stuck with rdfstore-js. Having said that - does anyone think there is potential to join efforts with the people working on/with rdflib.js?

@kidehen It's great to hear that such a strong player in the community as OpenLink is backing this project. Would you be willing to maintain a fork for as long as nobody merges pull requests into the main project?

Having said that, @antoniogarrote as people seem to be actively working on the code, might you be interested in creating an organization for rdfstore-js and adding those actively developing so we get unstuck?

@RubenVerborgh Thanks for that pointer, too - I'll look into it. However, as my server doesn't support RDF Fragments, I suppose it is probably not what I need (reading linked data, preferably in quads - TriG, n-quads or JSON-LD) and querying them in SPARQL - however, that last part may be achieved differently)

[trueg]

@antoniogarrote first of all: thank you for rdfstore-js.

@RubenVerborgh and thanks to you for N3.js. We actually use it in addition to rdfstore.js since our fork does not contain your patches.

@fkleedorfer I certainly do not have the time to maintain a fork of rdfstore.js all by myself but I am very open to the idea of doing it collaboratively. A first step would be to create a merged branch from all our patches and test the result in our individual projects.

[bblfish]

@antoniogarrote has helped integrate rdfstore.js into banana-rdf but I think he is very busy, and so things have not moved very far. I have tried fixing a few bug, but without the lack of progress on this branch has slowed my enthusiasm. We may have to remove it from banana-rdf now. But if a project presents itself that continues where rdfstore.js left off, then this will give us a good reason to continue maintaining that code.

[kidehen]

On 2/27/15 3:44 AM, Sebastian Trueg wrote:

@antoniogarrote https://github.com/antoniogarrote first of all: thank you for rdfstore-js.

Yes, thanks for this effort.

@RubenVerborgh https://github.com/RubenVerborgh and thanks to you for N3.js. We actually use it in addition to rdfstore.js since our fork does not contain your patches.

Yes, thank you too!

@fkleedorfer https://github.com/fkleedorfer I certainly do not have the time to maintain a fork of rdfstore.js all by myself but I am very open to the idea of doing it collaboratively. A first step would be to create a merged branch from all our patches and test the result in our individual projects.

Sebastian:

We have to ensure all our patches are available to both of these projects.

All:

I think we can collectively bring more attention to these important projects, thanks to this thread.

Regards,

Kingsley Idehen Founder & CEO OpenLink Software Company Web: http://www.openlinksw.com Personal Weblog 1: http://kidehen.blogspot.com Personal Weblog 2: http://www.openlinksw.com/blog/~kidehen Twitter Profile: https://twitter.com/kidehen Google+ Profile: https://plus.google.com/+KingsleyIdehen/about LinkedIn Profile: http://www.linkedin.com/in/kidehen Personal WebID: http://kingsley.idehen.net/dataspace/person/kidehen#this

[kidehen]

On 2/27/15 2:58 AM, fkleedorfer wrote:

@melvincarvalho https://github.com/melvincarvalho Thanks for the pointer - as I extract from open issues, there is no support for JSON-LD, and I don't see any named graph handling (I might have overlooked something), which I need, too. So there seems to be an arms race in terms of features, which rdflib.js will be winning in the long run, given developer activity, but if you need these (and maybe other) features right now, you're stuck with rdfstore-js. Having said that - does anyone think there is potential to join efforts with the people working on/with rdflib.js?

We should always collaborate, at every turn. We need to build an ecosystem of tools that build bridges into other specialty realms e.g., UI and generic controllers etc..

@kidehen https://github.com/kidehen It's great to hear that such a strong player in the community as OpenLink is backing this project. Would you be willing to maintain a fork for as long as nobody merges pull requests into the main project?

We are happy to work with @antoniogarrote https://github.com/antoniogarrote in regards to this effort. Basically, we can assist where needed in regards to ensuring there's a Javascript based SPARQL compliant RDF DBMS.

Regards,

Kingsley Idehen Founder & CEO OpenLink Software Company Web: http://www.openlinksw.com Personal Weblog 1: http://kidehen.blogspot.com Personal Weblog 2: http://www.openlinksw.com/blog/~kidehen Twitter Profile: https://twitter.com/kidehen Google+ Profile: https://plus.google.com/+KingsleyIdehen/about LinkedIn Profile: http://www.linkedin.com/in/kidehen Personal WebID: http://kingsley.idehen.net/dataspace/person/kidehen#this

[fkleedorfer]

Given that rdflib.js might be the better alternative in the long run, and not knowing how many of the feature requests that we care about we can actually implement, I'd say the safest way to proceed is to follow @trueg's suggestion and merge our fixes/extensions, and we'll see where we go from there.

So if you agree we:

• create an organization for the project, add all intersted parties as committers
• fork @trueg's version
• start a branch for merging the forks
• each fork owner merges their changes with that branch
• Finally, we see if we want to collaborate on some of the open issues.

That should at least leave us with a more stable version of what there is already and maybe somehing like a community around it, just in case.

I have already created an organization to save time, so if you agree, please post here if you'd like to be added to the organization. Also, if someone else wants to be the owner of the organization, please step up :)

Who's in?

[kidehen]

On 2/27/15 12:34 PM, fkleedorfer wrote:

Given that rdflib.js https://github.com/linkeddata/rdflib.js might be the better alternative in the long run, and not knowing how many of the feature requests that we care about we can actually implement, I'd say the safest way to proceed is to follow @trueg https://github.com/trueg's suggestion and merge our fixes/extensions, and we'll see where we go from there.

So if you agree we:

• create an organization https://github.com/rdfstore-js-community for the project, add all intersted parties as committers
• fork @trueg https://github.com/trueg's version
• start a branch for merging the forks
• each fork owner merges their changes with that branch
• Finally, we see if we want to collaborate on some of the open issues.

That should at least leave us with a more stable version of what there is already and maybe somehing like a community around it, just in case.

I have already created an organization to save time, so if you agree, please post here if you'd like to be added to the organization. Also, if someone else wants to be the owner of the organization, please step up :)

Who's in?

Okay.

I'll leave Sebastian to deal with the rest of the matter, from our side.

Regards,

Kingsley Idehen Founder & CEO OpenLink Software Company Web: http://www.openlinksw.com Personal Weblog 1: http://kidehen.blogspot.com Personal Weblog 2: http://www.openlinksw.com/blog/~kidehen Twitter Profile: https://twitter.com/kidehen Google+ Profile: https://plus.google.com/+KingsleyIdehen/about LinkedIn Profile: http://www.linkedin.com/in/kidehen Personal WebID: http://kingsley.idehen.net/dataspace/person/kidehen#this

[l00mi]

Hi, sorry I'm late to this conversation.

Thanks also from my side for all the effort from all participants put into this project. We used rdfstore.js in the beginning quite intensively for our projects, but eventually shifted because of the lack of progress a bit.

I had some exchanges with @antoniogarrote ca. a year ago. My interpretation is that after finishing his PhD he wanted to restart rdfstore.js with lessons learned in a complete rewrite (https://github.com/antoniogarrote/rdfstore-js/tree/1x). It seems though that he can not invest as many time he would like.

In general I have the impression he was never against involving other developers, that is one of the reasons he was ok two switch to the MIT license as we asked him back then (a894cd7d5aa68de569b4dfd65b18b0ed583a50c7).

I support the idea of @fkleedorfer to have at least a store where the patches get merged in a regular basis and also to have versioning. We should assure in this that @antoniogarrote is mentioned always as seed developer. Also please add him as an owner of the organization? Further I am happy to become a commiter of the fork. (Though not sure if of any use .. (-, )

[fkleedorfer]

@l00mi done. So I guess the next step is asking the people with unmerged pull requests to direct them at our new fork, and do ask people with parallel forks to make pull requests.. Right?

Also, do we agree to start with @trueg's fork?

[antoniogarrote]

Hi all.

Sorry for not having been able to answer before. It's great to see that there's still interest in the project.

Unfortunately I haven't had much time in the last months to dedicate to this project and my perception was that interest on it was very limited so I have been unable to make any progress with the project.

Any changes you want to make with the project or the code are more than welcome. If you need me to change the license, create an organization or transfer ownership of the code, please, just tell me. Having a true community of developers moving the project forward was always my intention.

I started a rewrite of the code on Christmas. It's in the '1x' branch. It cleans the code, add much needed support for tools like gulp and dependency management, test automation, etc. I's currently 60% complete. I would use it as the basis of any further development.

Scope of the ptoject needs to be reduced and focused. Part of the complexity of the code base originates on its research project nature where multiple engines are tried that need to be removed.

I'll try to follow the conversation to see if I can be of any help.

[kidehen]

On 2/28/15 2:28 AM, Antonio Garrote wrote:

Hi all.

Sorry for not having been able to answer before. It's great to see that there's still interest in the project.

Unfortunately I haven't had much time in the last months to dedicate to this project and my perception was that interest on it was very limited so I have been unable to make any progress with the project.

Any changes you want to make with the project or the code are more than welcome. If you need me to change the license, create an organization or transfer ownership of the code, please, just tell me. Having a true community of developers moving the project forward was always my intention.

Great to hear! As a community we really need to learn how to work together. Your contribution has been invaluable. We need intelligent clients and servers in the LOD ecosystem. Relying solely on intelligent servers is a flawed pursuit, from any conceivable perspective.

I started a rewrite of the code on Christmas. It's in the '1x' branch. It cleans the code, add much needed support for tools like gulp and dependency management, test automation, etc. I's currently 60% complete. I would use it as the basis of any further development.

Okay, so we somehow need to figure out the base for the pending merge via the new organization taking shape etc..

Scope of the ptoject needs to be reduced and focused. Part of the complexity of the code base originates on its research project nature where multiple engines are tried that need to be removed.

I'll try to follow the conversation to see if I can be of any help.

From my vantage point, the goal should be to have a SPARQL 1.1 DBMS for Javascript. This will enable the development of much smarter clients re., Linked Open Data exploitation.

— Reply to this email directly or view it on GitHub https://github.com/antoniogarrote/rdfstore-js/issues/86#issuecomment-76515324.

Regards,

Kingsley Idehen Founder & CEO OpenLink Software Company Web: http://www.openlinksw.com Personal Weblog 1: http://kidehen.blogspot.com Personal Weblog 2: http://www.openlinksw.com/blog/~kidehen Twitter Profile: https://twitter.com/kidehen Google+ Profile: https://plus.google.com/+KingsleyIdehen/about LinkedIn Profile: http://www.linkedin.com/in/kidehen Personal WebID: http://kingsley.idehen.net/dataspace/person/kidehen#this

[JuniperChicago]

My efforts on using this code has centered on the ability to persistently store a small graph locally while using the federation features. The most significant stumbling blocks in the existing code base for me has been the few missing components of SPARQL 1.1, and inability to query all named graphs by default (without UNION).

Antonio Garrote's 1x branch, imho, has set up the right direction in parsing (though incomplete) and indicates a possible abstraction layer or API between the query-engine and any kind of store one may choose to use... which is a great point of experimentation and development.

Antonio Garrote's rdfstore.js is a solid body of work and I think a good foundation for any revitalized effort... I would be very happy to see it continued.

[trueg]

While I very much like the idea of using the cleaned up 1.x branch as a basis I fear that none of us have the time to finish the missing 40% of it.

That is why I support the idea by @fkleedorfer to start with my branch and merge anything any of us might have in their branches.

Then at least we have a working version to maintain.

Thus, I propose this:

1. I will create a branch that extends the README to include the points that have been decided here
2. You will all create pull requests for your features on the new tree.

[RubenVerborgh]

About 2: With a tool such as hub, it is easy to get the code from all pull requests and merge it. Redoing the pull requests might not be realistic.

[trueg]

Very well, then I guess we need to gather the list of branches that we want to merge rather than re-do pull requests.

[ktk]

As it was not mentioned yet, one of the more feature complete projects is probably RDF-Ext by @bergos which is using RDFStore for the SPARQL store at least but that will be replaced with @RubenVerborgh SPARQL parser one day.

We basically add features as needed so pull requests are very welcome as well. Note that @bergos tries to focus on the parts no one else did. Turtle is done by Rubens parser, JSON-LD from the JSON-LD guys, SPARQL currently as mentioned from Antonio and there is a LDP implementation as well which Thomas implemented himself. It also provides Promise interfaces where appropriate.

[fkleedorfer]

@JuniperChicago

The most significant stumbling blocks in the existing code base for me has been the few missing components of SPARQL 1.1, and inability to query all named graphs by default (without UNION).

The inability to query all graphs is also the issue I have with rdfstore-js. Did you find a solution? Or does someone have a suggestion how this could be implemented in rdfstore-js?

[fkleedorfer]

@trueg I'm in favor of your suggestion - When I find the time, I'll try your version, most probably my modest changes are part of yours (adapting the JSON-LD implementation to changes in the JSON-LD specification). If this is not the case, I'll make a pull request.

[trueg]

@fkleedorfer I did not touch the JSON-LD code. So just point me to where those patches are and I will merge them.

[antoniogarrote]

Hi all. I have gone through the pending pull requests these last days, merged them and released a new version of the store: 0.8.4. It also inlcudes the latest version of the N3 parser from @RubenVerborgh.

You can use it as the basis for any future branch/fork. If somebody is interested in following development of this branch and help with pull requests in this github repository, please just send me an email to antoniogarrote@gmail.com and I can add him/her as a collaborator.

Some advice to follow development of this branch:

• Run the tests and the smoke tests for all the platforms
• Never fix or change generated code (e.g. the parser)
• Be careful with parsers and other dependencies. There's no proper support for them in the current branch so fixing bugs there is probably repeating the work already done in the original project for the dependency. It's better to update the code or report the bug in the original project.

Since there seem to be still some interest on the project, I will try to continue development of the 1.x branch that addresses a lot of the problems of the 0.x branch and should make the code easier for new developers. Any help is welcomed.

Regards.

[cygri]

Great work, @antoniogarrote. Good to see you back!

[satra]

thank you all for getting back and reviving this.

re: tests - one suggestion would be to start putting a travis.yml file so that every pr starts getting tested.

[JuniperChicago]

@fkleedorfer

The inability to query all graphs is also the issue I have with rdfstore-js. Did you find a solution? Or does someone have a suggestion how this could be implemented in rdfstore-js?

I have not coded a solution. I think the quickest and most definitive answer to the question as to how named graphs could be nested into a default graph, in the context of existing rdfstore-js code, would come from @antoniogarrote. If he could point to the best place to start, that would be ideal.

From what I can tell, the ability to traverse the contents of named graphs by default would both be contained in the query-engine (pattern used to discover named graphs within the triplestore) and the lexicon (system tracking of named graphs). Ideally, provision for enhanced treatment of named graphs could be provided for in the design of the triple store, likely benefiting from a quad-store (system level graph partitioning), or even quint-store (enhanced access control), but minimally speaking, the system could treat a named graph as a blank node.

I know I am oversimplifying this, and I have moments of clarity and then get lost in it. But I believe the traversing named graphs issues is another good reason for abstracting the query engine to enable a wider variety of persistent stores to be used and more features to be bolted on.

[antoniogarrote]

I've published the first version of the rewrite: 0.9.2. The codeis alread merged in master. Still not finished but integration tests are passing. The old code can be found in the 0x branch. Latest version published is 0.8.5.

I have tried to add some of the suggestions in the thread like, adding the project to Travis.

[antoniogarrote]

About the datasets for queries, this is a small summary of how the store works. I think the question is what should be the default behaviour:

Any query to the store accepts two sets of graphs as arguments.

• The first list is used as the default graph for queries without an explicit graph variable.
• The second argument is the collection of named graphs that will be used to resolve explicit graph variables. Additionally, the store mantains a default graph for the cases where no default graph is specified.

Provided the following insertion: INSERT g1 { t1 }

The following scenarios can happen:

1) query: select * { ? } default graph: [] named graph: []

• output: []

2) query: select * { g1 ? } default graph: [] named graph: []

-output: []

3) query: select * { ? } default graph: [g1] named graph: []

-output: [t1]

4) query: select * { g1 ? } default graph: [g1] named graph: []

-output: []

5) query: select * { ? } default graph: [] named graph: [g1]

-output: []

6) query: select * { g1 ? } default graph: [] named graph: [g1]

-output: [t1]

[trueg]

@antoniogarrote Thank you for taking this up again. It is very much appreciated. I am now trying to use the new version in my projects but I am running into some trouble: I did the build for the browser using $ gulp browserify but the resulting build does not define "rdfstore". Is any other configuration required?

[trueg]

I now created a simple file to enforce the "rdfstore" namespace. The file contains only one line and is the new input for browserify:

window.rdfstore = require('./store.js');

Not sure if that makes much sense - I am new to browserify and all its friends.

[antoniogarrote]

Hi everyone. This issue was becoming a pseudo-chat about the project.

I have created a gitter room to continue the conversation. You can join from the README of the project. Please go on with the conversation there, since it is really valuable.

I have a number of things almost ready, like the bower packaging, but things are going to slow a little bit for a few weeks since I'm looking for a new job.

Hope to start working on new functionality and making all the official W3C tests pass soon.