NoSQL database

[ng5]

Hi Craig,

thank you for building this open source project. i have checked out the source yesterday and i managed to import QIF transaction ( programmatically!! ) , code is very well written and it appears to be high customizable. I'm working on a web application that allows user to upload OFX or QIF investment accounts to their portfolio.

we use MongoDB for this application. We have good reasons to use MongoDB rather than SQL database, this was decided after recognizing data access pattern via single BSON are much faster than SQL even with an index. Anyway database is hard to switch to at the moment.

Is it possible to extend AbstractJpaDataStore to use nosql databases like MongoDB. i'm happy to work on it by myself and share the code with you.

But i would rather ask your opinion on it first. Did you choose to use SQL because this is ofcourse an accounting system and you definately need ACID properties. In our application there is no need for this and keeping records in a single document is good enough and it provides eventual consistency.

hope to hear your thoughts on it.

Regards, Rohit

[ccavanaugh]

Hey Rohit,

I've poked around potential use of NOSQL, but have not tried it. Hibernate offers a NOSQL backend that should be mostly transparent that supports MongoDB and Neo4j and plugs into the existing JPA architecture. I'm mostly holding off for the Java 9 transition as I expect a bumpy road ahead and would be more inclined to add another backend storage option after that.

The SQL approach was primarily chosen in support of the multiple client / server functionality that some users needed. If it was not for that, I may have not gone that direction. The SQL backends allows for remote storage without having to write custom code for jGnash. It certainly improves the robustness if a crash occurs, so I can't complain. I'm not much of an SQL export, so I let Hibernate do the heavy lifting.

The OFX standard is the way to go and I would abandon the QIF approach. QIF is quite broken and non-standard which make parsing and importing difficult.

Regards, Craig

On Tue, Jan 3, 2017 at 2:12 AM, Rohit Vishwakarma notifications@github.com wrote:

Hi Craig,

thank you for building this open source project. i have checked out the source yesterday and i managed to import QIF transaction ( programmatically!! ) , code is very well written and it appears to be high customizable. I'm working on a web application that allows user to upload OFX or QIF investment accounts to their portfolio.

we use MongoDB for this application. We have good reasons to use MongoDB rather than SQL database, this was decided after recognizing data access pattern via single BSON are nuch faster than SQL even with an index. Anyway database is hard to switch to at the moment.

Is it possible to extend AbstractJpaDataStore to use nosql databases like MongoDB. i'm happy to work on it by myself and share the code with you.

But i would rather ask your opinion on it first. Did you choose to use SQL because this is ofcourse an accounting system and you definately need ACID properties. In our application there is no need for this and keeping records in a single document is good enough and it provides eventual consistency.

hope to hear your thoughts on it.

Regards, Rohit

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ccavanaugh/jgnash/issues/20, or mute the thread https://github.com/notifications/unsubscribe-auth/AAm4Il8TTuABmCEUSC8BGb5FR6boy5_Iks5rOfTDgaJpZM4LZYqY .

[ng5]

Thank you Craig for your response.

[kumarpranay]

Out of curiosity; why would you choose to have multiple backends for storage on this software. If one is known to be most reliable but slow (say XML ZIP) and other the fastest (say db); should you not limit to these two. Having more means more support and different issues for different users.

[ng5]

Hi Craig,

I'm working on a separate web application that requires ofx and qif parser. This appplication uses MongoDB as storage because after a portfolio has been imported into the application( using ofx or qif format). there will be some analytics run on portfolio that will require timeseries access of market data.

MongoDB(NoSQL) is ideal for timeseries data because full timeseries can be stored as a single document that will give a much faster access.

Regards, Rohit

On 6 Feb 2017 7:59 a.m., "kumarpranay" notifications@github.com wrote:

Out of curiosity; why would you choose to have multiple backends for storage on this software. If one is known to be most reliable but slow (say XML ZIP) and other the fastest (say db); should you not limit to these two. Having more means more support and different issues for different users.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ccavanaugh/jgnash/issues/20#issuecomment-277608663, or mute the thread https://github.com/notifications/unsubscribe-auth/APWuBpPfaS5x9eWO-NKtozXh2sJALsXnks5rZtLigaJpZM4LZYqY .

[ccavanaugh]

The initial driver for multiple backends (JPA/Hibernate) was to support client / server access whereas the xml/bxds would be messy. The relational database provides fault tolerance on slow unstable systems.

The xml/bxds are more or less that same, just a different XStream constructor. These two are the fastest but not fault tolerant... especially on slow Windows systems with a large file.

The current two different relational databases (H2, HSQLDB) do have some different performance characteristics, but primary reason for have two was a backup just in case one fizzled out and it helped ensure I was not too closely tied to one database. There was a push to use Derby, but it's file structure is not a single file solution like H2 and HSQLDB.

I was bit by db4o being purchased and phased out and the xml tool I started out with years ago went closed source and as since died off.

I agree that more backend types is more effort and most users don't care.. they just want something that works well.

This is a low priority relative to UI and features. If there was to be a NoSQL solution, I think it only makes sense on the server size... I'm making up it also has a file structure that is difficult to manage for those who just want something that works.

[deepakbabu-vine]

Hi Craig Thanks for your application which is very clean. We are using the Postgres database, so can you use the Postgres database in our code?

Thanks for reply in advance.

[devanand-sysvine]

Hi Craig, Thank you for the open source application. Am trying to implement the same in connection with Postgres Database, I was facing the issue with "Did not receive a valid response from the server" with the post configuration of Postgres DB. Meanwhile with my configuration in persistence.xml got the message as - Connected to remote message server, from message bus class.

Is there any kinda configuration that i miss to achieve the Postgres DB.

Regards, Dev Anand