search

maxohq / surrealix

A lean and correct Elixir SDK for SurrealDB
MIT License
14 stars 0 forks source link

Examples integrating with umbrella elixir, phoenix, etc #3

Open aislanmaia opened 8 months ago

aislanmaia commented 8 months ago

Would be great to have some examples, maybe some repos integrating this package with real elixir projects.

jamilabreu commented 6 months ago

+1 @mindreframer

jamilabreu commented 6 months ago

@aislanmaia found this demo repo https://github.com/maxohq/surrealix_demo, but more directed instructions would be amazing!

mindreframer commented 6 months ago

@jamilabreu Yeah, sorry about missing docs. In our project we actually decided to move away from SurrealDB for now and stick to plain Postgres. It solved all the performance troubles we had with SurrealDB with non-trivial amounts of data. Also the visibility and troubleshooting is simpler. Maybe SurrealDB becomes more usable for real-life projects in future, then we will invest more time into documenting the usage of Surrealix, for now this project is not in active use at our company

lessless commented 4 months ago

@mindreframer what issue did you face with surreal? Would you advice to approach choosing SurrealDB as a primary data storage with caution?

mindreframer commented 4 months ago

@lessless we had large rather nested json payloads (JIRA issues), that we were storing in SurrealDB. Not too much, around 30k records. Query performance for single records with relations was good, but doing any searching / filtering / pagination was painfully slow (500ms or longer). Also the indexes at that time were not picked up by the query planner, so that it always had to do a full table scan for any "collections" query.

Those issues were quite visible in Github issues, yet the team was focusing on implementing new features rather than fixing the basics. There were also a couple minor casting issues, documentation discrepancies and overall too much hype around things I personally do not need and less focus on storage basics. I would suggest generating a realistic amount of data for your app, dumping it into SurrealDB and running some plain queries to see how it performs. Pay attention to the storage amount on the file system, memory usage during non-trivial queries, query execution times, handling of parallel-running queries. Do it with something simple, maybe use their Node.js SDK or similar. This is a throw-away project, do not over-engineer it. See how it works with low digit GBs, how long does it take to create indexes, are queries using them. A database that ignores indexes is mostly useless for real production cases.

See https://surrealdb.com/docs/surrealdb/faqs/#is-surrealdb-ready-for-production-use.

While mine intention was simplifying our stack, it ended up actually complicating it. Switching to Postgres resolved all query issues and I also realized that I did not need the flexibility of SurrealDB that much, as I imagined. Also the tooling around Postgres is so robust, that I could just focus on my app and not bother fixing DB issues.

YMMV, for me it was clear, that it's too early for real production for SurrealDB. Maybe next year it will be more robust, until then this SDK is not maintained.

lessless commented 3 months ago

Thank you for the detailed feedback, @mindreframer. We have a reasonably large dataset, and our queries can take over a minute to execute without indexes. So, it's a no-go from the onset.