A better backend for Plone

[volkerjaenisch]

Dear Guillotina people!

I read all your presentations and get lots of pointers from Plone involved people when I asked for a better Plone Backend towards your project.

I learned that Guillotina uses Postgres as Backend DB and become a bit happy. Since Postgres has a good security footprint, is flexible and has a quite sophisticated authorisation capabilities.

So a sythesis of the strength of PG with Zope-Component-Architecture and Plone like constructs like behaviors sounds like heaven to me.

The reality broke my angle wings fast:

• The PG-DB is used like a cheap NO-SQL object-store.
• The security is a Clone of the Zope/Plone security model, without leveraging any capabilities of the PG-Backend.
• All object data is a simple JSON dump in the Column "json" of the "objects" table. Tell me anyone what the improvement over the ZODB is?

Last year on a project for a big player I dumped the software CKAN for exactly the same reasons. So I will not use Guillotina. We came up successfully with a Plone based solution.

Truth is Plone is old and needs a replacement. I hear birds singing that Plone 6 or 7 will be based on Guillotina. Only over my dying corpse.

Together can come up with a good Backend for "Plone". At the moment a lot of Frontend-Developers are managing Plone. May be its time to shift the focus to the backend. E.G. utilizing the real power of postgres. It will be hard it will cost but in the end it will get us a solution that can stand against the old Plone and win.

Cheers,

Volker

[vangheem]

The PG-DB is used like a cheap NO-SQL object-store.

yes, "cheap NO-SQL object" was a design choice. How would you fit a relational model into plone object model? Do you have ideas/examples? (hint, we like traversal...)

The security is a Clone of the Zope/Plone security model, without leveraging any capabilities of the PG-Backend.

Yes, we like the zope/plone security. It is a major strength of plone. Also, our backend is pluggable and we are resistant to implementing too many things that would completely tie us to pg. The simple, nosql like implementation is an advantage. The jsonb field you noticed is an implementation of a search catalog for pg but you can use elasticsearch for example.

All object data is a simple JSON dump in the Column "json" of the "objects" table. Tell me anyone what the improvement over the ZODB is?

I don't understand comment. All the object data is not a json dump. I don't hate the zodb--it just didn't scale. Guillotina is closer to orm than zodb. We don't completely abstract db interfaces away like zodb but we also do full object serialization. There is interest in moving away from pickle to protobuffer. full json is quite slow. Full object serialization to column rdms modeling is generically slow and has lots of trade-offs I'm currently not interested in. I suppose someone could maybe write an object reader/writer for it?

I hear birds singing that Plone 6 or 7 will be based on Guillotina.

It will not. Guillotina is not a replacement for plone. There is https://github.com/plone/guillotina_cms which tries to reimplement CMS things in a plone-like way but it's incomplete.

Together can come up with a good Backend for "Plone"

Maybe; however, I'm not very interested in replacing plone right now--it's not the problems I'm solving.

@bloodbare might be a bit more interested in it so maybe he has some ideas.

I encourage you to read some of the docs if you're interested about the project.

[bloodbare]

Maybe you are looking for a SQL/URL dispatch framework like Django or Pyramid. Guillotina is mostly a schemaless data paradigm with a traversal URL design with ZCA and an API similar to Plone REST API.

If you want to talk about our technical decisions we are welcome to explain all of them!

Good luck!

[volkerjaenisch]

Dear @bloodbare

No. I am not looking for Django etc. I developed about 20 years with Zope/Plone and I like to stay. Guillotina is the first project that IMHO can be a Plone replacement - if some obstacles OF PLONE! can be overcome:

• slow security
• slow traversing due to slow securtiy, etc.
• slow indexes
• slow inserts, caused by slow indexers Most recent Plone projects I know of are
• using the Plone RestAPI
• utilizing external Indexers to speed up search operations
• bringing up fancy Frontends to Plone like Volto But there is no really development at the backend performace. I am a backend developer. I like performance optimization.

If I ask my Plone companions why they stay with Plone they say (in descending order)

• The security model
• The security footprint
• The batteries included (Workflow etc.) and all of them hate Plone for several reasons.

I really like that guillotina uses the zope/plone interfaces and the overall conception. In the backend I see lots of improvements, if postgresql is really maxed out. The API to the bakend is really neat - so I may dock on easily.

I will make a fork and play a bit around with the guillotina backend. Making is better than talking. :-)

Cheers,

Volker

[jordic]

Hey, guillotina, a part from the data layer, it's a full featured web framework with a lot of interesting things :)

Here I built just a POC of integrating guillotina with sqlalchemy (in async world). Just take a look, https://github.com/jordic/guillotina_asyncom

It's based on a package we built to work with sqlalchemy in asyncio. (asyncom),

[jordic]

Yep, @volkerjaenisch We are an active users of guillotina, and we use on a medium postgresql OLTP db, we are very interested in improving the postgresql backend, with as much as possible features from postgres that make it robust, simple and fast, and allows us to integrate better with the rest of the backend.

We started writing a new schema (using sql posibilities), and trying to normalize things and figure the best. We can share the spec with you and try to iterate over it. We are confident, that guillotina with a better postgresql schema and all the benefits of asyncpg, can be a powerful generic system for data management.

One of the first questions that had come after reading your issue, it's around security. Are you tinking on something similar as how postgresql is used with postgrest? Use the postgresql roles machinery to build the security around the obejct tree? And use a SET role 'xxx' on every request to enforce security on the db layer? (That sounds good, because with something like this we will be able to remove a big security vector)

If that's the case, there are at least a pair of things that are not clear to me:

1. Do you have any ideas on how to handle the security for local_groups adoption, after reviewing some projects around the (tree) on sql like (https://github.com/Kotti/Kotti/blob/master/kotti/resources.py#L204), anyway kotti, it's not using any form of RSP on postgresql.

2. Right now, guillotina tries to be user agnostic, there is no user model neither group model (something like on zope/plone) where authentication can be delegated to other systems (ldap, sql... ), we can also provide something like this, but we must think a bit on it. Anyway, on my opinion, I prefer to ship something ready, that can be costumized versus something that misses parts.

If what I'm writing makes sense for you, and you are still interested on evolving this part of (guillotina), and "plone", we can start working on something. Are you available to chat a bit on something like this, I'm pretty sure that together we can write a good postgresql backend.