State of the Repo: 2019-07-19 (Feedback Requested)

[AkenRoberts]

Hi everyone, I'm Aken. I've been doing some issue triage and PR maintenance here for a few months. I came to the project hoping to reinvigorate some life into it, however I've found it difficult to do so for various reasons.

I'd like to share what I've learned about the current state of the repo, what I think we can do for it in the meantime, and what my long term vision for Doctrine Extensions is. I'd love to hear your advice and feedback.

Where We Are Now

Doctrine Extensions 2.4.x is a relatively stable product, with most functionality working as expected. Of the issues I've read, the most common causes and/or roadblocks are:

• Out of date dependencies and a requirement to support old versions of PHP
• Complicated abstractions making it hard to address one extension without messing with another
• Unexpected issues when combining extensions

The documentation for extensions, while reasonably thorough, is a bit scattered and vague in areas. Some docs include old, inaccurate changelogs which can confuse users (especially now that I added the root-level changelog).

Third-party libraries that are related, like StofDoctrineExtensionsBundle are becoming ignored. Even if we released a new major version, it would conflict with the bundle and likely other packages.

There are a good number of tests with seemingly good coverage (I haven't verified with coverage tools yet). PHPUnit is a bit old, and you need a specific environment to be able to run tests at the moment.

What To Do Immediately

To enable any tangible contributions to Doctrine Extensions at both the maintainer and PR level, I believe we should:

1. Release a new major 3.0 version, with all dependencies (PHP, Composer, etc.) bumped to their lowest maintained versions, some code cleanup and minor modernization, etc.
2. Deprecate the 2.4.x branch, essentially making it end-of-life with no further maintenance
3. Make running the test suite easier by at least adding more documentation and making the environment more flexible, and even better providing a Docker setup with everything needed
4. Triage all existing issues, likely resulting in most being closed due to age or outdated code. This should help with making sure that any new and open issues are confirmed problems still in the code base, and also helps maintainers understand what features are being used.

This will give the project as it stands a modern footing to be developed upon, by whomever is willing to spend the time. I've already started on the 3.0 dependency bumps on master, as well as a few minor test improvements. The other items are more procedural than development.

Long Term Vision

I would like to see Doctrine Extensions become more in control over its ecosystem, and a more well-rounded product:

• GitHub organization housing all related repositories and libraries
• Official vendor name, namespacing, etc., not related to a specific person or agency
• Official Symfony bundle
• Official website with high-quality documentation

The hardest part accomplishing this isn't technical, it's who is willing to put in the time. And that's my personal dilemma. I'm usually against starting from scratch overall, especially with something already popular. It's way easier to get existing Doctrine Extensions users to adopt changes and update their projects than to switch to an entirely new package. But I don't know the code. I don't use Mongo. I don't use XML to configure mapping. I don't use Symfony 2.

The reality is, if I were to spend a significant amount of time developing on Doctrine Extensions and getting it to the vision I see above, I would start from scratch. In fact, we already started. It's simply quicker to do all of the major changes we'd like to see, both in code and in organization, without having to properly deprecate, release, document, educate, etc.

How You Can Help

If you've read this far, thank you. ❤️ I'd really just like to hear your advice. What part(s) of Doctrine Extensions are important to you? Do you use Mongo? Are you willing to switch to a new package if it means a better overall experience?

My goal is to help make a friggin' awesome set of Doctrine extensions. I'm just not sure if I can do it here. What do you think?

[ndench]

Sounds like you've got a fairly solid plan. My only thoughts would be that you should do a few 2.x releases just to deprecate things you want to delete/change in 3.0.

Personally, I use this with the StofDoctrineExtensionsBundle on a Symfony4 project with MySQL. The features I use are:

• SoftDeletable
• Blameable
• Timestampable

I would be willing to switch to a new package if it provided everything I needed and was better is some way (ie. better maintained or new features that I wanted). But I'm largely pretty happy with the way DoctrineExtensions are now. I'm constantly updating dependencies and haven't run into any problems with this using outdated deps yet.

From your long term vision the only thing that seems difficult would be putting everything in a github organisation. Creating on official symfony bundle and updating documentation can both just happen alongside maintenance of this as it is from what I can tell. I'm pretty sure composer lets you re-namespace packages fairly easy but I've never had any experience with it (either from a maintainer or a users perspective).

What are the things you wanted to add/change that are difficult to do at the moment?

[andrewtch]

Tree and Translatable are the main things for me. Actually, I've my own bundles for Sonata integration with Tree and Translatable - might extract them later.

Also, I have something that can replace Loggable and Blameable - basically logging Flush events, including collection changes. Might extract it from one of the projects later.

@AkenRoberts , what's the Doctrine version you want 3.0 to target to?

[ThomasLabstep]

First of all many thanks to everyone who worked on this bundle.

I'm using:

• Timestampable: easy, does the job

• Loggable: extensively to track every change in our DB. But we accumulated 20% of corrupted log entries (total is ~half a million). We're purifying as much as possible. But some stuff keep failing. Doctrine keep duplicating with empty object_id or username.

Because we use tons of custom doctrine listeners. I've also built an activity log layer on top of loggable with custom handlers.

At this point, we are about to eject the original source code. It's just one annotation + a custom listener + a custom entity. Better to integrate this as part of our source code.

I think this repo was a good starter kit for who wants to customize doctrine heavily.

Feel free to reboot everything with modern architecture, Doctrine 3, docker and other fun stuff.

[AkenRoberts]

Very grateful to hear from you so far, thank you!

My only thoughts would be that you should do a few 2.x releases just to deprecate things you want to delete/change in 3.0.

The idea with 3.0 is to not really change any functionality, just get rid of all the old baggage. I believe the only real conflict might be external packages like the bundle, or anyone using a cutting-edge dependency version (e.g. Doctrine 3). Deprecations in 3.0 for a 4.0 release would be good, though.

What are the things you wanted to add/change that are difficult to do at the moment?

The wanted feature list isn't particularly difficult, it's just the existing code is so tied together and abstracted that it makes adding some things harder. After some modernization, separation, simplification (any more -tions?), it will be more straight forward to then start getting dirty with updates.

To me, that's one of the biggest upsides of a new package / ecosystem, you can start new and not need to refactor a ton of stuff. That's the battle, though - there are pros and cons to both maintaining this library and creating a new one.

what's the Doctrine version you want 3.0 to target to?

Extensions 3.0 will have Doctrine ORM 2.6 and ODM 1.2 as minimums. That won't require any changes to Extensions' code, just the Composer dependency requirements. As for Doctrine 3, I'll have to see what their release timeline looks like. Maybe we can support it in Extensions 4.

[xelax90]

Hi Aken,

we have started a new project (about half a year ago) with ZF3 Mvc and are using doctrine extensions as one of the core parts of our system. We use translatable, blameable, timestampable, loggable and tree. We are using a MySQL database and xml mapping (because our entities are a bit unconventional ^^). All of them work really great, although we didn't implement a lot with loggable yet.

Setting them up was kind of straight forward if you know how the doctrine module works. We can easily extract a ZF3 modle if you need it. We will happily switch to a new version when it's stable since this project will be the basis for many projects to come we also want to upgrade to Doctrine 3.

For feedback I want to say that blameable and timestampable work perfectly. Blameable was easy to set up with our authentication system. Loggable is working as intended. Even together with translatable but we didn't really use it yet. The complicated part here will be combining the logs of translations and main entity. Tree is really awesome, works like a charm and all you ever need for nested sets. In our setup we really need the NestedTreeRepositotyTrait because we cannot use inheritance at that point (please bring it back ^^).

Translatable is the weird one. It's genius and painful at the same time. The setup was kind of simple after we figured out how to do language based routing (which is more if a ZF issue).

The good thing is that our controller doesn't need to be aware of the translations. It's exactly the same for uses with and without translation. This makes it easy for beginners to write code when they are familiar with ZF. The genius part is the query walker which let us write complex queries without having to worry about translation too much. The painful part is working with multiple translations at a time. In essence it came down to working with arrays instead of objects and refreshing the entity multiple times to get the stored locale in addition to the translations. Sure, it works but that's 3 additional queries per affected entity and we had to assume that there were no changes to the entity at that point.

We decided not to use soft deletable because it's very simple to implement manually and we had the feeling it would make it harder to work with deleted entities if we needed them.

In general we are pretty happy with the extensions they're working nicely and I didn't find anything comparable yet.

[stephanvierkant]

Thanks for your contribution! I've tried to contribute as well, but I found it really hard because of the complexity of this library.

About my use case: I'm using PHP 7.3, Symfony 4.3 and MySQL/MariaDB. No Mongo, not need for support old versions of PHP/Symfony.

I'm using timestampable, blameable, sortable, uploadable, loggable, tree. I'm using StofDoctrineExtensionsBundle, but Symfony is getting better and better in support libraries without having to use a bundle: Symfony's autowiring, Symfony Flex, etc. What about trying to support Symfony without a (separate) bundle?

At least we should try to get it under one organization so we can assure a new bundle version will be released shortly after a new release of the library. Some PR's (https://github.com/stof/StofDoctrineExtensionsBundle/pull/233) are open for more than 5 (!) years and I don't think that's acceptable.

Anyone experience with https://github.com/KnpLabs/DoctrineBehaviors? To me, it feels like a waste of time to have multiple libraries with this kind of extensions. They've got a serious problems with finding maintainers, so why not cooperate?

(a bit offtopic: why doesn't show Github all maintainers of a repository?)

[isopen]

Maybe it's worth starting to improve from here. This should help this great package. https://github.com/Atlantic18/DoctrineExtensions/tree/archive/master-2019-03-17 https://github.com/Atlantic18/DoctrineExtensions/issues/2045 Perhaps the package should be decomposed by vendors.

[malarzm]

Hi @AkenRoberts! Doctrine member here.

Extensions 3.0 will have Doctrine ORM 2.6 and ODM 1.2 as minimums. That won't require any changes to Extensions' code, just the Composer dependency requirements.

Just wanted to chime in and say that ODM 2.0 is out for some time now and for any new projects we're strongly recommending using that over 1.x (which will likely be EOLed in 6 months). If you'd be bumping dependencies in next version I think you are safe to require ~2.0. There's a chance that either me or @alcaeus could give you a hand, should you need help with the ODM parts of the extensions :)

[AkenRoberts]

Cheers for the info @malarzm. I was trying to avoid any code changes needed in the Extension 3.0 release. However, given the number of breaking changes that may happen anyway, I may just upgrade to ODM 2.0 out of the gate. It's been coming up more often in issues and PRs lately.

I'll definitely ping if I could use any assistance from Doctrine's end. Thank you! 😘

[maxhelias]

As the stof/StofDoctrineExtensionsBundle bundle is no longer maintained we could directly integrate a Symfony bridge into the lib on 3.0. The community is big enough to give a hand. If you're interested by this idea, I can help for this part 😃

[AkenRoberts]

@maxhelias I think a separate package for the bundle is still best, but agreed that we can manage our own at this point. Especially because we'll need an updated version for 3.0.

Any contributions are welcome to the fork I started over here: https://github.com/doctrine-extensions/symfony-bundle

[maxhelias]

@AkenRoberts I full agree, I did not know that the development had already begun. I'll watch this when I have a little time 😉

[AkenRoberts]

Yeah, I forked the Stof package and started doing some cleanup. I haven't done a Symfony bundle before, though, so I'm learning at the same time. And also wondering how many versions back we want to support at this time.

Any contributions or advice, please open an issue. Any help is welcomed. 😄

[maxhelias]

With pleasure but the issue tracker is not available on the repo 😄

[AkenRoberts]

Fixed, thank you for pointing that out!

[yellow1912]

Thank you for spending your time on this. I'm pretty sure many doctrine users are using this lib. For us we are using at least:

1. Sortable
2. Translatable
3. Nested set

So fare there have been issues here and there, but they got fixed eventually one way or another. I like the idea of breaking this into sub repos, it makes contributing (and perhaps adding new sub repo) much easier.

I believe you can also find contributors easier that way. I for one will volunteer my time in maintaining the parts that I use.

[jorrit]

How are things going? Is the master branch the one that will be 3.0 in the future? It might be a good idea to release alpha versions of 3.0 if there is sufficient progress.

[AkenRoberts]

Hi @jorrit. Things are slow, as to be expected. master is indeed the next 3.0. I haven't done anything significant since the ODM 2.0 update.

Devs can use the 3.0-dev Composer version constraint to try it out. I may tag an alpha soon just to try and get it in front of people.

[AkenRoberts]

Well, it's been over a year. I'm grateful for the discussions, feedback, and contributions from everyone in that time.

While I haven't been able to give this project the attention I feel it deserves, I'm still trying to address what I can when my free time allows. master and the 3.0-dev tag are available for those who are able to help me beta test and find any issues.

As always, please open issues and/or share your ideas and concerns when possible. Thank you. ❤️