New version 4

[divine]

Well, while we are here, time has come, we need to work out on new major version, I've created a new project #2 and ready to work out on it!

Question, are we ready in new chapter of this library? We need to create 3.6 branch and start to work on version 4 which will be current master.

Bugs and fixes will be accepted for 3.6 branch, making this release still usable.

Laravel 7 is coming on 3rd March, probably we will be able to release on that day too (hopefully, no guarantees) !

Of course accepting/reviewing/pushing prs will be appreciated as always!

[divine]

There are two important breaking changes, I will try to explain the reason why we've decided and what should be done.

[divine]

Auto-casting of ObjectID to string (or anything else) will be dropped, for a long time relations are having issues when parent key is ObjectID but related key is string.

We can add casts helper inside Casts which will have something like this:

<?php

namespace Jenssegers\Mongodb\Casts;

use Illuminate\Contracts\Database\Eloquent\CastsAttributes;
use MongoDB\BSON\ObjectID;

class ObjectIDCast implements CastsAttributes
{
    /**
     * Cast the given value.
     *
     * @param  \Jenssegers\Mongodb\Eloquent\Model  $model
     * @param  string  $key
     * @param  mixed  $value
     * @param  array  $attributes
     * @return array
     */
    public function get($model, $key, $value, $attributes)
    {
        return (string) $value;
    }

    /**
     * Prepare the given value for storage.
     *
     * @param  \Jenssegers\Mongodb\Eloquent\Model  $model
     * @param  string  $key
     * @param  array  $value
     * @param  array  $attributes
     * @return string
     */
    public function set($model, $key, $value, $attributes)
    {
        return new ObjectID($value);
    }
}

and later used like this:

<?php

namespace App;

use Jenssegers\Mongodb\Casts\ObjectIDCast;
use Jenssegers\Mongodb\Eloquent\Model;

class User extends Model
{
    /**
     * The attributes that should be cast.
     *
     * @var array
     */
    protected $casts = [
        'address_id' => ObjectIDCast::class,
    ];
}

Decision for this breaking change is final, it has been discussed in slack and has been agreed by all collaborators.

[divine]

Embedding relations will be dropped as well.

We need to drop this completely, it has unknown bugs and never worked as expected.

However, if you need it just stick with 3.6 version or just push new PR to fix it without making anything to break.

Eloquent code provided a magic, you could do embedding relation on your own inside your documents without any problem.

Also with introduced type casting in laravel 7 this will be much easier as well.

Decision for this breaking change is final, it has been discussed in slack and has been agreed by all collaborators.

[divine]

After reviewing a lot of code, I have found a few more things which I will try to explain in the next messages.

I know, I could have just written this in one message but splitting it will help to understand it clearly.

[divine]

Remove shouldUseCollections function (see here ) which checks Laravel version > 5.3, it was introduced to support version 5.3 in https://github.com/jenssegers/laravel-mongodb/pull/925.

There is no reason why we're still keeping this.

[divine]

Remove support for keys in dot notation, it's a lot of custom code, do we really need this in the new version?

Dropping this as well will help to manage this library a breeze. We should be follow up Laravel decisions on Eloquent.

See here https://github.com/jenssegers/laravel-mongodb/blob/master/src/Jenssegers/Mongodb/Eloquent/Model.php#L141

This has been discussed. It will be kept as it is so nothing will be removed.

[divine]

Auto-casting of UTCDateTime to Carbon (and vice-versa) might be dropped, this is also an issue, we need to get rid of this.

There is no reason why we should force conversion of dates to UTCDateTime, we need to let user decide in which format they want dates, yet this is an on going issue. See https://github.com/jenssegers/laravel-mongodb/issues/1458, https://github.com/jenssegers/laravel-mongodb/issues/1731 and many more.

We can add casts helper inside Casts which will have something like this:

<?php

namespace Jenssegers\Mongodb\Casts;

use Illuminate\Contracts\Database\Eloquent\CastsAttributes;
use Illuminate\Support\Facades\Date;
use MongoDB\BSON\UTCDateTime;

class UTCDateTimeCast implements CastsAttributes
{
    /**
     * Cast the given value.
     *
     * @param  \Jenssegers\Mongodb\Eloquent\Model  $model
     * @param  string  $key
     * @param  mixed  $value
     * @param  array  $attributes
     * @return array
     */
    public function get($model, $key, $value, $attributes)
    {
        // Convert UTCDateTime instances.
        if ($value instanceof UTCDateTime) {
            return Date::createFromTimestampMs($value->toDateTime()->format('Uv'));
        }
        return $value;
    }

    /**
     * Prepare the given value for storage.
     *
     * @param  \Jenssegers\Mongodb\Eloquent\Model  $model
     * @param  string  $key
     * @param  array  $value
     * @param  array  $attributes
     * @return string
     */
    public function set($model, $key, $value, $attributes)
    {
        return new UTCDateTime($value);
    }
}

and later used like this:

<?php

namespace App;

use Jenssegers\Mongodb\Casts\UTCDateTimeCast;
use Jenssegers\Mongodb\Eloquent\Model;

class User extends Model
{
    /**
     * The attributes that should be cast.
     *
     * @var array
     */
    protected $casts = [
        'created_at' => UTCDateTimeCast::class,
        'updated_at' => UTCDateTimeCast::class,
        'birthday_date' => UTCDateTimeCast::class,
    ];
}

Decision for this breaking change is final, it has been discussed in slack and has been agreed by all collaborators.

[Smolevich]

https://github.com/jenssegers/laravel-mongodb/issues/1974#issuecomment-592857370 I think that is main good point to improve this library. If we can fix bugs with relations by field with type ObjectId and auto-casting ObjectId to string, it become good thing

[Smolevich]

https://github.com/jenssegers/laravel-mongodb/issues/1974#issuecomment-593107200 looks like as not important thing, but i think that we cam drop this things

[Smolevich]

https://github.com/jenssegers/laravel-mongodb/issues/1974#issuecomment-593113728 I think that we need investigate how can we impove this mechanism or if community doesn't need this mechanism, we can drop it

[divine]

Just writing this again, Laravel 7 support will be ONLY for the next major release.

We don't have any plans to support Laravel 7 on the version 3.X and this decision is final!

Want Laravel 7 support instantly? Help us here https://github.com/jenssegers/laravel-mongodb/projects/2

[fosron]

@divine So you are going to delay Laravel 7 support just because there are tasks that are not done for the new version? I think that's not a good way to go and force this. We would like to migrate to Laravel 7 and not wait for a month or two for this to follow, and we do not really use any advanced functionality that's changing/is being removed.

[divine]

@divine So you are going to delay Laravel 7 support just because there are tasks that are not done for the new version? I think that's not a good way to go and force this. We would like to migrate to Laravel 7 and not wait for a month or two for this to follow, and we do not really use any advanced functionality that's changing/is being removed.

What you are talking or trying to push is a paid product, but still missing the point that this is open source, nobody is getting paid for the work that has been done for years here.

Would like to migrate? Come help us here, what's the problem?

This library has been taken care only recently, we need to get rid of the some problems before releasing something new. This way only we as community could take for this library, nobody is going to fix something broken because as you said mostly everyone doesn't care about the problems.

In the end, yes, we are pushing and delaying new release ONLY to make it better for everyone, nobody cares about your urgent upgrades when at the same you don't give anything to this community back (helping maybe?).

Thanks.

[Smolevich]

@fosron you can see on last changes (we (especially @divine ) actualize all issues and pr's, there is 400 open issues 2 month ago) in this library and understand that is big deal. Also, one of our little community team (I, @divine, @Giacomo92 ) and owner @jenssegers has main work and spend our free time on supporting and impovements

[khakanali]

@divine Please let me know how can i contribute in this new release. I look forward to the opportunity to work with you.

[fosron]

@divine @Smolevich sorry, maybe my tone was not 100% right, i'm not against your contributions or the community, we really love the package you built and we are grateful, but at the same time it seems strange that an update to a latest version of Laravel is blocked by seemingly unrelated stuff and you are forcing contributions so people would get other stuff done for that. Maybe i'm wrong, hopefully i am, but even in the spirit of open source, there should be priorities, voting, etc. and people could help you not only build code, but decide on what is needed the most.

[bcorcoran]

Would it be possible to create a dev branch for 4.0 and update the composer.json to support Laravel 7?

[divine]

@divine @Smolevich sorry, maybe my tone was not 100% right, i'm not against your contributions or the community, we really love the package you built and we are grateful, but at the same time it seems strange that an update to a latest version of Laravel is blocked by seemingly unrelated stuff and you are forcing contributions so people would get other stuff done for that. Maybe i'm wrong, hopefully i am, but even in the spirit of open source, there should be priorities, voting, etc. and people could help you not only build code, but decide on what is needed the most.

@fosron let me try to explain this.

Currently the main priority for this library is to make it perfect for users and make it manageable for collaborators.

There are tons of the bugs and problems and nobody is taking care for them because it's hard to fix and test so we can't continue developing this library without forcing new changes and removing some of the unrelated stuff this library can't handle due to laravel limitations.

We're not forcing contributions, everyone is free to contribute and we would be really happy.

However, it takes time for even our self to make a contribution, believe me and @Smolevich has invested too much time only for making that much changes for the new version.

Everything has been discussed so this has been planned/agreed, only if you want it immediately join our slack channel and help us resolving issues we're having with castings of objects.

Would it be possible to create a dev branch for 4.0 and update the composer.json to support Laravel 7?

@bcorcoran PR #1988 was merged on jenssegers:develop branch, so laravel 7 support is there.

[VeeeneX]

We tested support with Laravel Nova 3.0 and Laravel 7, and it works! :+1:

[fosron]

Hey, any updates on the progress of v4.0?

[Smolevich]

Hey, any updates on the progress of v4.0?

We don't have enough time for new release. I hope soon we can to do it

[sayid]

When will you publish the V4.0? Im waiting for upgrade my projects from 6 to Laravel7.0

[Smolevich]

@sayid you try use alpha release

[mean-cj]

Hello @divine

Eloquent code provided a magic, you could do embedding relation on your own inside your documents without any problem.

Please example alternative for embed inside document.

I'm try not working., I missed something?

 $item = Item::create(["name"=>"AAAAA"]);
 $addon = new Addon(['name' => 'Test Addon', 'price' => 100]);
 $item->update([
  'addons' => $addon
 });

Thank you for your details.

[Giacomo92]

Hi @mean-cj , this is the alternative:

//Create example
$item = Item::create([
"name"=>"AAAAA",
"addon" => ['name' => 'Test Addon', 'price' => 100]
]);
$item->save();

//Update example
$updated_addon = ['name' => 'Updated Test Addon', 'price' => 499]
$item->addon = $updated_addon;
$item->save();

You have to think about "addon" like an array/collection field

[mean-cj]

Hello @Giacomo92

I like embed because

1. embed read performance better than multiple collections.
2. embed easy to where, find _id, update, create on inner sub-document.

Example addos [ { _id : ObjectId , name: "extra A" , price: 100}, { _id : ObjectId , name: "extra B" , price: 200}, { _id : ObjectId , name: "extra B" , price: 300}, ]

embed document is easy to find , check exists _id , or find where price > 200 I realized that I could use collections

thank you for your code : )

[Smolevich]

https://github.com/jenssegers/laravel-mongodb/issues/1974#issuecomment-629040095 you can try to use alpha version https://github.com/jenssegers/laravel-mongodb/releases/tag/4.0.0-alpha.1

[iranagame]

@VeeeneX Since you successfully use it along with Nova, would you please share with us your solution for edit/store data on Nova's ResourceStoreController because MongoDB doesn't support transactions and everyone has this problem.

[elfeffe]

What you are talking or trying to push is a paid product, but still missing the point that this is open source, nobody is getting paid for the work that has been done for years here.

@divine what about find some sponsors for this project? I have been thinking for years why Laravel doesn't support MongoDB, they are very proud that they always use the latest technology, but they are stuck into 90's databases. I think MongoDB is like homosexuality, no one returns after they try it. Maybe we can push Studio 3T, MongoDB, and some other companies (Laravel?) to sponsor it. Laravel sponsors Livewire, why they will not sponsor this one?

[Smolevich]

@elfeffe we don't communicate with companies directly, because we (@divine, @Giacomo92 and i) are part of community spend free time on support and improvements. Now, each collaborator has own reasons preventing continuing full work with library

[sayid]

@sayid you try use alpha release

em....it is an alpha version,I worry that it maybe has bugs

[Smolevich]

@sayid you try use alpha release

em....it is an alpha version,I worry that it maybe has bugs

Now I can only support this library(because we don't have active contributors), if you find bugs, report please

[fakingfantastic]

Hi there 👋, thanks for all the hard work on this.

I wanted to bring up an issue I'm running into: I was only able to get Laravel 7 to work with 4.0.0-alpha.1, but I am getting the "compileColumnListing()" error outlined in issue #2083. It seems that a patch for this was rolled out in the 3.6.5 release, but it is not yet included in 4.0.0-alpha.1.

Can we get that in there? I'm happy to do the PR if needed, I'm just not sure the process with how the branches are made for tagged releases (which branch do I make my feature branch off of, etc). If someone wants to give me some guidance there, that would be great.

Hope this helps, and thanks again!

[divine]

@fakingfantastic it has been a while and I was planning to update with our decision.

We have decided that it's not the right time to introduce a breaking change and we don't know what exactly do with issues that we're still having.

The new plan is that version 4 will be delayed and at the same there will 3.7 for L7 support and 3.8 for L8.

That's the plan, I will be working on it in a few days. Probably on Monday everything will be ready (depends how fast it will be reviewed).

Thanks!

[fakingfantastic]

Thanks for the breakdown, @divine . I'm not really caught up with the full need for the breaking change rewrite for v4, but it sounds like what I will currently need is 3.7 branch (for Laravel 7), and once I'm ready to upgrade to Laravel 8, I switch to 3.8.

Id be happy to help to contribute to the 3.7 branch if you make a starting point for it. I guess the first thing I'd do is merge the compileColumnListing() into it - but let me know if there is a more thorough path for me to get ramped up to contribute (beyond reading Contributors.md)

Thanks!

[divine]

Thanks for the breakdown, @divine . I'm not really caught up with the full need for the breaking change rewrite for v4, but it sounds like what I will currently need is 3.7 branch (for Laravel 7), and once I'm ready to upgrade to Laravel 8, I switch to 3.8.

Id be happy to help to contribute to the 3.7 branch if you make a starting point for it. I guess the first thing I'd do is merge the compileColumnListing() into it - but let me know if there is a more thorough path for me to get ramped up to contribute (beyond reading Contributors.md)

Thanks!

I will try to clarify, breaking change was needed because there are still problems and unknown amount of bugs with embedded relations that was integrated as eloquent relation.

Embedded relation is a good choice for mongodb, however for example pagination doesn't work well - you need to fetch whole document first and after that "magically" paginate it. When document becomes larger than 16MB you'll get an error and a lot of issues I don't remember whole list now.

---

Regarding 3.7:

Current master branch needs to be copied as 3.6 branch in case if there will be any breaking change in L6.

After that this PR hould be pushed to master.

For L8, we need to copy current master to it's own 3.7 branch and push new PR with any changes that it needs to support L8.

I hope you understood.

---

Guys, don't forget that we're working with @Smolevich and @Giacomo92 in our free time and any help is appreciated. We are all here to help community which needs Laravels missing MongoDB integration.

Accept my apologies that it took so long to release version 4.

Thanks!

[elfeffe]

@divine In my opinion embedded relations are not really an advantage, MongoDB has a lot more to offer for Laravel, I use normal relations all the time and I don’t think I need embedded ones. They are great, but not great enough for all the pain that they add for the developers (you).

Where is @jenssegers? Did he abandoned the project? He is looking for sponsors https://tidelift.com/funding/github/packagist/jenssegers/mongodb I’m sponsoring the project here in GitHub, is it the correct place to sponsor?

[divine]

@divine In my opinion embedded relations are not really an advantage, MongoDB has a lot more to offer for Laravel, I use normal relations all the time and I don’t think I need embedded ones. They are great, but not great enough for all the pain that they add for the developers (you).

Embedded relations has it's own use case but it doesn't mean that it can be used always and for everything. The problem happens when eloquent magic doesn't work and people are complaining. They're indeed right to complain but I don't know is it even possible to fix. Fix on the fix over fix? Doesn't sound great so that was plan to get rid of it.

Where is @jenssegers? Did he abandoned the project? He is looking for sponsors https://tidelift.com/funding/github/packagist/jenssegers/mongodb I’m sponsoring the project here in GitHub, is it the correct place to sponsor?

@jenssegers honestly doesn't have time to maintain this library (busy with new company) and we (as community*) just tried to help.

Tidelift is already sponsoring @jenssegers for this project.

We already asked Jens if Tidelift could help us with maintainers to resolve some issues, we didn't have any reply still for many months. It's really sad that communication is lost and without leader or active maintainer this library will be still active in a few years - that's questionable.

* Community of people that are interested or somehow relies on this library that met here and cooperated via Slack (let me thank you @Smolevich for that) Otherwise without community touch or help this project was going to be long dead, believe me or not.

It has been already said that we aren't getting paid and doing this in our free time** only to help for all of us.

** By free it means we are normally working and I didn't try to mean to sponsor @jenssegers and after that we will take care.

I myself not interested in getting paid for this at all and I don't think it will motivate anyone with so many issues/bugs however I did try my best to clean up issues and PRs only to be helpful for this library and make it easily maintained.

Overall what I'm looking for is the library that could be used and be working. That's all.

Maybe one day we will have official laravel integration for this hopefully.

I hope that everything is clear now.

Thanks!

[elfeffe]

The problem with embedded relations is that you have to rewrite the Laravel code to make them work. I don’t see the point because it’s really fast to load relations, and we already save a lot of time because of json data. I mean, we almost don’t use relations. I think there are many things that are more important. Like add the rest of the relations, or keep the package updated to work with Laravel. In the other hand, I can’t believe that this project will die because nobody wants to get paid to keep it updated. There are many people making very good money with GitHub sponsors and this package is a must for Laravel. I use it in all my projects and I’m not using MySQL anymore since I know MongoDB.

[fakingfantastic]

@elfeffe I think there are many things that are more important. Like add the rest of the relations, or keep the package updated to work with Laravel.

☝️This. Embedded relationship support should most definitely come second to the package not being compatible with Laravel 7 or 8. That should be a front and center priority.

@elfeffe I use it in all my projects and I’m not using MySQL anymore since I know MongoDB.

☝️This also. This package is the defacto standard for using MongoDB with Laravel, and if embedded relationships was as critical of a feature to cause upgrades to be blocked, then it probably wouldn't have reached 5k+ stars with ~3.5MM downloads. I'm not saying it wouldn't be an awesome feature, I'm just saying if the only way to support it is monkey-patching core, then that seems like the right idea to postpone it for now. Maybe better energy would be spent putting together a PR for Laravel to propose the exposure of the APIs needed to cleanly code integrate such a task. Without knowing a lot about the inner workings of Eloquent or MongoDB - I wonder if another use case for support in Laravel core would be JSON columns in MySQL and Postgres to also utilize an embedded relationship technique.

Still interested in how to get the L7 support branch going, so let me know 👍

[divine]

Just let you know, I've done some work today and L7/L8 releases are almost ready, only left is @Smolevich reviews, PRs will be pushed today night and reviewed tomorrow, releases are planned on Monday.

In the other hand, I can’t believe that this project will die because nobody wants to get paid to keep it updated.

@elfeffe I'm not the owner of this library and not the person who could decide this type of things 😉. However, the real problem here is that we are missing the leader who could decide and push changes towards improving this library and spending time for any issues that arise (it's not easy as it looks!)

---

I wonder if another use case for support in Laravel core would be JSON columns in MySQL and Postgres to also utilize an embedded relationship technique.

@fakingfantastic that's an interesting idea.

Thanks!

[elfeffe]

I have sent an email to MongoDB using their contact page. They work to have a PHP driver, they should know how important is for us to have a proper Laravel integration. Maybe they can help. @jenssegers has abandoned this project, so nobody cares who is the owner. Anyone can create a fork and begin from there. Also we can tell Taylor (the Taylor) about how important MongoDB is. I can’t believe that Laravel, that is always using the latest technology, is not interested at all on MongoDB.

👉https://twitter.com/federeggiani/status/1305202757067042817?s=21

[divine]

I have sent an email to MongoDB using their contact page. They work to have a PHP driver, they should know how important is for us to have a proper Laravel integration. Maybe they can help. @jenssegers has abandoned this project, so nobody cares who is the owner. Anyone can create a fork and begin from there. Also we can tell Taylor (the Taylor) about how important MongoDB is. I can’t believe that Laravel, that is always using the latest technology, is not interested at all on MongoDB.

👉https://twitter.com/federeggiani/status/1305202757067042817?s=21

Taylor doesn't care - end of the story. This has been already a feature request - asked billion of times by many different people, I really don't know maybe it might be too much hassle for them and was postponed.

MongoDB team wouldn't able to help, some important issues are related to Laravel itself as Eloquent wasn't planned in integrating NoSql database.

This library isn't abandoned completely, however community needs well supported Laravel integration.

You might be surprised but I'm also hoping that one day everything will work perfectly and I would be really happy, who knows as I've already invested a lot of time personally to this maybe I need to accept and continue my work but I simply can't alone do this *. 🤔

* Some parts requires dedicating and making decisions, it's not always about writing a code.

[divine]

Hello,

Just an update, we have released new version 3.7 with compatibility for Laravel 7.

Release for 3.8 will be coming in a few days, sorry as it's being delayed because checking issues and confirming bugs takes time.

Thanks!

[elfeffe]

Great @divine ! Thank you!

[fakingfantastic]

@divine thank you for making this a priority - and let me know if there are specific actions I can take to help move the project forward

[fosron]

@divine thank you!

[divine]

Just another update, we've released new version 3.8 with compatibility for Laravel 8!

You're all welcome!

---

@divine thank you for making this a priority - and let me know if there are specific actions I can take to help move the project forward

For now everything looks good, except decisions should be made for new version 4, basically everything will be fine until Laravel 9 and even at that time we will probably release 3.9 without breaking changes.

Thanks!

[henrypham299]

@divine thank you

[9jaGuy]

thank you guys