Minimum code to test if an Entry at first root level seems possible

[Hettomei]

Linked to https://github.com/igrigorik/vimgolf/issues/307#issuecomment-813369879

[Hettomei]

@filbranden

it is in WIP but it would be good if you can confirm that the big part seems to work.

You need to bundle exec rake db:drop db:setup challenges=10 users=30000 entries=100000 to have the "entry as a document"

Of course, if it works, I will add the code to migrate from Challenge.entries to Entry as a document

[filbranden]

Interesting idea!

I wonder if we could instead look into challenge.entries.order_by([:score]).each to enumerate all challenges and then do the processing in Ruby (filter out multiple entries by the same user and count them until filling enough to populate the current page.) I'd expect that would work without requiring a change of the MongoDB schema...

[Hettomei]

I m not afraid of a database migration :

• can you make a backup ?
• then we put heroku down for maintenance
• run the migration
• restart heroku
• enjoy

Of course, if https://github.com/igrigorik/vimgolf/pull/318 works, then merge it, but I need to know : should I continue in this direction or you really don't want a database migration ?

[filbranden]

I think this is better than #318, that approach still have performance issues and doesn't seem to solve the problem as much as this one does. (My suggestion into avoiding data migration was due to ease of deployment, ease of rollback, etc. but I agree that's not a full reason to rule it out.)

So I agree with your suggestion to move Entries to become its own collection and move from there.

can you make a backup ?

Yes. There are automatic daily backups, but I just checked and I can use mongodump locally to do an ad-hoc backup before data migration.

then we put heroku down for maintenance, run the migration, restart heroku, enjoy

Sounds like a plan.

but I need to know : should I continue in this direction or you really don't want a database migration ?

Yes, let's go with your approach. Can you clean up this PR and write a script to migrate the data?

I'm happy to execute the migration.

[Hettomei]

Thanks for the detailled answer

Can you clean up this PR and write a script to migrate the data?

Yes of course. That was a kind of POC, and that s why it s a 'draft'

Maybe I'll take more than 2 weeks to do.

[filbranden]

Maybe I'll take more than 2 weeks to do.

Let me know if you'd like me to help... I'm happy to write parts of the code, or even pick up from where you left off... (Just don't want to do that if you'd prefer to work on it yourself.)

Cheers! Filipe

[Hettomei]

Go and do all the work you want. I already took 4hours for this week, that s a lot for me.

If you create a new branch, here or on your own repository, I will checkout your data before continuing.

So, feel free to exploit this POC

[Hettomei]

No more needed 😄