Programs not saving to database

[terracoda]

Copied Programs not Saving I copied my X and X squared programs Tuesday around 4 pm CT, and then when I loaded PP the next morning they were not there. I assumed I had missed a step or had done something wrong. I copied them all again, and just noticed my copies did not stick. The database looks the same as it did this morning.

To Reproduce Steps to reproduce the behavior:

1. Go to 'ts-creator-tests'
2. Copy the X Tile program
3. Change the auto-generated number
4. Save Program and send to Play ground
5. They copied program should exist and work.
6. Quit Paper Playground
7. Quit everything
8. Re-start and re-open and copy might not be there.

I just happened to have taken a screenshot this afternoon while making copies. You can see I have copies of X Tiles:

I don't have those or the rest of my copies anymore.

[jessegreenberg]

I just reproduced the problem with the tactiles project. It is another case of the payload being too large, we are getting this response back from the server: "413 (Payload Too Large)"

So it seems we cannot save the entire project all at once. This one is going to be tricker than #188 because the projects table only has one entry for the project data - a JSON object for the entire thing. We will probably have to change the table or edit the saved JSON without reloading the whole thing.

[jessegreenberg]

The above commit is working well, its in a feature branch chunk-saves. Before merging into main I want to

• [x] Make sure this strategy works/handles sending multiple save requests before the first request finishes.
• [x] Add a timeout so that if the server doesn't receive all chunks, we bail corectly.
• [x] Look for other places where this payload error can happen.
• [x] Lots of testing of save/load for existing projects.

[jessegreenberg]

Look for other places where this payload error can happen.

Inspecting the api, here is where this can happen again potentially:

• /api/spaces/:spaceName/programs (POST): - When you create a new program from the camera page. Will happen if code is too large.
• /api/spaces/:spaceName/programs/add-premade-program (POST): Sends a single program to playground from Creator, part of the chunking from #188. If the program is too large, we will hit this.
• /api/creator/templates (PUT): Saves a new template for creator. Just like this issue, if the template is has ~200 components or more, we will hit this.
• /api/creator/templates/update/save (PUT): Again, if template is massive when you update it, this will happen.
• /api/creator/maintenance/updateSchema (POST): (Not really worried about this, its for development use)

Could also happen if the user tries to upload massive image/audio files.

The size limits are way bigger for receiving data from the server. Since our entire db is less than 1mb, we don't have to worry about this when loading data from the db.

In summary, one of these is from the original paper programs code and the rest are not likely enough to hit to prioritize right now (IMO). However this may be something @brettfiedler wants to track for project management. I think this is most likely to happen again for massive programs (where a single program has >200 components or tons of custom code).

[jessegreenberg]

Lots of testing of save/load for existing projects.

I tested by

• Saving and loading small changes to a test project.
• Saving and loading every created template in a test project.
• Saving and loading changes to a copy of @terracoda's tactiles project.
• Saving and loading a project with 21 programs with 10 components each (210 components total)
• Saving and loading a project with 41 programs with 10 components each (410 components total - it took about 20 seconds to load)
• Saving and loading a project with 81 programs with 10 components each (810 components total - it took a while but was successful)
• Saving and loading a project with 101 programs with 10 components each (1010 components total) - After 10 seconds I hit the timeout error I added in https://github.com/phetsims/paper-land/commit/ad6784b23289a6732cb5d459d14b3810621d0d82. But it DID successfully save. Loading this much took ~30 seconds.
• Then delete all but 10 programs and save/load that.

[jessegreenberg]

This was merged into main. I did some spot checking on main and it is OK.

[jessegreenberg]

@brettfiedler can you please review https://github.com/phetsims/paper-land/issues/202#issuecomment-1814735146 and let me know if there is anything else you would like to do here? For example, place a limit on the number of components a single program can have or something like that? Or spend more time chunking up the data within each program?

[brettfiedler]

Opened a new issue in #250 to track future occurences.