search

jansmolders86 / mediacenterjs

A HTML/CSS/Javascript (NodeJS) based Media center
http://mediacenterjs.com
1.29k stars 243 forks source link

Prevent 2 shows with the same name from being created #206

Closed andreme closed 8 years ago

andreme commented 8 years ago

When 2 moviedb.searchTv requests for the same show ended at the same time, there was a chance that 2 records with the same name were created.

Jon889 commented 8 years ago

I'm slightly confused how they could end at the same time as Node is single threaded? Also this should be handled by in the database layer. Else we'll be sticking locks around all the findOrCreate calls.

Jon889 commented 8 years ago

I think upgrading sequelize will fix this. We are on 1.7, and the current version is 3, https://github.com/sequelize/sequelize/issues/3002

andreme commented 8 years ago

I have updated sequelize, but that has actually made the problem worse. I.e. when adding new episodes, all inserts except the first will fail because the first transaction is not completed. I will change it to serialise the inserts, but that's not really a great solution.

Executing (fdd42fdc-7301-4e84-91c0-02770a72aaca): BEGIN TRANSACTION; Executing (f82c8398-a760-4c2a-9dad-42cf2ad6c8e1): BEGIN TRANSACTION; ... Executing (fdd42fdc-7301-4e84-91c0-02770a72aaca): SELECT id, filePath, name, season, episode, ShowId FROM Episodes AS Executing (f82c8398-a760-4c2a-9dad-42cf2ad6c8e1): SELECT id, filePath, name, season, episode, ShowId FROM Episodes AS ... Executing (fdd42fdc-7301-4e84-91c0-02770a72aaca): COMMIT; Executing (7da002be-9488-4bcd-bc9f-3569a75c45c9): COMMIT; ... Unhandled rejection SequelizeTimeoutError: SQLITE_BUSY: database is locked Unhandled rejection SequelizeTimeoutError: SQLITE_BUSY: database is locked ...

Jon889 commented 8 years ago

Why is it even trying to insert multiple times?

On 13 Oct 2015, at 14:27, andreme notifications@github.com<mailto:notifications@github.com> wrote:

I have updated sequelize, but that has actually made the problem worse. I.e. when adding new episodes, all inserts except the first will fail because the first transaction is not completed. I will change it to serialise the inserts, but that's not really a great solution.

Executing (fdd42fdc-7301-4e84-91c0-02770a72aaca): BEGIN TRANSACTION; Executing (f82c8398-a760-4c2a-9dad-42cf2ad6c8e1): BEGIN TRANSACTION; ... Executing (fdd42fdc-7301-4e84-91c0-02770a72aaca): SELECT id, filePath, name, season, episode, ShowId FROM Episodes AS Executing (f82c8398-a760-4c2a-9dad-42cf2ad6c8e1): SELECT id, filePath, name, season, episode, ShowId FROM Episodes AS ... Executing (fdd42fdc-7301-4e84-91c0-02770a72aaca): COMMIT; Executing (7da002be-9488-4bcd-bc9f-3569a75c45c9): COMMIT; ... Unhandled rejection SequelizeTimeoutError: SQLITE_BUSY: database is locked Unhandled rejection SequelizeTimeoutError: SQLITE_BUSY: database is locked ...

Reply to this email directly or view it on GitHubhttps://github.com/jansmolders86/mediacenterjs/pull/206#issuecomment-147713591.

jansmolders86 commented 8 years ago

For what it's worth I bumped the sequilize version in the package.json to 3.12.0

*edit, bumped broke more than I had liked. Appearantly the syntax changed quite a bit

I'm kind of relying on your expertise @Jon889 on wether or not I should merge :)

Whichever the case may be, thanks for the work @andreme ! :+1:

andreme commented 8 years ago

The reason multiple findOrCreate calls run at the same time is because it's async:

1 file found 1 findOrCreate started, control given back to node 2 file found 2 findOrCreate started, control given back to node ... 1 findOrCreate finished 2 findOrCreate finished ...

I'll update the PR to have it serialise the requests.

andreme commented 8 years ago

I've fixed the last problems, everything is now working I believe. All changes outside apps/tv/metadata-processor.js are due to sequelise 3.

Jon889 commented 8 years ago

Yeah I get it's async but why are there multiple calls for the same show at the same time?

andreme commented 8 years ago

Oh because it's adding more than 1 episode (files) for a show. Let's say there is a new folder with a full season for a show that didn't exist before, processFile gets called X times and creates the episode, then after the moviedb lookup it tries to create/find the show X times.

jansmolders86 commented 8 years ago

Seems to work great for me! Thanks @andreme for the help! Sorry it took so long to get this approved!