NNTmux / newznab-tmux

Laravel based usenet indexer
GNU General Public License v3.0
210 stars 52 forks source link

Really stuck with insert issues #1193

Closed blimeybloke closed 2 years ago

blimeybloke commented 2 years ago

brand new master build how do I switch from master to dev build? Maybe that will help some of these issues.. Seem to be having issues with inserts into the db. I do see releases in the releases table but nothing on the actual site. So 2 seperate issues.

Php 7.4 and MariaDB 10.2

Here is a snippet from the laravel log tail -f laravel-2021-11-16.log VALUES (UNHEX('d1a948e43662ae42c049a6c42453ee70'), '(03/91) - ZnZZZZZZZZZZZZZZ - \" Release 09978334383805.vol00+01.par2\" - 4.55 GB - yEnc', 2042, 5, 1, 3, 150241) ON DUPLICATE KEY UPDATE currentparts = currentparts + 1, partsize = partsize + 150241) [2021-11-16 06:35:06] NNTmux.WARNING: 1 articles failed to insert! [2021-11-16 06:38:43] NNTmux.ERROR: SQLSTATE[23000]: Integrity constraint violation: 1452 Cannot add or update a child row: a foreign key constraint fails (nntmux.binaries, CONSTRAINT FK_Collections FOREIGN KEY (collections_id) REFERENCES collections (id) ON DELETE CASCADE ON UPDATE CASCADE) (SQL: INSERT INTO binaries (id, partsize, currentparts) VALUES (4063,58679846,148),(4062,2790928,7),(8411,59478520,150),(8412,59476937,150),(8413,59477357,150),(8414,59478084,150),(8415,59474894,150),(8416,59476121,150),(8417,59477303,150),(8418,59478138,150),(8419,59476784,150),(8420,59479613,150),(8421,59478395,150),(8422,59477838,150),(8423,59477018,150),(8424,59480515,150),(8425,59478589,150),(8426,59476870,150),(8427,59478665,150),(8428,59475264,150),(8429,59476921,150),(8430,59476984,150),(8431,59477714,150),(8432,59480609,150),(8433,59476223,150),(8434,59478427,150),(8435,59478897,150),(8436,59478033,150),(8437,59479251,150),(8438,59478416,150),(8439,59476742,150),(8440,59476804,150),(8441,59477993,150),(8442,59476344,150),(8443,59476791,150),(8444,59478312,150),(8445,59476260,150),(8446,59476710,150),(8447,59477739,150),(8448,59478122,150),(8449,59476235,150),(8450,59478035,150),(8451,59478890,150),(8452,59476718,150),(8453,59477481,150),(8454,2790650,7),(8455,0,0),(8456,0,0),(8457,1595122,4),(8458,3039587,8),(8459,6392878,17),(8460,13011381,33),(8461,26162350,66),(8462,52380089,132),(8463,89946599,226),(8464,48591605,122),(8465,48591674,122),(8466,48592411,122),(8467,48592139,122),(8468,48591597,122),(8469,48593073,122),(8470,48594388,122),(8471,48593006,122),(8472,48592868,122),(8473,48591883,122),(8474,48591996,122),(8475,48592487,122),(8476,48593035,122),(8477,48592983,122),(8478,48592518,122),(8479,48593022,122),(8480,48591447,122),(8481,48591429,122),(8482,48590703,122),(8483,48590515,122),(8484,48592095,122),(8485,48591624,122),(8486,48593059,122),(8487,48591833,122),(8488,48591801,122),(8489,48591434,122),(8490,48591329,122),(8491,48591031,122),(8492,48592011,122),(8493,48591180,122),(8494,48590132,122),(8495,48589968,122),(8496,48592017,122),(8497,48591680,122),(8498,48592589,122),(8499,48591924,122),(8500,48592465,122),(8501,48590085,122),(8502,48592972,122),(8503,48591199,122),(8504,48592252,122),(8505,48591447,122),(8506,48593852,122),(8507,48591962,122),(8508,48591633,122),(8509,48592239,122),(8510,48593154,122),(8511,48591397,122),(8512,48591670,122),(8513,48591680,122),(8514,48592649,122),(8515,48591826,122),(8516,48591677,122),(8517,48592881,122),(8518,48591575,122),(8519,48593456,122),(8520,48592155,122),(8521,48592169,122),(8522,48590215,122),(8523,48592183,122),(8524,48592225,122),(8525,48592245,122),(8526,48591150,122),(8527,48592018,122),(8528,48588884,122),(8529,48590592,122),(8530,48591964,122),(8531,48592965,122),(8532,48592250,122),(8533,48592060,122),(8534,48593035,122),(8535,48592424,122),(8536,48590803,122),(8537,48593551,122),(8538,48592395,122),(8539,48593200,122),(8540,48591921,122),(8541,48591238,122),(8542,48590829,122),(8543,48590910,122),(8544,48592707,122),(8545,48592697,122),(8546,48591469,122),(8547,48589663,122),(8548,48592787,122),(8549,48591407,122),(8550,48591312,122),(8551,48592021,122),(8552,48592632,122),(8553,48592776,122),(8554,48591189,122),(8555,48591017,122),(8556,48592305,122),(8557,48592179,122),(8558,48591357,122),(8559,48592585,122),(8560,48592517,122),(8561,48593538,122),(8562,45258808,114),(8563,0,0),(8564,0,0),(8565,1575057,4),(8566,3549161,9),(8567,7407422,19),(8568,15035599,38),(8569,30200835,76),(8570,60441558,152),(8571,103784229,261),(8572,48590869,122),(8573,48589829,122),(8574,48591063,122),(8575,48591752,122),(8576,48590132,122),(8577,48592537,122),(8578,48592069,122),(8579,48589521,122),(8580,48590078,122),(8581,48590454,122),(8582,48590035,122),(8583,48592362,122),(8584,48590181,122),(8585,48590933,122),(8586,48591997,122),(8587,48591981,122),(8588,48591666,122),(8589,48592235,122),(8590,48591000,122),(8591,48591077,122),(8592,48593002,122),(8593,48593526,122),(8594,48589687,122),(8595,48591630,122),(8596,48592325,122),(8597,48591422,122),(8598,48591354,122),(8599,48591567,122),(8600,48591119,122),(8601,48592133,122),(8602,48592996,122),(8603,48590758,122),(8604,48591599,122),(8605,48590003,122),(8606,48593221,122),(8607,48591367,122),(8608,48592278,122),(8609,48592342,122),(8610,48591490,122),(8611,48592124,122),(8612,48592473,122),(8613,48589997,122),(8614,48590563,122),(8615,48591092,122),(8616,48592993,122),(8617,48592065,122),(8618,48590646,122),(8619,48590646,122),(8620,48591565,122),(8621,48591539,122),(8622,48591115,122),(8623,48590761,122),(8624,48589515,122),(8625,48590618,122),(8626,48592885,122),(8627,48590994,122),(8628,48591863,122),(8629,48591208,122),(8630,48590621,122),(8631,48591683,122),(8632,48589589,122),(8633,48591138,122),(8634,48590785,122),(8635,48593065,122),(8636,48593174,122),(8637,48590348,122),(8638,48592798,122),(8639,48591764,122),(8640,48590870,122),(8641,48592303,122),(8642,48591485,122),(8643,48592434,122),(8644,48591610,122),(8645,48590385,122),(8646,48592395,122),(8647,48589988,122),(8648,48590004,122),(8649,48591563,122),(8650,48592604,122),(8651,48594280,122),(8652,48593301,122),(8653,48593221,122),(8654,48592259,122),(8655,48592922,122),(8656,48592247,122),(8657,48590475,122),(8658,48590604,122),(8659,48591092,122),(8660,48591717,122),(8661,48592122,122),(8662,48592832,122),(8663,48594864,122),(8664,48593372,122),(8665,48591210,122),(8666,48592600,122),(8667,48595018,122),(8668,48592093,122),(8669,48592443,122),(8670,48594879,122),(8671,48592961,122),(8672,48593804,122),(8673,48590873,122),(8674,48592415,122),(8675,48591732,122),(8676,48590949,122),(8677,48591603,122),(8678,48591541,122),(8679,48592804,122),(8680,48590720,122),(8681,48592173,122),(8682,48591509,122),(8683,48592422,122),(8684,48588716,122),(8685,33503521,85),(8686,0,0),(8687,0,0),(8688,1594524,4),(8689,6776765,17),(8690,3040726,8),(8691,13012217,33),(8692,26164076,66),(8693,52382019,132),(8694,91578421,230),(8695,48586501,122),(8696,48586285,122),(8697,48588338,122),(8698,48586762,122),(8699,48585518,122),(8700,48588471,122),(8701,48588458,122),(8702,48588213,122),(8703,48588172,122),(8704,48586644,122),(8705,48589890,122),(8706,48588938,122),(8707,48590566,122),(8708,48586721,122),(8709,48587233,122),(8710,48587373,122),(8711,48585134,122),(8712,48587894,122),(8713,48585189,122),(8714,48587394,122),(8715,48587124,122),(8716,48587016,122),(8717,48587061,122),(8718,48590637,122),(8719,48587175,122),(8720,48587674,122),(8721,48588528,122),(8722,48586777,122),(8723,48587989,122),(8724,48586494,122),(8725,48587104,122),(8726,48589026,122),(8727,48588896,122),(8728,48588755,122),(8729,48589005,122),(8730,48588394,122),(8731,48588995,122),(8732,48590081,122),(8733,48588246,122),(8734,48589193,122),(8735,48588717,122),(8736,48587200,122),(8737,48588166,122),(8738,48587955,122),(8739,48589558,122),(8740,48584644,122),(8741,48588546,122),(8742,48587878,122),(8743,48589082,122),(8744,48586602,122),(8745,48587890,122),(8746,48586499,122),(8747,48586616,122),(8748,48586261,122),(8749,48587390,122),(8750,48588022,122),(8751,48586832,122),(8752,48587324,122),(8753,48588531,122),(8754,48587783,122),(8755,48588269,122),(8756,48588452,122),(8757,48587671,122),(8758,48584735,122),(8759,48585410,122),(8760,48586696,122),(8761,48584605,122),(8762,48584202,122),(8763,48587048,122),(8764,48588130,122),(8765,48584844,122),(8766,48584679,122),(8767,48586482,122),(8768,48587283,122),(8769,48585196,122),(8770,48586151,122),(8771,48586093,122),(8772,48586614,122),(8773,48587158,122),(8774,48587921,122),(8775,48586034,122),(8776,48587567,122),(8777,48586233,122),(8778,48585316,122),(8779,48587969,122),(8780,48587604,122),(8781,48587235,122),(8782,48584477,122),(8783,48585975,122),(8784,48588278,122),(8785,48588449,122),(8786,48586655,122),(8787,48585007,122),(8788,48586976,122),(8789,48586884,122),(8790,48587295,122),(8791,48586534,122),(8792,48587015,122),(8793,48575780,122),(8794,29347454,74),(8795,0,0),(8796,0,0),(8797,1993339,5),(8798,3986680,10),(8799,8042585,21),(8800,16299246,41),(8801,32723175,83),(8802,65481544,165),(8803,112436349,282),(8804,48588683,122),(8805,48589077,122),(8806,48585151,122),(8807,48586642,122),(8808,48585609,122),(8809,48587988,122),(8810,48585989,122),(8811,48587750,122),(8812,48589084,122),(8813,48589440,122),(8814,48589149,122),(8815,48586947,122),(8816,48586871,122),(8817,48588088,122),(8818,48588508,122),(8819,48588147,122),(8820,48586128,122),(8821,48588792,122),(8822,48587499,122),(8823,48587441,122),(8824,48589610,122),(8825,48588982,122),(8826,20331821,51) ON DUPLICATE KEY UPDATE partsize = VALUES(partsize) + partsize, currentparts = VALUES(currentparts) + currentparts) [2021-11-16 06:38:54] NNTmux.WARNING: 50000 articles failed to insert! [2021-11-16 06:39:52] NNTmux.ERROR: SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction (SQL: INSERT INTO binaries (binaryhash, name, collections_id, totalparts, currentparts, filenumber, partsize) VALUES (UNHEX('84a1ae357c3cf8440fdd01d55d633a54'), 'Guardians of the Galaxy Vol. I -1080p-BluRay.mkv - [24/59] - \"Guardians of the Galaxy Vol. I -1080p-BluRay.mkv.rar.av.split\" yEnc', 2124, 274, 1, 24, 27876) ON DUPLICATE KEY UPDATE currentparts = currentparts + 1, partsize = partsize + 27876)

And now seeing: [2021-11-16 06:45:25] NNTmux.ERROR: SQLSTATE[23000]: Integrity constraint violation: 1452 Cannot add or update a child row: a foreign key constraint fails (nntmux.binaries, CONSTRAINT FK_Collections FOREIGN KEY (collections_id) REFERENCES collections (id) ON DELETE CASCADE ON UPDATE CASCADE) (SQL: INSERT INTO binaries (binaryhash, name, collections_id, totalparts, currentparts, filenumber, partsize) VALUES (UNHEX('88712abd62e5430d99b914dc49e15b74'), '(18/23) - \" Balance 778089009090004.x09\" - 209.54 MB - yEnc', 2184, 10, 1, 18, 1671295) ON DUPLICATE KEY UPDATE currentparts = currentparts + 1, partsize = partsize + 1671295) [2021-11-16 06:45:25] NNTmux.ERROR: SQLSTATE[23000]: Integrity constraint violation: 1452 Cannot add or update a child row: a foreign key constraint fails (nntmux.binaries, CONSTRAINT FK_Collections FOREIGN KEY (collections_id) REFERENCES collections (id) ON DELETE CASCADE ON UPDATE CASCADE) (SQL: INSERT INTO binaries (binaryhash, name, collections_id, totalparts, currentparts, filenumber, partsize) VALUES (UNHEX('48de5e6cc19aa03e4bac24efe752baae'), '(17/23) - \" Balance 778089009090004.x08\" - 209.54 MB - yEnc', 2184, 10, 1, 17, 1671441) ON DUPLICATE KEY UPDATE currentparts = currentparts + 1, partsize = partsize + 1671441)

ghost commented 2 years ago

Master is newer than dev which is obsolete at this point. You need to run postprocessing on the releases before they show up.

The deadlock error is probably a MySQL config issue, the second one I have no clue about. I would start with a full reinstall.

blimeybloke commented 2 years ago

thanks, I been playing and went to dev, now about to go back. Do I need to run artisan or composer after I do the git update?

Here is the latest error when I ran php artisan fixtures:up:

`php artisan fixtures:up Populating all tables

Illuminate\Database\QueryException

SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1' for key 'PRIMARY' (SQL: insert into binaryblacklist (description, groupname, id, last_activity, msgcol, optype, regex, status) values (Blacklists non-english releases., alt.binaries.., 1, ?, 1, 1, (brazilian|chinese|croatian|danish|deutsch|dutch|estonian|flemish|finnish|french|german|greek|hebrew|icelandic|italian|latin|nordic|norwegian|polish|portuguese|japenese|japanese|russian|serbian|slovenian|spanish|spanisch|swedish|thai|turkish)[)]?( -)? -.|(anime.)|(fs|ws)|dsr|pal|ntsc|iso|complete|cracked|ebook|extended|dirfix|festival|proper|game|limited|read.?nfo|real|rerip|repack|remastered|retail|samplefix|scan|screener|theatrical|uncut|unrated|incl|winall)\b|doku|doc|dub|sub|(uncut)), 0), (Blacklists non-english abbreviated releases., alt.binaries.., 2, ?, 1, 1, -.[ -.]((19|20)\d\d|(480|720|1080)(i|p)|(bd|dvd.?|sat|vhs)?rip?|(bd|dl)mux|( -.)?(dub|sub)(ed|bed)?|complete|convert|(d|h|p|s)d?tv|dirfix|docu|dual|dvbs|dvdscr|eng|(h|x).?2?64|int(ernal)?|pal|proper|repack|xbox), 0), (Blacklists non-english abbreviated (reversed) releases., alt.binaries.., 3, ?, 1, 1, -. -.[ -.], 0), (Blacklists non-english subtitled releases., alt.binaries.., 4, ?, 1, 1, [ -.](chinese.subbed|dksubs|fansubs?|finsub|hebdub|hebsub|korsub|norsub|nordicsubs|nl( -.)?sub(ed|bed|s)?|nlvlaams|pldub|plsub|slosinh|swesub|truefrench|vost(fr)?)[ -.], 0), (Blacklists non-english (release group specific) releases., alt.binaries.., 5, ?, 1, 1, [ -.](4u.nl|nov[ a]+rip|realco|videomann|vost)[ -.], 0), (Blacklists non-english (lang specific) releases., alt.binaries.., 6, ?, 1, 1, -.[ -.], 0), (Blacklists porn releases., alt.binaries.., 7, ?, 1, 1, -.[ -.], 0), (Bad releases., alt.binaries.., 8, ?, 1, 1, hdnectar|nzbcave, 0), (Removes passworded releases., alt.binaries.., 9, ?, 1, 1, Passworded, 0), (Virus codec posters., alt.binaries.(boneless|movies.divx), 10, ?, 2, 1, ((Frkz|info)@XviD2?|x?VIDZ?@pwrpst|movies@movies?).net|(hsv.stoned@hotmail|unequal87@gmail|ilove@movies).com, 0), (Passworded/Encrypted junk., alt.binaries.tun, 11, ?, 1, 1, [PRiVATE]\s+[a-z0-9]+ , 0), (Blocks posts by koreantv., ^alt.binaries.teevee$, 12, ?, 1, 1, ^[KoreanTV] , 0), (Block 16 character hash floods in kenpsx frogs., ^alt.binaries.(kenpsx|frogs)$, 13, ?, 1, 1, ^\s([a-f0-9]{16})\s[\d+\/\d+]\s-\s\"\1\"\s+yEnc$, 0), (Posts by TESTMAN (jpegs), ^alt.binaries.multimedia.korean$, 14, ?, 2, 1, TESTMAN, 1), (torrent uploads ::: yEnc "SBS ÃñⰡ¿ä.E690.120916.HDTV.H264.720p-KOR.avi.torrent", ^alt.binaries.multimedia.korean$, 15, ?, 1, 1, ^yEnc ".+torrent"$, 1), (Incomplete releases, ^korea.binaries.movies$, 16, ?, 1, 1, ^.[?(Kornet|SK|xpeed|KT)]?, 1), (incomplete cryptic releases, ^korea.binaries.movies$, 17, ?, 2, 1, ^(top@top.t (top)|shit@xxxxxxxxaa.com (shit)|none@nonemail.com (none))$, 1), (Virus Poster, ^korea.binaries.movies$, 18, ?, 2, 1, ^filzilla6@web.de (Baruth)$, 1))`

blimeybloke commented 2 years ago

rebuilding so closing