After cutover migration cleanup

zhang4952 commented 6 years ago

[x] #234 ingest the Brief Series, Books and Pamphlets collection
[x] #233 re-ingest items failed with browse-everything -> not needed
[x] Correct visibility_during_embargo and visibility of Purchased E-resource
[x] Delete duplicate works with the same handle
[x] Clean Review for Submission list

revgum commented 6 years ago

For the "Clean Review Submission List" here's the code that we need to run for every work that isn't on a workflow

# 1. get the first user, which is the ScholarsArchive admin user 
work = ActiveFedora::Base.find(‘WORK_ID’)

# 2. Query migration user
user = User.first 

# 3. Add the item to its workflow (deposit)
Hyrax::Workflow::WorkflowFactory.create(work, {}, user)

# 4. Approve the item
subject = Hyrax::WorkflowActionInfo.new(work, u)
action = PowerConverter.convert_to_sipity_action('Approve', scope: subject.entity.workflow)
Hyrax::Workflow::WorkflowActionService.run(subject: subject, action: action, comment: 'Approve by migration user.')

revgum commented 6 years ago

Here are all of the work id's, in yaml format

---
- '05741v03s'
- '44558g457'
- 's7526f64b'
- 'tb09j777k'
- '1831cn70j'
- 'qr46r3589'
- 'pg15bh539'
- 'ht24wn24x'
- '41687m082'
- '1v53k0866'
- 't722hc59b'
- 'dz010s57c'
- 'bz60d002h'
- 'm900nw74p'
- 'r207ts517'
- 'hx11xg005'
- 'cn69m7356'
- 'wh246v22x'
- 'rr172079v'
- 'g158bm21c'
- 'k643b350c'
- 'qz20sw37v'
- 'r494vp59n'
- 'qb98mj53w'
- '736665972'
- 'cr56n4698'
- 'bg257f85d'
- '3f462916b'
- 'gt54kq684'
- '08612s08m'
- 'tq57nv27m'
- '3484zm06z'
- 'td96k563n'
- 'r781wj98c'
- 's1784p72r'
- '3t945v033'
- 'dr26z1442'
- 'xk81jp64h'
- 'hh63t047m'
- '1g05ff75p'
- 'fj2365385'
- '0r967688v'
- '0z709083z'
- '0z7090384'
- 'tm70mz672'
- 'k643b433x'
- 'ng451n013'
- 'j3860968x'
- 'x346d746h'
- 'dn39x499d'
- '02871037r'
- '5x21tj829'
- 'cv43p108s'
- '1g05ff96f'
- 't435gh492'
- 'kw52jc77t'
- '6w924g098'
- 'm039k824b'
- '6d5700995'
- 'v405sd73s'
- 'mw22v874n'
- 'xd07gx38n'
- 'n870zv46j'
- 'hh63t0516'
- 'n583xz30d'
- 'ks65hg73g'
- '9880vt62z'
- 'h415pc97d'
- 'js956k66d'
- '4j03d242n'
- 'vx021h24v'
- 'v405sc76d'
- 'j9602278f'
- 'qn59q6115'
- 'ht24wm799'
- 'xg94hr952'
- 'w0892d21j'
- 'w6634598f'
- '736667036'
- '44558g49b'
- 'v692t866c'
- 'vh53wz27h'
- 'ks65hf93r'
- 'gt54kq306'
- 'mk61rk42c'
- 'xs55mf60z'
- 'wm117r33p'
- 'g732dc325'
- 'c534fr150'
- '0g354h542'
- 'ng451k84f'
- 'nk322g892'
- 's7526g05c'
- '7m01bn97c'
- '8g84mp324'
- 'jq085n147'
- '7m01bn81g'
- 'mg74qp94w'
- 'qb98mj34p'
- 'ww72bg10j'
- '4t64gr59m'
- '4q77fv184'
- 'g158bn145'
- 't148fk978'
- '0c483n52g'
- 'ws859j55f'
- 'jq085p56z'
- '0r9677394'
- 'z890rz64k'
- 'cn69m841j'
- '47429f22k'
- 'd791sk41p'
- 'qf85nf70s'
- 'wp988n955'
- '7s75dg847'
- '1n79h7566'
- 'zc77st60s'
- '8s45qd24n'
- 'kp78gm38v'
- 'g445cj42k'
- 'js956m29g'
- 'nv9355866'
- 'p8418r69n'
- 'rn3014834'
- 'dv13zx51j'
- 'nv935719r'
- 'pc289p195'
- '3r074z11k'
- 'j96024904'
- 'hq37vs55w'
- '9s161b24n'
- '0g354k48b'
- 'kp78gm37k'
- 'np193f44n'
- 'kh04ds816'
- 'js956k697'
- 'cr56n462b'
- 'cz30pz31v'
- 'd504rq44h'
- 'gx41mp02p'
- 'n583xz90v'
- 'ks65hg858'
- '7p88cm668'
- '2f75rd00n'
- 'cn69m8433'
- '8336h616b'
- 'w66347862'
- '6682x743m'
- '4j03d308j'
- 's4655k852'
- 'k930c174d'
- '0c483p041'
- 'g445ch367'
- 'th83m329s'
- 'tb09j966g'
- '3x816r58x'
- 'q811kp865'
- 'pc289p20x'
- 'dz010t17m'
- '5425kf91d'
- '9p290f480'
- '6d5700596'
- '0v838477s'
- 'w9505447m'
- 'cn69m842t'
- 'pz50h0958'
- 'xd07gx142'
- 'qv33s036h'
- 'pz50h096j'
- '00000308t'
- 'zk51vm64c'
- '9w032732k'
- '9880vv85g'
- 'dv13zx444'
- '5h73q030p'
- '3f462854j'
- 'gf06g696p'
- 'mk61rm89g'
- 'hh63t124h'
- '05741w960'
- 'qv33s173v'
- 'v118rh47x'
- 'bk128g137'
- 'n583xz893'
- 'r781wk95r'
- '2r36v3069'
- '7p88cm64q'
- 'br86b682z'
- 'r781wk15t'
- '7w62fd493'
- 'tb09j873p'
- '7p88cm650'
- '0c483n630'
- 'tm70n049h'
- 'j9602491d'
- 'h989r7305'
- 'kd17cx840'
- 'cz30pz33d'
- 'w0892f107'
- '12579v62c'
- 'j9602396k'
- 'qv33s023f'
- 'dj52w7759'
- 'z603r106f'
- 'ng451m246'
- 'sx61dp93m'
- 'gh93h271t'
- '7m01bp38d'
- 'ww72bf77q'
- '0r967647m'
- 'ks65hg36b'
- 'cz30pw63q'
- 'fx719q34w'
- 'c821gn35q'
- '7d278w59p'
- 'tm70mx84h'
- '76537400m'
- '0k225f28w'
- '9g54xm44s'
- 'b8515r131'
- 'wh246v61m'
- '6t053h92w'
- 'hd76s1975'
- 'qz20sv34t'
- 'cz30pv82q'
- 't148fk048'
- '00000179c'
- '3r074w913'
- 'fq977w72m'
- 'fq977w809'
- 'ft848t34n'
- 'dr26z083j'
- 'jw827f66f'
- 'b8515r62z'
- '7m01bp32r'
- 'gb19f8461'
- 'hm50tv88q'
- 'bn999977v'
- 'js956k433'
- 'sf2686965'
- 'r781wh828'
- '1z40kv926'
- '1n79h643x'
- 'x633f244s'
- '7d278v810'
- '5138jg476'
- '3t945s04z'
- 'qn59q5462'
- '5138jg45n'
- 'k643b541t'
- 'gx41mn99k'
- '5d86p416t'
- 'ft848v176'
- '76537525p'
- 'x059cc456'
- 'j67317584'
- 'b5644t45v'
- 'v692t8076'
- 'z029p672b'
- 'vd66w1498'
- '6682x584f'
- '6108vg334'
- 'cn69m839h'
- 'zs25xd65d'
- 'qz20sx54q'
- '6108vg32v'
- '1831cp91h'
- 'cz30pz28j'
- '7d278x99v'
- 'g732df166'
- '4j03d367q'
- 'b5644w42f'
- 'cc08hk62b'
- 'v979v6884'
- '3j333661d'
- '4t64gs37s'
- 'm039k879x'
- 'bv73c4415'
- 'dn39x5751'
- '4x51hp00c'
- 'd217qt804'
- 'n009w619b'
- 'rv042z030'
- 'tq57nw194'
- '44558j68z'
- 'pr76f741p'
- 'c247dx308'
- '5t34sp59g'
- 'bv73c4394'
- 'kp78gm351'
- '0g354f54n'
- 'x633f4004'
- 'rf55zb21h'
- '9k41zj19w'
- 'wm117s54n'
- '4x51hn447'
- 'vm40xv26g'
- '0z7090961'
- '0v838325t'
- '5x21th84n'
- 'jq085p34x'
- '0p096b28d'
- 'm039k804v'
- 'h989r614k'
- 'n296x1860'
- 'r494vp441'
- 'v405sf04k'
- 'r781wj973'
- '0c483n70d'
- 'xw42nb232'
- 'zs25xc036'
- '08612r562'
- 'q811kn460'
- 'c247dv74x'
- '6q182n909'
- 'j9602380p'
- '9593tz65r'
- 's7526f83j'
- '76537463w'
- 'zc77ss799'
- 'p2676z14k'
- 'n870zt65j'
- 'b8515r91d'
- 'qj72pb787'
- 'm613n106p'
- 'ks65hh358'
- 'j6731622j'
- 'bz60d0017'
- '5h73q070n'
- 'rv042x548'
- 'cn69m789h'
- 'br86b632r'
- 'v405sf23s'
- 'sf2688517'
- 'ww72bg06z'
- '9z9032292'
- 'sq87bw756'
- 'dv13zw677'
- 'dz010s45k'
- 'tt44pr152'
- '1z40kw96h'
- 'm900nx11m'
- '2v23vx15x'
- '5t34sp15d'
- '2b88qf816'
- '8910jx192'
- 'pc289m74t'
- 'mp48sh48r'
- 'js956j60j'
- '6m311s88z'
- 'cf95jg08r'
- '0p096944k'
- 'gq67jv97j'
- 'c821gn17s'
- 'xs55mg80n'
- '1n79h736q'
- 'st74cv233'
- 'm039k8321'
- '8s45qc677'
- '2n49t418f'
- '7s75df82g'
- 'bk128d410'
- 'j098zf21v'
- 'gb19f893d'
- 'v118rj075'
- '2227ms06w'
- '1g05fg35x'
- 'qv33s0713'
- '7s75df87v'
- 'j6731708x'
- 'k0698c313'
- '8049g8973'
- 'm039k8364'
- 'wd375z52d'
- 'q524js42v'
- 'j098zf56z'
- 'jh343x091'
- 'ww72bg05p'
- 'vt150n49p'
- 'c247dv870'
- 'br86b6349'
- 'h702q848q'
- 'cr56n472k'
- 'g158bm52c'
- '1z40kw362'
- 'm900nz17g'
- '8c97kt986'
- '0p096b21g'
- 'vq27zs18h'
- 'z603r190z'
- 'qb98mk03w'
- '9p290d75p'
- '000003513'
- '4j03d1750'
- 'gx41mm185'
- 'v692t9331'
- 'cv43p0435'
- '3197xq21t'
- '3n204180f'
- 'mw22v828j'
- '5712m9795'
- 's4655k71q'
- 'st74cs72m'
- 'mg74qp734'
- '7d278x21z'
- 'x346d821v'
- '8k71nm14n'
- 'xg94ht37k'
- 'rf55zb61g'
- '4m90f0376'
- 'fj236610p'
- 'qn59q719k'
- '3f462819z'
- 'm326m474k'
- 'gq67jt17d'
- 'np193c19w'
- '2b88qf396'
- 'm900nx32c'
- 'cc08hh814'
- '02870z15b'
- '6108vf22d'
- 'qf85nf755'
- 'cv43p0214'
- 'p5547t742'
- 'x920g033q'
- 'fn107226m'
- 'cj82k973w'
- '0k225f09p'
- 'w9505281h'
- '00000208m'
- 'kh04dt11q'
- 'pr76f556w'
- 'r781wj18f'
- 'kh04dr92h'
- '6t053j31c'
- '6w924f65d'
- '1r66j340b'
- '73666789s'
- '1831cn73c'
- 'ff365765m'
- '5425kd52h'
- '2b88qf417'
- 'hq37vq49b'
- 'dr26z074k'
- 'sn00b1265'
- 'sn00b106p'
- 'f1881p24b'
- 'kd17cw00r'
- 'gx41mm07n'
- 'g158bm31m'
- 'd504rp139'
- 't435gh66q'
- 'v405sf07d'
- 'rx913s548'
- '8623j185k'
- 'fx719q67f'
- '9w032624p'
- 'zc77ss33p'
- '5m60qv722'
- 'gq67jt45k'
- 'jw827f10k'
- 'zk51vk50t'
- 'd791sk038'
- '8k71nk96x'
- '4j03d273n'
- 's1784q028'
- 'ww72bd945'
- 'dz010s406'
- 'k930c062d'
- 'd217qs40z'
- 'd504rn710'
- '8p58pg45z'
- '02870z142'
- '9z9032819'
- 'x346d8003'
- 'ww72bf908'
- 'z316q488p'
- 'xd07gw36w'
- 'cr56n3446'
- 'z316q4023'
- 'r781wk28w'
- 'zs25xb58k'
- 'fj236445c'
- 'sj139421j'
- 'j3860917f'
- '2j62s7217'
- 'x346d706j'
- 'v405sc96w'
- '05741v69w'
- '1544br467'
- '6m311s40s'
- 'tt44pp70q'
- 'pk02cd10r'
- 'rr1720850'
- '9z903212w'
- '8w32r8313'
- 'zc77ss37s'
- 'gb19f884f'
- '3484zk68r'
- 'rf55z9732'
- 'nz8063088'
- 'z029p763k'
- '1v53k0114'
- '2f75rb72g'
- '8910jw67h'
- '8336h428q'
- 'ws859h824'
- 'bz60cz05z'
- 'hq37vq51c'
- '9c67wr47k'
- '3484zk712'
- 'c821gm771'
- '8336h3722'
- '1z40kw850'
- 'bn999892r'
- '3197xp919'
- 'qr46r272x'
- '4t64gq185'
- 'rx913r852'
- '1j92g923x'
- '2r36v064k'
- '4b29b751x'
- '5999n4797'
- 'vm40xs33g'
- '2j62s645m'
- '0k225g11x'
- 'ms35td544'
- 'b5644s13c'
- 'fx719m89v'
- '4x51hj529'
- '6m311t16x'
- 'th83m3145'
- '8s45q9640'
- 'js956h881'
- 'gm80hx41z'
- 'c534fq90b'
- '3j333435c'
- 'pn89d8721'
- 'v118rg52k'
- 'bv73c248p'
- 'qb98mh379'
- '1n79h651m'
- 'kw52j926x'
- 'gm80hx028'
- 'd791sg69r'
- '12579s86j'
- 'pr76f464c'
- '9p290c037'
- '8k71nh71n'
- 'np193b60d'
- 's1784m25z'
- 'dj52w627f'
- 'rb68xd18r'
- '4b29b742z'
- '2r36v0176'
- 'f4752h23w'
- 'pc289j65f'
- '2z10wr41c'
- '9c67wp36n'
- 'db78td65c'
- '2r36tx925'
- '3t945s146'
- 'jh343t11f'
- 'z029p620k'
- '5h73px54p'
- 'v979v3592'
- 'xs55mc89h'
- 'v405sc02m'
- 'gb19f627d'
- '707958363'
- 'h415pb09k'
- '6q182m62x'
- 'sn009z993'
- 'xs55md50h'
- 'vt150k65n'
- '0k225c57x'
- 'gx41mk439'
- 'f7623d209'
- 'r494vm16f'
- 'k643b180p'
- 'g445cd805'
- 'm326m235g'
- '5h73pw76h'
- '6t053g840'
- 'z029p619t'
- 'z029p6313'
- 'jq085m61d'
- 'vq27zp07c'
- 'd217qq16g'
- 'gt54kn77p'
- 'dr26xz992'
- '1831cm39v'
- 'vd66w1633'
- '3x816n083'
- 'zk51vh64r'
- 'xk81jk833'
- 'd504rq36t'
- 'jq085q212'
- '5425kf78v'
- '6d570139x'
- '1c18dg30p'
- 'z316q324x'
- 'f7623d07r'
- 'rr172227h'
- '5425kb33w'
- 'r781wg555'
- 'kw52j862k'
- 'm039k627r'
- 'w0892b55z'
- '7p88cm51n'
- 'w9505105g'
- '1z40kt37x'
- '4b29b660p'
- 'n009w279z'
- 'tm70mv83t'
- 'mk61rh551'
- 'tq57nr580'
- 'gf06g339f'
- 'fq977w41m'
- 'z890rv82w'
- 'vh53wx414'
- '5h73q1030'
- '4q77fs01j'
- 'k3569593q'
- 'bn999827f'
- 'k643b174j'
- '1g05fc33r'
- 'ms35t899n'
- 'vd66w4305'
- 'td96k6512'
- 'pz50h079w'
- 'xk81jk91s'
- '9p290987v'
- '5q47rp33g'
- '41687j82x'
- 'rr171z99z'
- '8p58pj47x'
- 'vt150k71s'
- '47429b71w'
- 'z029p525r'
- 'zk51vh39d'
- 'k0698808p'
- 'pz50gx58j'
- 'd504rm854'
- 'b5644s964'
- '2z10wr845'
- 'pc289j79s'
- 'qn59q541p'
- '76537279c'
- '4q77fs97c'
- '5x21tg90k'
- 'bz60cz14x'
- 'rn301286j'
- '1831cm40m'
- 'nk322f67t'
- '7s75dd012'
- '9w032441x'
- 'z890rv80b'
- 'dv13zv760'
- 'm039k624x'
- 'xd07gv49r'
- 'bv73c0933'
- '5h73pw72d'
- '02870w35d'
- 'k643b181z'
- 'w0892c708'
- 'st74cs04g'
- '5m60qt39t'
- 's7526d88q'
- '765372740'
- 'ff3656776'
- 'n296x0694'
- '4t64gp81q'
- 'b5644x27j'
- 'h989r718w'
- 'rn3015491'
- '0p096b79w'
- 'tm70n0401'
- '7h149t73z'
- 'pz50h087k'
- 'mk61rm797'
- '3484zm921'
- 'hq37vs45n'
- '2b88qh36s'
- 'sq87bz31j'
- 'mc87pv70g'
- 'zs25xd57q'
- 'm613n257c'
- 'zg64tq90r'
- 'kk91fq53f'
- '3x816r53j'
- 'vd66w440d'
- 'cj82kc43k'
- '8w32r9743'
- 'qb98mk43v'
- 'dv13zz211'
- 'rj4308477'
- '9p290f40s'
- 'g445cj36f'
- 'qr46r477q'
- '9019s6723'
- 'nz806373k'
- '6m311t28q'
- 's1784q72z'
- 'g158bn633'
- 'mk61rm76d'
- 'vm40xw69g'
- '8623j272q'
- 'vh53x084h'
- 'zp38wh591'
- 'd791sm34g'
- 'k930c225p'
- '00000396f'
- '9p290f42b'
- 'xw42nc815'
- '2514nq59c'
- 'cc08hk55x'
- 'mc87pv69q'
- 'm613n2563'
- 'n870zv989'
- '1g05fg75w'
- '1r66j5101'
- '12579x07q'
- '7w62fd41w'
- 'qj72pc67x'
- 't435gj26z'
- 'bz60d1340'
- 'f1881r02q'
- 'f1881r005'
- 'xp68km28c'
- '8g84mr008'
- 'jw827g88p'
- 'x633f474x'
- 'cf95jg66n'
- 'tt44pr54r'
- '4t64gs34z'
- 'qb98mk42k'
- 'fn107300p'
- 'wm117t08r'
- 'r781wk91n'
- '12579x06f'
- '2j62s900w'
- '7m01bq779'
- '00000397q'
- 'vx021k07m'
- '3j333657t'
- 'k35698409'
- 'wd376118p'
- 'v692tc20c'
- '3f4629683'
- 'm326m5948'
- '8049g945j'
- '0v8384693'
- 'c534ft119'
- 'z029p9096'
- 'bv73c4351'
- 'cv43p171j'
- '44558j629'
- 'p26770360'
- 'sx61dr18g'
- 'wp988p75w'
- 'p26770378'
- 'xp68km29n'
- '3j333656j'
- 'q524js91s'
- '7p88cm59v'
- 'pc289p11z'
- 'gt54kr84q'
- 'sf2688941'
- 'j3860b748'
- 'g158bn587'
- 'd217qt72f'
- '5x21tk27f'
- 'm326m586k'
- 'cv43p167z'
- '9s161b14d'
- '2f75rc80c'
- 'mw22v9690'
- 'st74cv53t'
- 'br86b769m'
- '2b88qf018'
- '7p88cj118'
- '44558f83f'
- 'xd07gv629'
- 'fb494993b'
- 'sj139348r'

revgum commented 6 years ago

For the "Delete Duplicates" here's a workflow;

Query all works from SOLR that have a replaces_ssim starting with 'http*', include the ID and the created date
Write Ruby code to "Group by replaces_ssim" for the list of works, and narrow down all groups that have more than 1 work (these are works that were ingested multiple times)
Iterate through the list, ordering by the created date so that we leave the most recent and delete the duplicates
- Determine if the work has children (members), and delete each of them before deleting the work

vantuyls commented 6 years ago

Consider: The most recently migrated duplicate item is probably the correct one, but one might run a quick check on duplicate items to see if they have the same number of metadata fields. if the items have the same number of metadata fields, then use the most recent. however, if the items have different number metadata fields, the most recent version of the item might not be the 'correct' one and this duplicate pair warrants additional review.

I suspect (?) that duplicates with different amounts of metadata will be fairly small in number and we may be able to review them manually.

revgum commented 6 years ago

completed through migration

osulp / dspace2hydra

After cutover migration cleanup #236