Testing strategies for switch to WoS REST API

[peetucket]

In order to validate the work described in #1609 and in #1612, proposal is:

• go to prod, look at the harvest schedule in cron...pick the day after a harvest was run
• look at the total number of contributions added in the last day and fetch the WoS IDs. The goal is to see what was added by the current code (SOAP API) harvest and then later compare to what was harvested by the new code in the same time period:

timeframe  =  Date.today - 1.day # look back 1 day
total_contributions_added = Contribution.where('created_at > ?',timeframe).where(status:'new').size;
wos_uids_added = Publication.where('created_at > ?',timeframe).pluck(:wos_uid).compact.size;

• run a full harvest with the new code on the qa server (cap-dev-a) and monitor HB and server logs for any errors
• wait for harvest to complete and then do the same thing:

timeframe  =  Date.today - 1.day # look back 1 day
total_contributions_added = Contribution.where('created_at > ?',timeframe).where(status:'new').size;
wos_uids_added = Publication.where('created_at > ?',timeframe).pluck(:wos_uid).compact.size;

• compare the total_contributions_added count and the wos_uids_added and see if the pass they sanity check .... i would expect them to be very similar, as the authors are very similar between prod and cap-dev-a and the harvest timeframes are very similar, only off by a day or so

Next steps:

• [x] deploy to UAT
• [x] inform #sul-cap-collab channel
• [x] percolate for a few weeks
• [x] merge this puppet change https://github.com/sul-dlss/puppet/pull/9661/files
• [x] set uat environment to use main puppet branch again (they were manually changed to use the puppet branch above)
• [x] merge if no errors
• [x] add back in uat as a non-standard deploy environment to sdr-deploy so dependency updates go to UAT again, i.e. revert https://github.com/sul-dlss/sdr-deploy/pull/220
• [x] Go directly to #1619 and complete tasks there

[mjgiarlo]

• [x] go to prod, look at the harvest schedule in cron...pick the day after a harvest was run
  • A harvest was run on prod on August 5th, so I chose August 6th as the day after a harvest was run
• [x] look at the total number of contributions added in the last day and fetch the WoS IDs. The goal is to see what was added by the current code (SOAP API) harvest and then later compare to what was harvested by the new code in the same time period:

  3.2.0 :001 > Date.today
   => Mon, 07 Aug 2023 
  3.2.0 :002 > timeframe = Date.today - 3.days
   => Fri, 04 Aug 2023 
  3.2.0 :003 > total_contributions_added = Contribution.where('created_at > ?',timeframe).where(status:'new').size
   => 598 
  3.2.0 :004 > wos_uids_added = Publication.where('created_at > ?',timeframe).pluck(:wos_uid).compact
   => ["WOS:001022267100001", "WOS:000945948100012", "WOS:000540822800008", "WOS:000446312300024", "MEDLINE:37535407", "WOS:000879891500008", "WOS:000429198000010", "WOS:000314075500003", "WOS:000476715500071", "WOS:000356668400004", "WOS:000712127900004", "WOS:000227980401497", "WOS:000086246705130", "WOS:000086246704254", "WOS:000086246704253", "WOS:000086246700915", "WOS:000433203504187", "WOS:000416073000015", "WOS:000410536700016", "WOS:000409080200017", "WOS:000409080200016", "WOS:000402473100003", "WOS:000398221300015", "WOS:000393104400015", "WOS:000393103900016", "WOS:000393103200015", "WOS:000393098300016", "WOS:000222010600045", "WOS:000222010600055", "WOS:000303399800042", "WOS:000299950300048", "WOS:000184606602530", "WOS:000184606701573", "WOS:000184606701567", "WOS:000184606701561", "WOS:000184606701452", "WOS:000184606600596", "WOS:000184606600557", "WOS:000184940100049", "WOS:A1995QM91500563", "WOS:000168392103442", "WOS:000079269201706", "WOS:000079269200160", "WOS:000079269201707", "WOS:A1997WN21501705", "WOS:A1995QM91504803", "WOS:A1994MZ58502175", "WOS:A1993KT89300946", "WOS:A1992HK13500569", "WOS:A1991FC76200947", "MEDLINE:37531248", "MEDLINE:37542155", "WOS:000596231904063", "MEDLINE:37537332", "WOS:001008232900576", "WOS:001008232900449", "WOS:001008232900500", "WOS:001008232900479", "WOS:001008232900528", "WOS:001008232900583", "WOS:001008232900380", "WOS:001008232900593", "WOS:001008232900587", "WOS:001008232900591", "WOS:001008232900604", "WOS:001008232900354", "WOS:001008232900534", "WOS:001008232900520", "WOS:001008232900485", "WOS:001008232900573", "WOS:001008232900571", "WOS:001008232900331", "MEDLINE:37535372", "MEDLINE:37536515", "MEDLINE:37534743", "MEDLINE:37527836", "MEDLINE:37534466", "WOS:001026213000005", "WOS:001026245200004", "MEDLINE:37529840", "MEDLINE:37532139", "WOS:000505300400003", "WOS:000463378100055", "WOS:000455493000012", "WOS:000455493000010", "WOS:000404906900061", "MEDLINE:37531843", "MEDLINE:37483562", "MEDLINE:37345823", "WOS:001002154400001", "WOS:000986067400002", "WOS:001000671400007", "MEDLINE:37530745", "WOS:001010551400002", "WOS:000802249500001", "WOS:000427460000001", "WOS:000898874100001", "WOS:000967230800001", "MEDLINE:37529924", "WOS:001010953400003", "WOS:001008232900323", "WOS:001008232900581", "WOS:001008232900311", "WOS:001008232900572", "WOS:001008232900698", "MEDLINE:37531157", "WOS:001008232900318", "MEDLINE:37529368", "WOS:001008232900057", "MEDLINE:37532724", "WOS:001023480400001", "MEDLINE:37530764", "WOS:001008232900019", "WOS:001008232900315", "MEDLINE:37530677", "WOS:001032224900001", "WOS:001031418200001", "MEDLINE:37529278", "MEDLINE:37530426", "MEDLINE:37528747", "WOS:001030464100001", "MEDLINE:37528731", "MEDLINE:37528739", "WOS:001035378600001", "WOS:001034961500001", "MEDLINE:37528533", "WOS:001030766500001", "MEDLINE:37530549", "WOS:001032097000001", "WOS:001009232500489", "MEDLINE:37535053", "WOS:001035190100001", "WOS:000999229700012", "WOS:000949190900003", "MEDLINE:37523803", "MEDLINE:37529853", "WOS:001029274700002", "WOS:001009232500334", "WOS:001024877000003", "MEDLINE:37537428", "MEDLINE:37011205", "MEDLINE:37528205", "WOS:001011463700019", "WOS:001033418500001", "WOS:001032594900001", "WOS:001004513900010", "WOS:001026855700001", "WOS:001026156500001", "WOS:001033724800001", "WOS:001033152400001", "WOS:001009158400001", "WOS:001008232900662", "MEDLINE:37529401", "WOS:001032341400001", "MEDLINE:37532936", "WOS:001025419600003", "WOS:001032204100001", "WOS:001032730500001", "MEDLINE:37096274", "WOS:001032688200001", "MEDLINE:37530784", "WOS:001011420500081", "WOS:001024868400009", "WOS:001024611400006", "WOS:001023245500012", "WOS:001033754800001", "WOS:001032179400001", "MEDLINE:37529406", "WOS:001030716800001", "MEDLINE:37530399", "MEDLINE:37527375", "WOS:000999174800014", "MEDLINE:37503014", "MEDLINE:36993432", "WOS:001030455900003", "WOS:001034891600001", "WOS:001021499500005", "MEDLINE:37440404", "MEDLINE:37530747", "WOS:001008232900148", "WOS:001025646000003", "WOS:001024195800004", "WOS:001029786000001", "MEDLINE:37510724", "MEDLINE:37528757", "WOS:001033356600001", "MEDLINE:37445385", "WOS:001017754700116", "WOS:000968517000001", "MEDLINE:37509647", "WOS:001032699800001", "WOS:000984168400016", "WOS:001008232900518", "WOS:001035299600001", "WOS:000931744100001", "MEDLINE:37531203", "WOS:001031031700001", "MEDLINE:37510920", "WOS:001034561200001", "WOS:001012129000006", "WOS:001004940700001", "WOS:000938252000002", "WOS:001030444500001", "WOS:001029109100001", "MEDLINE:37526085", "WOS:000636180100002", "WOS:000406901300003", "WOS:000932270300001", "WOS:000766218500001", "WOS:000425450000015", "WOS:000730652100001", "WOS:000630190600001", "WOS:000984474000019", "WOS:000696518100024", "WOS:000479055200017", "WOS:000459524100016", "WOS:000435576500022", "WOS:001009447100006", "WOS:000432474200008", "WOS:000784052000001", "WOS:000691924600002", "WOS:001035713900001", "WOS:001022730600001", "WOS:001031635100006", "WOS:000973641800002", "WOS:000949830600001", "WOS:000916492700001", "WOS:000774687000001", "WOS:000823968400006", "WOS:000670551100001", "WOS:000661391100001", "WOS:000650156000002", "WOS:000651022300001", "WOS:000661412500001", "WOS:000610615200001", "WOS:000580713500004", "WOS:000538222500001", "WOS:000553109000002", "WOS:000515028200013", "WOS:000511195600012", "WOS:000504091900004", "WOS:000468310500013", "WOS:000464654200014", "WOS:000424778600018", "WOS:000369303700010", "WOS:000362061400001", "WOS:000358417800004", "WOS:000352809600006", "WOS:000347515300091", "WOS:000341228200016", "WOS:000341239200015", "WOS:000339774900001", "WOS:000281882800014", "WOS:000261683700010", "WOS:000260662700014", "WOS:000256643900036", "WOS:000253176000029", "WOS:000251746100001", "WOS:000265314600003", "WOS:000355214100005", "WOS:000450359000009", "WOS:000599347900001", "WOS:000892865200001"]
  3.2.0 :005 > wos_uids_added.size
   => 263

• [x] run a full harvest with the new code on the qa server (cap-dev-a) and monitor HB and server logs for any errors

  RAILS_ENV=production bundle exec rake "harvest:all_authors_update"

  • Completed 8:58pm Pacific, August 7th
• [x] wait for harvest to complete and then do the same thing:

  3.2.0 :001 > timeframe = Date.today - 2.days
   => Sun, 06 Aug 2023 
  3.2.0 :002 > total_contributions_added = Contribution.where('created_at > ?',timeframe).where(status:'new').size
   => 1819 
  3.2.0 :003 > wos_uids_added = Publication.where('created_at > ?',timeframe).pluck(:wos_uid).compact
   => 
  ["MEDLINE:37540559", "WOS:001010953400003", "WOS:001008232900604", "WOS:001008232900323", "WOS:001008232900581", "MEDLINE:37536886", "WOS:001008232900311", "MEDLINE:37542155", "MEDLINE:37541186", "WOS:001008232900380", "WOS:001008232900572", "WOS:001008232900698", "MEDLINE:37537179", "WOS:001008232900318", "WOS:001008232900331", "WOS:001029109100001", "MEDLINE:37537332", "WOS:001008232900057", "MEDLINE:37535655", "MEDLINE:37540816", "MEDLINE:37537242", "WOS:001008232900576", "WOS:001008232900449", "WOS:001008232900500", "WOS:001008232900479", "WOS:001008232900528", "WOS:001008232900583", "WOS:001008232900593", "WOS:001008232900587", "WOS:001008232900591", "WOS:001008232900354", "WOS:001008232900534", "WOS:001008232900520", "WOS:001008232900485", "WOS:001008232900573", "WOS:001008232900571", "WOS:001023480400001", "MEDLINE:37539354", "WOS:001031242800001", "MEDLINE:37536515", "WOS:001008232900019", "WOS:001008232900315", "WOS:001032224900001", "WOS:001032143100001", "WOS:001033081300001", "WOS:001031418200001", "MEDLINE:37539093", "MEDLINE:37541197", "WOS:001026250700002", "MEDLINE:37537304", "WOS:001019624300001", "WOS:001029211300001", "MEDLINE:37541210", "MEDLINE:37535885", "MEDLINE:37537483", "WOS:001035378600001", "WOS:001030849100001", "WOS:001021468700001", "MEDLINE:37537500", "MEDLINE:37541803", "WOS:001034961500001", "MEDLINE:37541280", "WOS:001025035500001", "MEDLINE:37542372", "WOS:001009232500489", "WOS:001028820600001", "WOS:000999229700012", "WOS:001035190100001", "WOS:000949190900003", "MEDLINE:37537502", "MEDLINE:37523803", "MEDLINE:37536287", "WOS:001028782900001", "MEDLINE:37537499", "WOS:001029274700002", "MEDLINE:37538833", "MEDLINE:37529840", "WOS:001009232500334", "WOS:001024877000003", "MEDLINE:37537428", "WOS:001027400700001", "MEDLINE:37011205", "MEDLINE:37540534", "MEDLINE:37528205", "WOS:001011463700019", "WOS:001033418500001", "WOS:001032594900001", "WOS:001004513900010", "WOS:000958181300006", "WOS:001026156500001", "WOS:001033724800001", "MEDLINE:37540856", "WOS:001033152400001", "WOS:001009158400001", "WOS:001034134300001", "WOS:001010551400002", "WOS:001008232900662", "WOS:001030813800001", "WOS:001031559800001", "MEDLINE:37536879", "WOS:001035713900001", "WOS:001032341400001", "MEDLINE:37536253", "MEDLINE:37542111", "MEDLINE:37537254", "WOS:001032204100001", "WOS:001032730500001", "MEDLINE:37096274", "MEDLINE:37541715", "WOS:001032873900001", "MEDLINE:37541189", "WOS:001026056600016", "WOS:001032688200001", "WOS:001030983400001", "WOS:001010423100017", "WOS:001010423100013", "MEDLINE:37530784", "MEDLINE:37537264", "WOS:001011420500081", "WOS:001024868400009", "WOS:001024611400006", "MEDLINE:37537482", "WOS:001030852600001", "MEDLINE:37540524", "WOS:001033754800001", "WOS:001032179400001", "MEDLINE:37541937", "MEDLINE:37530399", "MEDLINE:37527375", "WOS:001031305000001", "WOS:001024444200005", "WOS:001030455900003", "WOS:001034891600001", "WOS:001027742600002", "MEDLINE:37530747", "WOS:001008232900148", "MEDLINE:37459329", "WOS:001023713600002", "MEDLINE:37460386", "MEDLINE:37536814", "MEDLINE:37460060", "WOS:001028302300001", "WOS:001029398900001", "MEDLINE:37488420", "MEDLINE:37500730", "WOS:001021462100037", "WOS:001022325400001", "WOS:001014670400001", "MEDLINE:37494935", "MEDLINE:37535693", "WOS:001025493900001", "WOS:001025524400001", "WOS:001025114100001", "WOS:001024195800004", "WOS:000956844800013", "WOS:001024897600001", "WOS:001025681600001", "WOS:001022906800001", "WOS:001013228900002", "WOS:001022423300001", "WOS:001025473200015", "MEDLINE:37463421", "WOS:001016812300001", "MEDLINE:37478451", "MEDLINE:37520763", "MEDLINE:37454638", "MEDLINE:37481236", "WOS:001013947900001", "WOS:001010870100001", "WOS:000989629700489", "WOS:001018090600001", "MEDLINE:37463507", "WOS:001012311500013", "WOS:001024864600001", "WOS:001029786000001", "WOS:001023252000001", "MEDLINE:37536272", "WOS:001022730600001", "WOS:001008477300001", "MEDLINE:37459967", "WOS:001016106700001", "MEDLINE:37540551", "WOS:001034363200001", "MEDLINE:37451259", "WOS:001026035200001", "MEDLINE:37458551", "MEDLINE:37528757", "MEDLINE:37455813", "WOS:000995807700002", "WOS:001027764700001", "WOS:001033356600001", "WOS:001025630000005", "WOS:001025103400001", "WOS:001026321700001", "MEDLINE:37531330", "MEDLINE:37524088", "WOS:001021298800001", "MEDLINE:37478529", "WOS:001030572400011", "MEDLINE:37528149", "MEDLINE:37498822", "MEDLINE:37542149", "WOS:001021251800001", "MEDLINE:37519756", "WOS:001009827900001", "WOS:001030913600001", "WOS:001024928800001", "WOS:001025320900001", "WOS:001025745200007", "WOS:001000830500002", "WOS:001005632500006", "MEDLINE:37461243", "MEDLINE:37451900", "MEDLINE:37530745", "WOS:000971500200016", "WOS:001023645700027", "MEDLINE:37451964", "MEDLINE:37533429", "WOS:001026977100001", "WOS:001025438900081", "WOS:001023913300001", "WOS:001021677000001", "WOS:001017754700116", "WOS:000989629702191", "MEDLINE:37500919", "WOS:001010758900001", "WOS:001028812100001", "WOS:001002868700001", "WOS:001011088400036", "WOS:001032997500001", "WOS:001030836800001", "WOS:001023659700001", "MEDLINE:37453602", "WOS:001020550500001", "WOS:001032699800001", "MEDLINE:37466229", "WOS:001027903000004", "WOS:001032045200001", "MEDLINE:37463471", "WOS:001011240000002", "WOS:001008232900518", "WOS:001022772500001", "MEDLINE:37498558", "WOS:001035299600001", "MEDLINE:37454627", "WOS:001025340000001", "MEDLINE:37482530", "WOS:001016217000002", "WOS:001007554700004", "MEDLINE:37498692", "WOS:001022073500002", "WOS:001023257300001", "MEDLINE:37474091", "WOS:001017781500015", "MEDLINE:37473645", "MEDLINE:37501609", "WOS:001016413500002", "WOS:001004708900001", "MEDLINE:37480920", "MEDLINE:37505211", "WOS:001027338600002", "WOS:001035115500001", "MEDLINE:37459233", "WOS:001000080700022", "WOS:001023742100004", "WOS:001024037300001", "MEDLINE:37534980", "MEDLINE:37507765", "MEDLINE:37491375", "WOS:001032291400001", "WOS:001021926000001", "MEDLINE:37466290", "MEDLINE:37531203", "WOS:001023250500001", "WOS:001025081300001", "WOS:000971500201078", "MEDLINE:37520018"]
  3.2.0 :004 > wos_uids_added.size
   => 277 

• [x] compare the total_contributions_added count and the wos_uids_added and see if the pass they sanity check .... i would expect them to be very similar, as the authors are very similar between prod and cap-dev-a and the harvest timeframes are very similar, only off by a day or so
  • Contributions added: 598 ➡️ 1,819
  • WOS UIDs added: 263 ➡️ 277

[mjgiarlo]

@peetucket @justinlittman I did the above work but I'm not sure I did it right and I'm also not sure the results pass the sanity check. See my methodology and results above. I'll keep this assigned to me for now but I'm not sure what to do at the moment.