TRACK: sprint review 6.2.24

[cmarooney-stfc]

Still uncovered

• alex: cylindrical projection - needs more input based on needs of spherical projection - scale: q is angstr only, need rlu
• detector information, waiting on Duc may not happen pre-construction
• pace-python (Duc) - finished 20.2 2 weeks to finish minor issues if available

prioritizing:

bugs (alex, end w/c 5/2/24) (jacob #1525 quick, #1524 lo-pr, #1519 longer, characterise for Duc, #1510 fixed) (joint #1505 to alex) docs spherical, cylindrical (infrastructure working)

jacob profiling is at data assembly stage

profiling - slowdown due to IDAAAS issues, but same as 3.6.3 push on idaaas behaviour incl Toby nb slurm setup may have multiple caches

use toby timing script

New issues since last sprint review 23.1.24

• [ ] by Jacob #1525
• [ ] by Jacob #1524
• [ ] by Jacob #1519
• [ ] by Jacob #1513
• [ ] by jacob #1510
• [ ] by Alex #1511
• [ ] by Alex #1508
• [ ] by Alex #1505

NB H4UX epic is at

• [ ] #1464 which includes Alex' stuff above

New at last sprint review 23.1.24

Re: Toby's post-construction plans: action to review and record of storage at #1529. Ticket includes David Voneshen feedback noted previously in the last sprint review ticket.

From ALEX:

• [x] #1501
• [x] #1499
• [x] #1498
• [x] #1497
• [x] #1495
• [x] #1493
• [x] #1491
• [ ] #1487
• [ ] #1483
• [ ] #1481
• [ ] #1475
• [x] #1474
• [x] #1472

From JACOB

• [ ] #1490
• [ ] #1489
• [ ] #1488 assigned to TOBY

From TOBY:

• [ ] #1485
• [ ] #1488 (see Jacob list above)

From last time:

(a) Major items in progress or planned

ALEX:

• [ ] Filebacked: #1302 by 18.12.23 possibly earlier - EPIC modified, most things finished/under review or minor side issues - separate tickets for remaining items #1352 #1147 (only if there is tim) OTHERWISE DONE
• [ ] Tutorial validation: #1314 start 2.1.24 latest, end 18.1.24. Plus any tickets arising. At 8.1.24 20% complete. (based on Alex local copy, check if current given multifit comments, check Sharepoint, circulate, cm to create sharepoint folder, by 10.1.24) DONE
• [ ] Spherical projection - done bar docs, Christian concerns - Christian has docs and example - ALEX to discuss with Duc on whether this meets user needs, soon (based on Christian's uncertainties, get representative dataset) - image of what Christian wants and what Christian got (+rb-no. if available)
• [ ] Fan IDAAAS problem - Alex to contact Fan re has she run script, what results
• [ ] Cylindrical projections (5d) after spherical and IDAAAS performance
• [ ] focussed projections (5d) insert TICKETS 8.1.24 - 22.1.24
• [ ] Horace help issue needs ticketing to continue having Fan work on this
• [ ] Spherical projection with Tobyfit (a) plot ellipsoid (b) resconv (needs 2 tickets) #1040 23.1.24 to 23.2.24 (latest, could be 15.1.24) depending on scoping exercise TOBY - working, docs only, by 26.1.24 ALEX (DUC?) TO DISCUSS 8.1.24 do first to inform scope of next 2: post-construction

JACOB

• [ ] Tobyfit memory: #760 in progress, by 8.1.24 PR await submission, nb still not filebacked, may need pageop implementation for 4-way zip plus extra optimisations. Ensure this solves Duc's issue. PR awaits approval
  • [ ] Profiling: #676 BACKGROUND ONGOING 2.1.24-2.2.24 have fake data, need real data, want known time-hoggers JACOB TOBY TO DISCUSS w/c 15.1.24 (profilng, parallelization) - JW plan emailed and in ticket - finish Feb w1
  • [ ] Fix CI, need ticket, SL7 nodes retiring. (2d) Add additional Matlab versions to new R8 nodes.
  • [x] Jacob #1172 JENKINS ONLY -
  • [ ] Review documentation changes (how do we do this?) still need to ask Excitations/David eg at Excitations awayday
  • [ ] email from Boothroyd post-doc - need to investigate their problems
  • [x] pace-builder a/c needs 2fa - sort within week

JW notes: 17/1/24

• David harangued will schedule and arrange documentation review within Excitations. Plan akin to example script development assigning sections to individuals.
• Viviane's (Boothroyd's post-doc) IDAaaS workspace expired over New Year's break email sent requesting new one
• CI pipelines set up and "working" but lacking cppcheck, email sent to Alan Kyffin. Followup when time has passed to chase.

Update 18/1/24:

• Alan has installed more MATLAB versions and newest cppcheck

CHRIS

Detector tickets

• [ ] #1432 and earlier (#1350, #1298, #1338, #1398)

Container tickets

• [ ] #1157
• [ ] #1078
• [ ] Note on hashing issues, Toby to ticket

TOBY

• [ ] Talk to Duc about info in nxspe; DUC/TOBY Chase Mantid team detector info (script failing): Duc says Merlin and Maps raw files have info - Toby to browse hdf5 output - need to get this into nxspe, raise Mantid epic - Toby and Jacob talk to Duc
• [ ] Toby ticket hashing issues, yes, Chris to add.
• [ ] troubleshoot, ongoing, background
• [ ] run challenging cases (generate them) with H4 e.g. cut-sym
• [ ] Radu Epic - TGP review and comment on JW tickets - this sprint

---

(b) Suggestions for next areas to do

• [ ] Parallelization #1253, includes:-
• [ ] #1203 struggled to reproduce but needs to be confirmed & fixed, JW HIGH PRIORITY early Jan
• [ ] Performance - #1419 Tentative for JW 9.1.24
• [ ] Resolution function plot issues - #1408 Toby to deal with BACKGROUND, Jan 24
  • [ ] ??? #922 Toby progressing; required for mgenie detector tools (???)#921

---

(c) New issues prioritised up to 28.11.23

Filebacked

• [ ] #1404

Performance

• [ ] #1372, #1364 WAIT ON BENCHMARKING

Tobyfit/Detectors

• [ ] #1430
• [ ] #1419
• [x] #1398

Parallel

• [ ] #1393 [POST-CONSTRUCTION, SCARF check of fixes to IDAaaS #12No tickets92, #1391]

Tests

No open tickets

Filebacked epic #1302

• remaining time estimated as 3 weeks = 19.12.23

---

(e) Previous issues still outstanding

Think these are largely dealt with above See previous sprint review 12.12.23 for details when the main issues are cleared

[cmarooney-stfc]

Closed as superseded by #1555