Note taking MVP meeting 18 May 2023

[mialondon]

Please add notes as comments below.

See also https://github.com/alan-turing-institute/Living-with-Machines/issues/3211

[mialondon]

Today's key topics:

Is anyone actively using the database in research for work to be substantially completed by mid-July? Demo of what's available What do we put on the roadmap for potential future development?

[mialondon]

Griff's slides for today https://hackmd.io/@griff-rees/H1VN7S7r2

[mialondon]

Access to the server requires some specific IP/SSH work.

User accounts don't have admin access by default.

Currently offers access to newspapers (metadata), not census, (newspapers) fulltext, Mitchells or gazetteer data.

[mialondon]

'Data provider' values need to be updated for accuracy

[DavidBeavan]

Demo: Accessible (via auth and routing) from anywhere. Two views: django (python) and admin browse. Data (203m+ items) is from last summer. Titles are truncated to <= 100 chars. The schema is available (link pending - someone help out).

[mialondon]

Demo. Accessible (via auth and routing) from anywhere. Two views: django (python) and admin browse. Data is from last summer. The schema is available (link pending - someone help out).

Searching slack, previous links to schemas https://dbdiagram.io/d/62bb46ba69be0b672c5d2a15

[mialondon]

Example notebooks e.g. looking for extra long titles in newspapers metadata

[DavidBeavan]

DECISION: if there are research needs, focus will be on the cloud hosted deployment (for lwm people access only). Local install should still be an option. Both should be well documented and deployment as automated as possible

[DavidBeavan]

DECISION: For environmental scan, their work is not dependent on lwm-db

[DavidBeavan]

INFO: linking census and Mitchell's will take additional work (we can build on previous work)

[dcsw2]

DECISION: For environmental scan, their work is not dependent on lwm-db

Not for the paper currently being completed, but the next steps for ES work should make use of the DB

[mialondon]

DECISION: For environmental scan, their work is not dependent on lwm-db

Not for the paper currently being completed, but the next steps for ES work should make use of the DB

Is that a roadmap thing, post-July 2023?

[thobson88]

My suggestion based on the discussion:

Things needed for LwM:

• Access via Jupyter [already available]
• Updates to the database:
  • Import Mitchell's data [Kalle: this was done last summer so should be straightforward]
  • Import the latest version of the BNA metadata [with Kalle's help to run intermediate steps]
• conference poster
• data paper?

Things needed for legacy:

• Tidy up repo and make public
• Documentation for future cloud deployment
• Downloadable database dump [BL repository]

Things we should leave out due to lack of time:

• Access via Django admin app
• Gazetteer
• Census

[dcsw2]

From @mcollardanuy's comment here: is the material for the Gazetteer not already more-or-less in place?

[ruthahnert]

Seconding @thobson88's list (with Daniel's caveat acknowledged), and messaging you on slack now. Let's revisit priority list and sign off on Monday. Thank you for all you've done so far!

[mialondon]

Thanks @thobson88 for your summary! Can I add 'review and confirm column labels' so that sources e.g. FMP, BNA, etc are accurately labelled / documented?

[griff-rees]

Closing this as it fed into the lwmdb roadmap.