Missing protocols from the 1990s

[fredrik1984]

According to Lotta at the Riksdag Library, the protocols below might be missing in our data (if they do, we can get them from the library in a digital format). Make sure these are included in our corpus.

prot 1991/92:76 prot 1991/92:77 prot 1991/92:78 prot 1990/91:22 prot 1990/91:88 prot 1990/91:94

prot 1992/93:3 prot 1992/93:4 prot 1992/93:9

[MansMeg]

Great! I think we should - in general go through all years by the protocol numbers to check if there are further protocols that are missing.

[BobBorges]

I've checked the sequence of numbering for all our protocols, as a possible indication of absence. I'm attaching a CSV of the results. In general: protocol-completeness-issues.csv

• 1867 -- 1874 appear not to be numbered sequentially, or if they are, we're missing a lot.
• There are no significant duplicates (there's a good reason for all the ones listed in the sheet)
• Aside from 1867 -- 1874, all sequences start at 1 or zero (or continue from the previous sequence e.g. 1941--fk, goes until 43, 1941-höst-fk starts at 44) except:
• --- 197576 starts with 2
• We miss protocols in the 1990's as mentioned above
• We miss protocols in the 1970's
• There are a handful of others (ca. 5) scattered around the rest of the time span
• 200809 and 202122 seem to have a misnumbered "last in the range" protocol (678 and 999, respectively) causing it to look like we're missing a bunch there.

[BobBorges]

Do we want this code as a unit test, or just a list of what's missing so we can try to get them?

[fredrik1984]

Great work! Regarding the unit test, @MansMeg can give an answer for that.

Regarding the missing protocols, the 1970s protocols should be fixed from KBLab's side by now. We can also check with the Riksdag Library after the summer if they can provide us with the rest of the missing protocols

[BobBorges]

Ok, will check which ones I can find on KBLab and make a list of the rest.

[MansMeg]

I think it is a great idea with a unit test. Then there would be a warning if any documents were accidentally removed.

Great. Then I expect there to be the following tasks?:

• [ ] Check 1867 -- 1874 with @Lottabrorsson how the protocols were numbered so we know.
• [ ] Include the missing protocols in the corpus
• [ ] Setup a unit test that fails if any protocols are missing

[BobBorges]

1860--70s looks like they're labeled by date.

[BobBorges]

There's another issue here regarding completeness, and I'm not sure how to systematically check this, but some protocols seem to be incomplete, like 1869ak0118 is only one page that appears to end mid-sentence.

[fredrik1984]

There's another issue here regarding completeness, and I'm not sure how to systematically check this, but some protocols seem to be incomplete, like 1869ak0118 is only one page that appears to end mid-sentence.

I checked this and the protocols are not incomplete. However, since they seemed to be labeled by date, and follow one after the other, there are sometimes no page breaks between two protocols. For example, protocol 1869ak0118 concerns January 18, which is very short meeting and the next "protocol day meeting" (January 19) thus starts on the same page. protocol 1869ak0119, consequently, both contain the same page for January 18 and the following pages for the January 19 meeting.

They seem to do this through 1871 when I checked. For 1872, 1873, and 1874 the protocols are also marked with date, but this problem of conflicting pages does not seem to exist for those years. In 1875 they start to name protocol with a number instead (starting with 1, 2, 3...)

This is a similar issue that we have raised regarding the motions, which also follow one after the other, making it possible to have the end of one motion and the start of another (if I remember correctly).

@BobBorges, can we somehow automatically solve this to avoid duplicate texts? Like trying to match the correct date with the date headline on the first page of every protocol between 1867–1871? I suppose it would be too much job to do this manually?

[BobBorges]

@fredrik1984 it should be possible to remove text that occurs above the current date's appearance in the text, or similarly below another date, indeed.

[BobBorges]

Closed as duplicate of #211

[Lottabrorsson]

According to Lotta at the Riksdag Library, the protocols below might be missing in our data (if they do, we can get them from the library in a digital format). Make sure these are included in our corpus.

prot 1991/92:76 prot 1991/92:77 prot 1991/92:78 prot 1990/91:22 prot 1990/91:88 prot 1990/91:94

prot 1992/93:3 prot 1992/93:4 prot 1992/93:9

@BobBorges @fredrik1984 @MansMeg We will send you a list of what we can deliver to you, regarding both protocols and motions 1971-1994/94. This will be done next week. I'm wondering if there are some protocols that you know you are missing? So we can ensure we send them to you. I think this partly has to do with the fact that we published documents after that KB-lab downloaded it from the riksdag.

[BobBorges]

@Lottabrorsson reply over in #211