Add more recent Läroplaner and confirm permission to re-post them

[santiagohermo]

This issue has two goals: 1) Add more recent versions of the Läroplaner to the build 2) Confirm that we are allowed to re-post the pdfs of the Läroplaner in this repo (see here)

[jmshapir]

@santiagohermo thanks!

Per https://github.com/JMSLab/LaroplanOCR/pull/2#discussion_r793923912, my advice is to wait until @dagese weighs in before we reach out to the library on (2).

[santiagohermo]

Thanks @jmshapir! Unfortunately I already sent an email via their contact form :/

[jmshapir]

@santiagohermo no worries! Keep us posted. :-)

[dagese]

@santiagohermo @jmshapir

As you may have seen in the email I forwarded, it is fine for us to share materials based on the digital Läroplans.

[santiagohermo]

Thanks @dagese! I also got a positive response from Gothenburg University Library.

fyi @jmshapir @miikapaal

[santiagohermo]

Reminder to self. Confirm that we use the 'subject' version of the 1980 laroplan because it includes the non-subject version

[santiagohermo]

Notes to self:

• A link with all laroplaner from the Gothenburg University is here
• The 1994 curriculum has a different name. See wikipedia entry for Lpo94
• The 2011 version went back to the Läroplan för grundskolan name, see wikipedia entry for Lgr11

[santiagohermo]

Maybe @miikapaal or @dagese can give me a quick hand here? I'm trying to figure out what files correspond to the 1994 and 2011 versions of the curriculum. Gothenburg University posts curricula here

1994 års läroplan för det obligatoriska skolväsendet (Lpo 94)

My impression is that the introduction to the 1994 curricula is the file I highlighted in the screenshot below, and all other files correspond to subjects. Do you agree?

Screenshot


2011 Läroplan för grundskolan

The 2011 seems more straightforward. I think it should be the one in the middle or on top of the files I highlighted below (they are the same file). See here

The bottom file of the ones I highlighted below (this one) is actually for pre-school, so it shouldn't be included.

Do you agree?

Screenshot


---

If my intuition is correct I think that adding the 1994 curriculum will ~ill~ give a bit of work, since we should decide what files are relevant to include in the build. The 2011 curriculum seems straightforward though

[jmshapir]

If my intuition is correct I think that adding the 1994 curriculum ill give a bit of work, since we should decide what files are relevant to include in the build. The 2011 curriculum seems straightforward though

@santiagohermo FWIW, if this is the case, it seems fine to me to include only 2011 in the initial release, and to plan to include 1994 in a later release. Among other things, this would allow us to gauge interest / utilization before devoting significant additional effort.

[santiagohermo]

Thanks @jmshapir! That makes sense. I think that if we can quickly figure it out in the call, then let's include them now. If it takes more effort we can plan to include in a later release.

[santiagohermo]

Per today's call:

• For the 1994 Laroplan, use the files Läroplaner för det obligatoriska skolväsendet och de frivilliga skolformerna :Lpo 94 : Lpf 94 and Kursplaner för grundskolan. Make sure to caveat that this include pre-school
• For 2011 use Läroplan för grundskolan, förskoleklassen och fritidshemmet 2011. Do not use the Läroplan för grundsärskolan 2011, which doesn't work because grundsärskolan != grundskolan (they mean respectively special primary school, GTranslate, and primary school, GTranslate)
• Add a note in the wiki with a plan to revise these files in more detail and include only relevant portions of the files in the build.

Thanks for the help @dagese @jmshapir @miikapaal!

[jmshapir]

Thanks @santiagohermo!

For 2011, I think we said that we should try if possible to exclude the pages that relate to preschool (förskola).

For 1994, I'm not sure if preschool is included there?

@dagese please let us know if I have this wrong, thank you!

[dagese]

Thanks @jmshapir @santiagohermo!

I reviewed the docs carefully. None of the curricula include pre-school.

Förskoleklass != förskola (= preschool). At some point, perhaps in 2011, the Swedish primary school was extended from ages 7-16 to 6-16. The first grade was called pre-schoolclass, förskoleklass, and is therefore included in the 2011 curriculum. So no need to discuss pre-school as we think of it.

There is a separate curriculum for pre-school but it does not look to be included in these documents.

Sorry for confusion.

[jmshapir]

Thanks @dagese! Your language is subtle. :-)

In that case I think the plan reverts to the one in https://github.com/JMSLab/LaroplanOCR/issues/3#issuecomment-1031544754, minus the point about preschool.

@dagese let us know if that doesn't sound right, and thanks again!

[dagese]

Thanks @dagese! Your language is subtle. :-)

In that case I think the plan reverts to the one in #3 (comment), minus the point about preschool.

@dagese let us know if that doesn't sound right, and thanks again!

Thanks @jmshapir! This plan sounds great.

[santiagohermo]

Thanks for the discussion @dagese @jmshapir! I'll move forward and implement the plan then. I'll let you know when I make some progress and we can make sure I got the right files.

[santiagohermo]

Quick question @jmshapir. I noticed that the pdf of laroplaner are actually quite heavy. Currently, the raw folder weighs 389mb, which makes cloning the repo a bit slow.

I think it's convenient to host the pdfs in the repo, even if cloning is a bit slower. But maybe there are other downsides (is there a problem with the cost?). Alternatively, we could ignore the pdf files as well and have a python script that downloads them. What do you think?

[jmshapir]

@santiagohermo I think at this scale the size is manageable, and I agree it's nice to have everything self-contained if we can.

If storage impact starts getting much bigger we could revisit and try an approach like what you say (e.g. grabbing PDFs from website).

Another note is that we should avoid making revisions to the PDF files when we can. Since git can't really "diff" those, the storage impact of even a small change to the PDF can be very large.

Thanks!

[santiagohermo]

Thanks @jmshapir! That makes sense, and I don't think we'll ever need to modify the pdfs. I'll proceed with this issue then.

Let me note that the new laroplaner bring the weight of the /raw/ folder to 419mb, ~30mb more, which seems manageable as well.

[santiagohermo]

I finished the addition of the new files to the pipeline @dagese @jmshapir @miikapaal. The updated figure from the example looks like this

I'll move to PR now

[santiagohermo]

Continues in its PR #8

[santiagohermo]

Summary: In this is issue we did the following:

• Confirmed that we are allowed to re-post the curricula, see here
• Added new files for the 1994 and 2011 curricula, see here

Changed merged to master in https://github.com/JMSLab/LaroplanOCR/commit/bd61c91d53edc2bdec6f6c2e61fad26d36913eef.