statisticssweden / PxWeb

Web application for disseminating statistical tables
https://www.scb.se/pxweb
Apache License 2.0
31 stars 23 forks source link

Footnotes are split when containing at least two hashtags (#) #202

Open emilwest opened 2 years ago

emilwest commented 2 years ago

Describe the bug When a footnote such as VALUENOTE:s or NOTE("variablename") contains at least two hashtags (#) for adding line breaks, the footnotes are split into multiple footnoes with duplicated headings.

To Reproduce Steps to reproduce the behavior:

  1. Go to 'Nordic Statistics Database/AGRI10.px'
  2. Scroll down to open 'Footnotes'
  3. Look at footnotes for 'crops' and 'area/main area' and compare with content of px-file.

Expected behavior

I expect a VALUENOTE or a NOTE("variablename") to have the same title and not be split up into multiple footnotes if they contain hastags #. Something like this:

image

Actual behavior

In the px-file, the metadata is as follows:

NOTE("crops")="Crops classified according to according to Regulation (EC) No 543/2009, Commission Delegated Regulation (EU) 2015/1557 and the ESS agreement. ##In EU standard humidity.";

Shown in pxweb as:

image

And this footnote is shown as:

VALUENOTE("area","Main area")="The concept of 'main area' corresponds, in general, to the area of the land parcel. The crop/occupation linked to that area is the unique or main crop having occupied the parcel during the crop year. In the case of annual crops, the main area should corr"
"espond to the sown area. For permanent crops it refers to the total planted area (including non-productive plants).  In the case of successive crops (mostly vegetables) it refers to the main crop that occupied the parcel during that year. These special c"
"ases are described in detail in the Annual Crop Statistics Handbook found at Eurostat. ##Measure: Sums of main areas of land parcels cultivated with crops.";

image

Environment information

emilwest commented 2 years ago

Here is an example where the double hashtags work: WORK30

image

Where the metadata looks like this:

NOTE="Registered residents: Foreign-born with two foreign-born parents. Foreign-born with native-born adoptive parents are excluded. # #The frequency is calculated by dividing the amount of employed persons in a given group by to total population in the group."
" Example: Frequency Employed, age group 20-64 = (20-64 employed) / (Total 20-64 population). Percentages or frequencies are produced with a base of a minimum of 50 individuals per cell. #Country background: Foreign-born with two foreign-born parents = th"
"eir own country of birth. Descendants =  mother's country of birth, or father's country of birth if mother's is unknown. Rest of the population = Reporting country  i.e. one or both parents' country of birth is the reporting country. # #Employed: Employe"
"d in per cent is the proportion of employed in the whole population aged 20-64. See valuenotes for individual reporting #countries.";
VALUENOTE("reporting country","Denmark")="Reference date/week: Last working day of November. # #Employed: People who are working at least one hour in the week of reference. The employed are either employees, self-employed or assisting spouses.  Military service is included as employment.";
VALUENOTE("reporting country","Finland")="Includes Åland. # #Reference date: End of the year. # #Employed: Those employed during the reference week and who were neither registered as unemployed jobseekers at the labour exhange office nor undergoing military or non-military service. Employment st"
"atistics classifies those performing compulsory military service (conscripts) as a group of its own in the classification of main type of activity.";
VALUENOTE("reporting country","Norway")="Reference date/week: Usually third week in November. # #Descendants: Native-born with two foreign-born parents and four foreign-born grandparents. # #Employed: Registered residents who performed work for pay for at least one hour in the reference week (N"
"ovember), or who were temporarily absent from work because of illness, holidays, maternity leave etc. Those in military service are considered employed and are part of the population.";
VALUENOTE("reporting country","Sweden")="Reference date/week: November. # #Employed: Persons registered as gainfully employed in the Labour statistics based on administrative sources. Payed military work is considered employment. Between July 1 2010 and July 1 2017 Sweden did not have mandatory"
" military services.";

The only difference I can note here is that the hashtags are written as # # instead of ##.

likp commented 2 years ago

The problem is reproducible.

A possible fix could be to let the AddFotenotes in FootnoteList aggregated all footnotes for a specific entity.