Automatic Report Generation from CKAN Contents

[p-a-s-c-a-l]

Export datasets from CKAN on the command line with ckanapi and use carbone.io to generate documents out of Data Package JSON.

Example: Dump Resource Summer days (SD) as Data Package:

ckanapi dump datasets "summer-days-sd" -r https://ckan.myclimateservice.eu/ \
--datapackages=./dump_directory/

See also https://github.com/clarity-h2020/ckan/issues/16

Related Tasks:

• [x] Create Dataset Template
• [x] Create ckanapi export statements
• [x] Support For Markdown

[p-a-s-c-a-l]

Hello “{d.firstname} {d.lastname}”!

[p-a-s-c-a-l]

Hello {d.firstname} {d.lastname} !

[p-a-s-c-a-l]

Hm, https://github.com/clarity-h2020/data-management-plan/issues/1#issuecomment-604264810 probably caused by bookmark xml tags. 🙄

[p-a-s-c-a-l]

{d.firstname}

<w:r w:rsidRPr="002E7781">
    <w:rPr>
        <w:rFonts w:asciiTheme="minorHAnsi" w:hAnsiTheme="minorHAnsi"/>
        <w:sz w:val="22"/>
    </w:rPr>
    <w:t>d.firstname</w:t>
</w:r>

-> carbone

:hurtrealbad:

[p-a-s-c-a-l]

ckanapi Action Arguments:

Very ambiguous. What's the role of the JSON Here? The result of the API call? But why the parameter id in the API result and the parameter namein the JSON result?

This does not work:

ckanapi action package_show id="Summer days (SD)" -r https://ckan.myclimateservice.eu/

raceback (most recent call last): File "B:\Python27\Scripts\ckanapi-script.py", line 11, in load_entry_point('ckanapi==4.3', 'console_scripts', 'ckanapi')() File "b:\python27\lib\site-packages\ckanapi\cli\main.py", line 112, in main for r in action(ckan, arguments): File "b:\python27\lib\site-packages\ckanapi\cli\action.py", line 57, in action files=file_args) File "b:\python27\lib\site-packages\ckanapi\remoteckan.py", line 87, in call_action return reverse_apicontroller_action(url, status, response) File "b:\python27\lib\site-packages\ckanapi\common.py", line 126, in reverse_apicontroller_action raise NotFound(emessage) ckanapi.errors.NotFound: Not found

WHAT not found? command? resource?

[p-a-s-c-a-l]

Example from Action Arguments doesn't work either:

ckanapi action package_search 'facet.field:["organization"]' rows:0 -r https://ckan.myclimateservice.eu/

KEY:JSON argument u"'facet.field" has invalid JSON value u"[organization]'"

😞

[p-a-s-c-a-l]

Array Manipulation does not work:

{d.tags:arrayMap("; ",|,name)}

CLARITY";"Climate Indicators";"EURO-CORDEX";"WP3";"open-data";"output-data

[p-a-s-c-a-l]

{d.tags:arrayMap(; ,|,name)}

CLARITY;Climate Indicators;EURO-CORDEX;WP3;open-data;output-data

It seems that no quotes nor commas are supported in (ODT) templates. 😞

[p-a-s-c-a-l]

"tags": [{
            "display_name": "CLARITY",
            "id": "fdb805ad-2f49-4cf5-b498-01a7d7afdd14",
            "name": "CLARITY",
            "state": "active",
            "vocabulary_id": null
        }, { ...

None of these works:

{d.tags:arrayMap(“, “ ,| ,display_name)}
{d.tags:arrayMap(`, ` ,| ,display_name)}
{d.tags:arrayMap(", " ,| ,display_name)}

“CLARITY““Climate Indicators““EURO-CORDEX““WP3““open-data““output-data `CLARITYClimate IndicatorsEURO-CORDEXWP3open-data``output-data "CLARITY""Climate Indicators""EURO-CORDEX""WP3""open-data""output-data

[p-a-s-c-a-l]

As workaround use {d.tags:arrayMap(;,|,display_name)}:

CLARITY;Climate Indicators;EURO-CORDEX;WP3;open-data;output-data

[p-a-s-c-a-l]

It seems that there are more general problems with quotes and spaces: https://github.com/Ideolys/carbone/issues/59#issue-538315672

    • {d.test:ifEmpty(“some other sentence”)}
    • {d.test:ifEmpty(very strange)}
    • {d.test:ifEmpty(very, strange )}
    • {d.test:ifEmpty( very, strange )}
    • {d.test:ifEmpty(very; strange)}
    • {d.test:ifEmpty(\n  1    2   3)}
    • {d.test:ifEmpty(;       123 4)}
    • {d.test:ifEmpty(;   \n    123 4 X)}

[p-a-s-c-a-l]

ENV VAR substitution does not work:

CLARTIY H2020 Data Management Plan v${GITHUB_REF} generated by ${GITHUB_ACTOR} from CLARITY CKAN Build #${GITHUB_RUN_NUMBER} triggered by ${GITHUB_SHA}

That's the the typical variables mess in YAML: ${{ github.ref }} vs ${GITHUB_REF} : github-context vs env-context. WTF?

Now, how to get the output on an ENV var?

• ${{ ENV_VAR }}
• ${ENV_VAR}
• $ENV_VAR
• ${{ env.VAR }}
• ???

[p-a-s-c-a-l]

According to using environment variables we can use e.g. $GITHUB_REF . But in reality this leads to:

Your workflow file was invalid: The workflow is not valid. .github/workflows/release.yml (Line: 71, Col: 15): Unrecognized named-value: 'GITHUB_REF'. Located at position 1 within expression: GITHUB_REF

[p-a-s-c-a-l]

done