ukwa / w3act

w3act is an annotation and curation tool for building web archive collections
Apache License 2.0
19 stars 6 forks source link

Add TSV & JSON data format support for CrawlPermissions export #697

Closed min2ha closed 1 year ago

min2ha commented 1 year ago

add TSV and JSON data format support for CrawlPermissions' filtered data export along with CSV and TSV at: Targets -> Reports (Open UKWA licences TAB)

crarugal commented 1 year ago

Looking at: https://dev.webarchive.org.uk/act/reports/search?crawlPermissionsStatus22=PENDING&p=0&exportType=&curator=-1&organisation=-1&startDate=09-08-2021&endDate=11-08-2021&action=search image

the JSON and TSV export well, but one of the keys for the JSON export looks to be incorrect, as it's referencing the created date of the target, not the created date of the licence request, looking at the first record: image

The TSV of the same record shows the requested date: image

https://www.webarchive.org.uk/act/targets/16 image

The same is true when filtering by "Granted"; the JSON export is only showing the date the target was created and not the granted date: https://dev.webarchive.org.uk/act/reports/search?crawlPermissionsStatus22=PENDING&p=0&exportType=&action=searchGranted&curator=-1&organisation=-1&startDate=09-08-2021&endDate=11-08-2021 image

image

https://www.webarchive.org.uk/act/targets/143342 image

min2ha commented 1 year ago

thanks @crarugal, fix in progress

crarugal commented 1 year ago

Looking at: https://dev.webarchive.org.uk/act/reports/search?crawlPermissionsStatus22=PENDING&p=0&exportType=&action=searchGranted&curator=-1&organisation=-1&startDate=09-08-2021&endDate=11-08-2021 image

TSC export, when formatted to the appropriate date format, works well: image

the exported JSON also has the correct keys, showing granted, or requested, or both: image