MI-DPLA / combine

Combine /kämˌbīn/ - Metadata Aggregator Platform
MIT License
26 stars 11 forks source link

create Job.mapped_fields_analysis() method #413

Closed ghukill closed 5 years ago

ghukill commented 5 years ago

Currently, an analysis of all fields in ElasticSearch that were mapped and created for a Job are calculated and stored when running the view views.job.job_details. There is a code block that looks for them in the Job.job_details dictionary, and if not found, generates and saves:

    # mapped field analysis, generate if not part of job_details
    if 'mapped_field_analysis' in job_detail.keys():
        field_counts = job_detail['mapped_field_analysis']
    else:
        if cjob.job.finished:
            field_counts = cjob.count_indexed_fields()
            cjob.job.update_job_details(
                {'mapped_field_analysis': field_counts}, save=True)
        else:
            LOGGER.debug('job not finished, not setting')
            field_counts = {}

But it would be handy to have a shorthand method on the Job model that would handle this calculation & storage and/or returning. Something like Job.mapped_fields_analysis. The fact that it is plural would be important to distinguish from the CombineJob.field_analysis() singular that is expecting a particular field string as argument.

This would allow retrieving that analysis from other contexts other than the job details view, e.g. graphql endpoint.

ghukill commented 5 years ago

Addressed with this commit in graphql branch: https://github.com/MI-DPLA/combine/commit/00bb1bea51da2a81c0f90b95ffedcd9179d3cefc