Better guessing of content types

[haesleinhuepf]

When adding a new Zenodo or Github repository, we should "guess" the content type. E.g. if its "code" or "slides" or "notebooks" or ...

It would be nice if this could be done automatically.

[haesleinhuepf]

git-bob think about this

[github-actions[bot]]

^{This message was generated by git-bob (version: 0.11.0, model: github_models:gpt-4o, log), an experimental AI-based assistant. It can make mistakes and has limitations. Check its messages carefully.}

To address the issue of better guessing content types, one approach is to analyze the metadata from Zenodo. Here's a minimal code change suggestion in the complete_zenodo_data function to include content type guessing:

def complete_zenodo_data(zenodo_url):
    from generate_link_lists import read_zenodo, remove_html_tags
    zenodo_data = read_zenodo(zenodo_url)
    entry = {}
    urls = [zenodo_url]

    if 'doi_url' in zenodo_data.keys():
        doi_url = zenodo_data['doi_url']
        if doi_url not in urls:
            urls.append(doi_url)
    entry['url'] = urls

    if 'metadata' in zenodo_data.keys():
        metadata = zenodo_data['metadata']
        entry['name'] = metadata['title']
        if 'publication_date' in metadata.keys():
            entry['publication_date'] = metadata['publication_date']
        if 'description' in metadata.keys():
            entry['description'] = remove_html_tags(metadata['description'])
        if 'creators' in metadata.keys():
            creators = metadata['creators']
            entry['authors'] = ", ".join([c['name'] for c in creators])
        if 'license' in metadata.keys():
            entry['license'] = metadata['license']['id']

        # Guess content type based on keywords in the title/description
        title_desc = metadata.get('title', '') + " " + metadata.get('description', '')
        if any(keyword in title_desc.lower() for keyword in ['code', 'github']):
            entry['type'] = 'code'
        elif any(keyword in title_desc.lower() for keyword in ['slides', 'presentation']):
            entry['type'] = 'slides'
        elif any(keyword in title_desc.lower() for keyword in ['notebook', 'jupyter']):
            entry['type'] = 'notebooks'
        else:
            entry['type'] = 'unknown'

    if 'stats' in zenodo_data.keys():
        entry['num_downloads'] = zenodo_data['stats']['downloads']

    return entry

This basic implementation uses keywords to guess the content type from the title and description. Adjust the keywords or refine the logic as needed.

[github-actions[bot]]

^{This message was generated by git-bob (version: 0.11.0, model: github_models:gpt-4o, log), an experimental AI-based assistant. It can make mistakes and has limitations. Check its messages carefully.}

To implement automatic guessing of content types for Zenodo or Github repositories, we can use a simple heuristic approach based on the presence of specific files. Here's a basic outline of how you could implement this:

def guess_content_type(file_list):
    if any(file.endswith('.ipynb') for file in file_list):
        return 'notebooks'
    elif any(file.endswith('.pptx') or file.endswith('.ppt') for file in file_list):
        return 'slides'
    elif any(file.endswith('.py') or file.endswith('.js') for file in file_list):
        return 'code'
    else:
        return 'unknown'

# Example usage:
repo_files = ['main.py', 'README.md', 'plot.ipynb']
content_type = guess_content_type(repo_files)
print(f"Guessed content type: {content_type}")

This is a minimal implementation and can be expanded with more file types and categories as needed.

[haesleinhuepf]

I like the file-list idea a lot. Both github and zenodo allow deriving file-lists from repositories.