cern-sis / issues-scoap3

0 stars 0 forks source link

Revise what fields are required #287

Open ErnestaP opened 5 months ago

ErnestaP commented 5 months ago

Now, is not clear which fields are required and which are not, the flags to mark the required field in parsers are added almost randomly. As we know, Anne wants to see the article uploaded to db even if it's non-compliant. As I mentioned, the output of Airflow workflows should create a record in DB, bypass this function from scoap3 backend:

def import_to_scoap3(data, migrate_files):
    licenses = _create_licenses(data["license"])
    article = _create_article(data, licenses)
    if migrate_files:
        _create_article_file(data, article)
    _create_article_identifier(data, article)
    _create_copyright(data, article)
    _create_article_arxiv_category(data, article)
    publishers = _create_publisher(data)
    _create_publication_info(data, article, publishers)
    _create_experimental_collaborations(data)
    authors = _create_author(data, article)
    _create_author_identifier(data, authors)
    _create_affiliation(data, authors)
    return article

https://github.com/SCOAP3/scoap3/blob/main/scoap3/tasks.py#L286-L300

If the record does not have all of the required fields, it will crash on this function with an error:

400 Client Error: Bad Request for url: https://backend.dev.scoap3.org/api/article-workflow-import/; 2552

The error is not informative! We can:

  1. The flag "Required" on for the fields that are needed in order to create a record. The error will be straightforward, which value is missing
  2. Improve the Error handling in the backend, with clear Error messages