Rewrite Met Museum Provider API Script using new ImageStore class

[mathemancer]

The old version can be found at src/cc_catalog_airflow/dags/provider_api_scripts/MetMuseum.py. The script should be rewritten with the following goals:

• The new script should be at src/cc_catalog_airflow/dags/provider_api_scripts/metropolitan_museum_of_art.py (note the capitalization change from the old script name).
• The new script should have a test suite at src/cc_catalog_airflow/dags/provider_api_scripts/test_metropolitan_museum_of_art.py.
• The new script must use the new ImageStore class (Import this from src/cc_catalog_airflow/dags/provider_api_scripts/common/storage/image.py).
• The new script should use the new DelayedRequester class (Import this from src/cc_catalog_airflow/dags/provider_api_scripts/common/requester.py).
• The new script must not use anything from src/cc_catalog_airflow/dags/provider_api_scripts/modules/etlMods.py. (That module is deprecated)
• The new script must take a --date parameter when run as a script, giving the date for which we should collect images. The form should be YYYY-MM-DD (so, the script can be run via python thingiverse.py --date 2018-01-01).
• The new script must provide a main function that takes a date parameter of the same form, and does the exact same thing that running the script from the CLI with the date parameter would do.
• The new script should conform to PEP8. Please use pycodestyle (available via pip install pycodestyle) to check for compliance.
• The new script should use smaller, more testable functions than the old version.
• The test suite for the new script may break PEP8 rules regarding long lines where appropriate (e.g., long strings for testing).
• For an example of how all this should look, see the new Wikimedia Commons script at src/cc_catalog_airflow/dags/provider_api_scripts/wikimedia_commons.py.
• Also see the Wikimedia Commons Provider API Script test suite at src/cc_catalog_airflow/dags/provider_api_scripts/test_wikimedia_commons.py.

[ayanchoudhary]

Is this PR ready to be worked on? If yes, can I take this?

[Gbahdeyboh]

@AyanChoudhary I don't think anyone is currently working on this issue. You could take it and send a PR, but note that the PR wouldn't be merged if tests are not written.

[akshgpt7]

@gauravahlawat81 Regarding #277, the Contributing Guidelines say the following:

Once you have identified an issue you'd like to work on, follow these steps:

• Comment on it and say you'd like to work on that issue.
• Wait for someone to confirm that you may work on the issue before writing any code. The person who confirms will add an "in progress" label to the issue to indicate that the issue has been assigned.
• Once the issue has been updated to "in progress", write your code and submit your pull request (be sure to read and follow our pull request guidelines!)
• Wait for code review and address any issues raised as soon as you can.

(Kindly go through the page once.)

[ayanchoudhary]

I had started working on this as soon as I was given the go.

[akshgpt7]

You need the approval of a maintainer, who will label the issue as in progress before starting work.

[ayanchoudhary]

@AyanChoudhary I don't think anyone is currently working on this issue. You could take it and send a PR, but note that the PR wouldn't be merged if tests are not written.

Actually he did approve it

[gauravahlawat81]

Oh, @akshgpt7 Thanks for that. But as the issue wasn't marked In progress, I took the Liberty of sending a WIP PR, since I didn't see one. If you guys think that I broke rules here, I would happily delete the PR .

[gauravahlawat81]

I've deleted the PR. @AyanChoudhary Go ahead and work on it if you had already started.

[mathemancer]

The met has just updated their API. The new response json is:

{

    "objectID": 45734,

    "isHighlight": false,

    "accessionNumber": "36.100.45",

    "accessionYear": "1936",

    "isPublicDomain": true,

    "primaryImage": "https://images.metmuseum.org/CRDImages/as/original/DP251139.jpg",

    "primaryImageSmall": "https://images.metmuseum.org/CRDImages/as/web-large/DP251139.jpg",

    "additionalImages": [

        "https://images.metmuseum.org/CRDImages/as/original/DP251138.jpg",

        "https://images.metmuseum.org/CRDImages/as/original/DP251120.jpg"

    ],

    "constituents": [

        {

            "role": "Artist",

            "name": "Kiyohara Yukinobu",

            "constituentULAN_URL": "http://vocab.getty.edu/page/ulan/500034433",

            "constituentWikidata_URL": "https://www.wikidata.org/wiki/Q11560527",

            "gender": "Female"

        }

    ],

    "department": "Asian Art",

    "objectName": "Hanging scroll",

    "title": "Quail and Millet",

    "culture": "Japan",

    "period": "Edo period (1615–1868)",

    "dynasty": "",

    "reign": "",

    "portfolio": "",

    "artistRole": "Artist",

    "artistPrefix": "",

    "artistDisplayName": "Kiyohara Yukinobu",

    "artistDisplayBio": "Japanese, 1643–1682",

    "artistSuffix": "",

    "artistAlphaSort": "Kiyohara Yukinobu",

    "artistNationality": "Japanese",

    "artistBeginDate": "1643",

    "artistEndDate": "1682",

    "artistGender": "Female",

    "artistWikidata_URL": "https://www.wikidata.org/wiki/Q11560527",

    "artistULAN_URL": "http://vocab.getty.edu/page/ulan/500034433",

    "objectDate": "late 17th century",

    "objectBeginDate": 1667,

    "objectEndDate": 1682,

    "medium": "Hanging scroll; ink and color on silk",

    "dimensions": "46 5/8 x 18 3/4 in. (118.4 x 47.6 cm)",

    "creditLine": "The Howard Mansfield Collection, Purchase, Rogers Fund, 1936",

    "geographyType": "",

    "city": "",

    "state": "",

    "county": "",

    "country": "",

    "region": "",

    "subregion": "",

    "locale": "",

    "locus": "",

    "excavation": "",

    "river": "",

    "classification": "Paintings",

    "rightsAndReproduction": "",

    "linkResource": "",

    "metadataDate": "2020-02-07T15:50:59.677Z",

    "repository": "Metropolitan Museum of Art, New York, NY",

    "objectURL": "https://wwwstg.metmuseum.org/art/collection/search/45734",

    "tags": [

        {

            "term": "Birds",

            "AAT_URL": "http://vocab.getty.edu/page/aat/300266506"

        },

        {

            "term": "Leaves",

            "AAT_URL": "http://vocab.getty.edu/page/aat/300011892"

        }

    ],

    "objectWikidata_URL": "https://www.wikidata.org/wiki/Q29910832",

    "isTimelineWork": false

}

[mathemancer]

I don't think this will affect anything about the script, but we should use this response json (or subobjects of it) in testing

[Nikitag0329IITkgp]

Is this issue still open to working on?

[ayanchoudhary]

The pr concerning this is complete.

[akshgpt7]

I think @AyanChoudhary has a linked PR to this and is currently working on it (#278).

[mathemancer]

This is correct, there's a PR for this currently.