Police-Data-Accessibility-Project / data-sources-app

An API and UI for using and maintaining the Data Sources database
MIT License
4 stars 5 forks source link

Streamline Archives `PUT` request data retrieval #361

Open maxachis opened 4 months ago

maxachis commented 4 months ago

The PUT method for the /archives endpoint has some odd behaviors that seem to be causing issues with data retrieval.

    def put(self) -> Dict[str, str]:
        #...
        json_data = request.get_json()
        data = json.loads(json_data)
        id = data["id"] if "id" in data else None
        last_cached = data["last_cached"] if "last_cached" in data else None
        broken_as_of = (
            data["broken_source_url_as_of"]
            if "broken_source_url_as_of" in data
            else None
        )

        with self.setup_database_client() as db_client:
            response = update_archives_data(db_client, id, last_cached, broken_as_of)

This requires the dictionary entry to be "dumped" beforehand, as in update_pdap_archives in cache_url.py of the pdap-automatic-archives repository:

def update_pdap_archives(entry: dict):
    """
    Update data in PDAP archives
    :param entry:
    :return:
    """
    entry_json = json.dumps(entry)
    response = requests.put(
        f"{os.getenv('VITE_VUE_APP_BASE_URL')}/archives",
        json=entry_json,
        headers={"Authorization": API_KEY},
    )
    response.raise_for_status()
``

Other resources don't require this. For example, in the `PUT` method for the `DataSourceById` resource class in `DataSources.py`:

```python
    def put(self, data_source_id: str) -> Response:
        # ...
        data = request.get_json()
        with self.setup_database_client() as db_client:
            return update_data_source_wrapper(db_client, data, data_source_id)

And it is thus easier to make a response, as seen in test_data_sources_by_id.py in tests/integration:

    response = client_with_db.put(
        f"/api/data-sources-by-id/SOURCE_UID_1",
        headers={"Authorization": f"Bearer {api_key}"},
        json={"description": desc},
    )

By contrast, if you attempt to do something like this with the archives PUT method, you'll get the error: the JSON object must be str, bytes or bytearray, not dict.

So the Archives PUT method appears to require extra steps which are immediately reversed in the backend. This makes request-making more difficult and makes documentation more challenging: currently, the Gitbook documentation doesn't mention this step.

TODO

maxachis commented 1 week ago

Marking as post-v2 just because it'll be easier once we're in that phase