search

FDA / openfda

openFDA is a research project to provide open APIs, raw data downloads, documentation and examples, and a developer community for an important collection of FDA public datasets.
https://open.fda.gov
Creative Commons Zero v1.0 Universal
567 stars 131 forks source link

Maude-Device-Events #198

Open JI-Michael opened 10 months ago

JI-Michael commented 10 months ago

Hi,

This isn't a bug but rather a question.

We'd like to utilize the openFDA API once again to access the Maude database data. However, in the past, we noticed that the data through the API was updated infrequently. Refer to the issue: https://github.com/FDA/openfda/issues/187.

Because of this, we have been obtaining the data directly from the provided Zip files here: https://www.fda.gov/medical-devices/mandatory-reporting-requirements-manufacturers-importers-and-device-user-facilities/about-manufacturer-and-user-facility-device-experience-maude.

Yet, since the openFDA data is better structured, we'd prefer to return to using the API.

With this in mind, has there been an increase in the frequency of updates for the Maude data via the API? We've typically found new data every month through the Zip files mentioned above.

Furthermore, all maude data points are relevant to us. So, another question arises: do we need to download/process all the maude data with each update (see: https://open.fda.gov/apis/downloads/) or is it sufficient to only process the latest data?

I'm asking in relation to this issue: https://github.com/FDA/openfda/issues/176

best regards, Michael

dkrylovsb commented 10 months ago

Hello,

openFDA updates the Device Event endpoint on a weekly basis, and it does so by processing the MAUDE data files, which you linked above. Even though the updates are done weekly, if the MAUDE data on the FDA website has not changed, the Device Event endpoint won't change, either.

Long story short, one can expect MAUDE updates to appear in openFDA no later than a week after the updates have been posted in the MAUDE data files.

And yes, unfortunately, one would have to download the entire dataset each time because openFDA currently has no way to provide incremental updates.

martinleo commented 3 months ago

Hi @dkrylovsb Given the limit of 1000 records per request, the limit of 1000 request a day, and the inability to incrementally load from Events, I just wanted to ask you if you know a more efficient way of getting Event data without an API key. I was thinking of using date_received as a watermark for incremental updates. Do you think date_received can change in a given record from one moment to another? Thanks in advance

dkrylovsb commented 3 months ago

Hi @martinleo curious why not just obtain an API key?