search

FDA / open.fda.gov

openFDA web site.
http://open.fda.gov
MIT License
160 stars 73 forks source link

Drugs@FDA data set is inconsistent #261

Open sho13 opened 3 months ago

sho13 commented 3 months ago

Hello!

Just attempted to search for drugs that were previously existing on the Open FDA API for Drugs -

Drugs@FDA [/drug/drugsfda]
This endpoint's data may be downloaded in zipped JSON files. Records are represented in the same format as API calls to this endpoint. Each update to the data in this endpoint could change old records. You need to download all the files to ensure you have a complete and up-to-date dataset, not just the newest files. For more information about openFDA downloads, see the API basics.

There are 1 files, last updated on 2024-07-16.

As you can see it's been updated today on July 16th, 2024, but there's inconsistencies with the accessdata website

For example - Eylea HD and Beovu are nonexistent in the API but available on the accessdata website

Is this discrepancy a bug?

Thanks!

ketkijsane commented 3 months ago

openFDA and AccessData do not share a release cycle, so there may be differences in data between the two.

outstandy commented 3 months ago

@ketkijsane the new release of openFDA is now missing data. We've found at least 4 drugs that were present in the previous release that are not now, and none of them have had a change in their approval or marketing status as far as we can tell. Do you know why they might have been removed?

ketkijsane commented 3 months ago

Thank you @outstandy for your response. Can you please provide more information about what endpoint you are hitting and which specific drugs you have found that are missing? Thanks!

sho13 commented 3 months ago

Hi @ketkijsane the endpoints that have been hit are - the https://api.fda.gov/drug/drugsfda.json?search=openfda.brand_name end point and the drugs are -

Eylea, Beovu, Vabysmo and Zolgensma. Eylea previously had two results (Eylea and Eylea HD) Beovu, Vabysmo and Zolgensma all previously had a single result

Again is this discrepancy a bug or have these drugs been pulled?

ketkijsane commented 3 months ago

Thank you @sho13 for your response. We've investigated the raw data files which serve as our data source and found that the data in openFDA matches what was most recently released: https://www.fda.gov/drugs/drug-approvals-and-databases/drugsfda-data-files. Please let us know if this looks good. Thanks!

sho13 commented 3 months ago

hey @ketkijsane thank you for investigating! I can confirm your investigation of raw data files and data are aligned, however the confusion is that it's missing drugs from the Purple Book (if you search for all the drugs mentioned above on purple book, they are still present),

Do you know if the FDA has pulled these from the data set?

Thanks!

sho13 commented 3 months ago

@ketkijsane a follow up, in the raw data set I was able to find the SubmissionPropertyType and Submissions via Application number for the drug Beovu - 761125, there is not Product for it though, the other three drugs in question only Eylea existed but not the High dose version.

ketkijsane commented 3 months ago

@sho13 , thank you for the confirmation. Unfortunately, we don't have any information on how the FDA puts together the source data files. The Drugs@FDA team would know better for that data. Not having a listing in the product source file is why there's no results for "brand_name", as that field is populated from the product file. Please let us know if this sounds good. Thanks!

sho13 commented 3 months ago

@ketkijsane thanks, I would love to reach out to the appropriate team- do you know the best way to reach the Drugs@FDA team?

ketkijsane commented 3 months ago

https://www.fda.gov/drugs/drug-approvals-and-databases/drugsfda-data-files has a link at the top with an email address.

Thanks!

sho13 commented 3 months ago

Thank you @ketkijsane , but that link is an a tag 

<a href="mailto:?subject=Contact%20FDA&amp;body=https://www.fda.gov%2Fabout-fda%2Fcontact-fda" class="lcds-share__btn lcds-share--default__btn-mail"><span class="fa icon-envelope" aria-hidden="true"></span>Email</a>

That doesn't send an email to anyone in Drugs@FDA, it fills my subject and body.

Is there any direct contact you can provide me with, thank!

ketkijsane commented 3 months ago

@sho13, we're looking for contact info now and will let you know as soon as we have it. Please note that there may be a delay due to the global tech outage. Apologies for any inconvenience.

Thanks!

sho13 commented 2 months ago

Hello @ketkijsane i see that the dataset's been updated yesterday but it's still missing the drugs as mentioned... would you know if this issue has been resolved and these drugs are just pulled permanently from the FDA?

ketkijsane commented 2 months ago

Hello @ketkijsane i see that the dataset's been updated yesterday but it's still missing the drugs as mentioned... would you know if this issue has been resolved and these drugs are just pulled permanently from the FDA?

Hi @sho13, that would also be a question for the Drugs@FDA people since we had no knowledge about inclusion of specific results in the source data. We are still looking for a contact information and will let you know as soon as we have it.

Thank you!