dgunning / edgartools

Navigate SEC Edgar data in Python
MIT License
445 stars 87 forks source link

Balance Sheet #58

Closed jdy214 closed 3 months ago

jdy214 commented 4 months ago

Similar issue experiences same items are missing from balance sheet. Otherwise very intuitive design

jdy214 commented 4 months ago

Marketing Expense missing from Income Statement.

dgunning commented 4 months ago

Fixing the balance sheet now.

I don't see Marketing Expenses here .. do you mean Sales, General and Administrative Expenses?

apple-incomestatement
Deffro commented 4 months ago

Indeed, Marketing Expenses are not missing, and refer to the Income Statement.

The Income Statement is only missing:

If you would like to look into the Balance Sheet, here is what is missing:

For completeness, the Cashfow Statement is only missing:

jdy214 commented 4 months ago

Hi Dwight,

Thank you for following up. I may have not been clear on the post, there is an issue with the balance sheet and then a sperate issue with the Marketing Expenses which appears on the income statement problem. You see it for instance if you look at AAPL or MBLY, for two examples. In the AAPL example Marketing shows as a line item, even though it does not appear as an item on AAPL's income statement. For Mobile Eye (MBLY) it shows as a line item but does not seem to be picking up any values. This happens with a few other of the income statement items. Rather than reading the file and retrieving the line item titles, its seems that it is searching for pre-defined line items like :Marketing expense".

My knowledge of coding is limited, but if there is any way I can help improve edgar tools, I would be happy to help.

Very Truly, Justin

On Mon, Jun 3, 2024 at 8:13 AM Dwight Gunning @.***> wrote:

Fixing the balance sheet now.

I don't see Marketing Expenses here .. do you mean Sales, General and Administrative Expenses? apple-incomestatement.png (view on web) https://github.com/dgunning/edgartools/assets/1523271/b596233c-498a-412d-a921-651cf64ea039

— Reply to this email directly, view it on GitHub https://github.com/dgunning/edgartools/issues/58#issuecomment-2145038295, or unsubscribe https://github.com/notifications/unsubscribe-auth/BIZJ6LL3W3SSX2ZAKFT7Z23ZFRMVLAVCNFSM6AAAAABINUOKQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBVGAZTQMRZGU . You are receiving this because you authored the thread.Message ID: @.***>

dgunning commented 4 months ago

You are right .. it could be enhanced to not show empty line items.

Thanks, working on it this week

jdy214 commented 4 months ago

For AAPL the values is empty, however for MBLY there are values there it is not picking them up

Justin

On Mon, Jun 3, 2024 at 12:05 PM Dwight Gunning @.***> wrote:

You are right .. it could be enhanced to not show empty line items.

Thanks, working on it this week

— Reply to this email directly, view it on GitHub https://github.com/dgunning/edgartools/issues/58#issuecomment-2145595933, or unsubscribe https://github.com/notifications/unsubscribe-auth/BIZJ6LNEAPCEYBX3KPKKFSTZFSH5JAVCNFSM6AAAAABINUOKQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBVGU4TKOJTGM . You are receiving this because you authored the thread.Message ID: @.***>

dgunning commented 4 months ago

Hi Justin,

I uploaded a screenshot of the financial statements for Apple along with the xbrl and asked ChatGPT to generate the mappings here

https://github.com/dgunning/edgartools/tree/main/data/financials

Take a look, maybe you can see if they are correct and would work for other companies

jdy214 commented 4 months ago

Sure I will have a look at it tonight!

This could really be a stellar alternative to the sec's official scraping platform.

Justin

On Mon, Jun 3, 2024, 5:09 PM Dwight Gunning @.***> wrote:

Hi Justin,

I uploaded a screenshot of the financial statements for Apple along with the xbrl and asked ChatGPT to generate the mappings here

https://github.com/dgunning/edgartools/tree/main/data/financials

Take a look, maybe you can see if they are correct and would work for other companies

— Reply to this email directly, view it on GitHub https://github.com/dgunning/edgartools/issues/58#issuecomment-2146128592, or unsubscribe https://github.com/notifications/unsubscribe-auth/BIZJ6LJHUMXZ4P6M4QZSUNTZFTLSNAVCNFSM6AAAAABINUOKQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBWGEZDQNJZGI . You are receiving this because you authored the thread.Message ID: @.***>

jdy214 commented 3 months ago

Hi Dwight,

THe issue seems to be persisting. with the company MBLY, it is not picking up the Marketing Expenses.

Very Truly, Justin

On Mon, Jun 3, 2024, 5:09 PM Dwight Gunning @.***> wrote:

Hi Justin,

I uploaded a screenshot of the financial statements for Apple along with the xbrl and asked ChatGPT to generate the mappings here

https://github.com/dgunning/edgartools/tree/main/data/financials

Take a look, maybe you can see if they are correct and would work for other companies

— Reply to this email directly, view it on GitHub https://github.com/dgunning/edgartools/issues/58#issuecomment-2146128592, or unsubscribe https://github.com/notifications/unsubscribe-auth/BIZJ6LJHUMXZ4P6M4QZSUNTZFTLSNAVCNFSM6AAAAABINUOKQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBWGEZDQNJZGI . You are receiving this because you authored the thread.Message ID: @.***>

jdy214 commented 3 months ago

Hi Dwight,

I see the issue is that there are simply too many variations of the contextRef used in the xml files. So except for the subtotal the companies do not use the same references for the line items.

Do you think there is an efficient way to recategorize groups of the contextRef under a more general line item name? I believe this is how CapitalIQ does it on their end.

Justin

On Fri, Jun 7, 2024, 11:14 AM Justin Yaroni @.***> wrote:

Hi Dwight,

THe issue seems to be persisting. with the company MBLY, it is not picking up the Marketing Expenses.

Very Truly, Justin

On Mon, Jun 3, 2024, 5:09 PM Dwight Gunning @.***> wrote:

Hi Justin,

I uploaded a screenshot of the financial statements for Apple along with the xbrl and asked ChatGPT to generate the mappings here

https://github.com/dgunning/edgartools/tree/main/data/financials

Take a look, maybe you can see if they are correct and would work for other companies

— Reply to this email directly, view it on GitHub https://github.com/dgunning/edgartools/issues/58#issuecomment-2146128592, or unsubscribe https://github.com/notifications/unsubscribe-auth/BIZJ6LJHUMXZ4P6M4QZSUNTZFTLSNAVCNFSM6AAAAABINUOKQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBWGEZDQNJZGI . You are receiving this because you authored the thread.Message ID: @.***>

dgunning commented 3 months ago

That's a related issue but not the core one.

The missing data is mostly there for MBLY in the default context but they weren't mapped because the reference companies I used initially are different company types and didn't have those facts. Companies can choose to map data to different fact names.

I'm making progress today on the core issue and should have a release coming out of this weekend.

The second issue is as you say the context issue which has company specific data. I'm working on a way to map these and thanks for the suggestion I'll look into it.

Dwight

dgunning commented 3 months ago

Release 2.24.1 with fixes for missing balance sheet and income statement items.

jdy214 commented 3 months ago

Just tried it looks great, can I ask conceptually how did you resolve the issue?

I ran through some solutions myself using chatGPT, but I couldnt figure out how do you determine which references belong to the income statement, (versus income statement notes or some alternate source in the xml file)? What marker or tag lets you know that a certain fact belong to the income statement vs. balance sheet vs. the rest of the file, where other items have xbrl tags as well?

Very Truly, Justin

On Sun, Jun 9, 2024 at 1:07 PM Dwight Gunning @.***> wrote:

Closed #58 https://github.com/dgunning/edgartools/issues/58 as completed.

— Reply to this email directly, view it on GitHub https://github.com/dgunning/edgartools/issues/58#event-13092054439, or unsubscribe https://github.com/notifications/unsubscribe-auth/BIZJ6LMAP3T4HPYQRFQTEADZGSDUPAVCNFSM6AAAAABINUOKQ2VHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJTGA4TEMBVGQ2DGOI . You are receiving this because you authored the thread.Message ID: @.***>

dgunning commented 3 months ago

I uploaded a screenshot of the financials on the sec site, and the XBRL facts csv to ChatGPT and asked it to produce a mapping

jdy214 commented 1 month ago

Hello,

I noticed there was an update for edgar. Now that I have updated, I am unable to import the module, seems to be an issue with accessing the xbrl portion?

Do you have any solutions to suggest to resolve?

Very Truly, Justin

On Sun, Jun 9, 2024 at 9:23 PM Dwight Gunning @.***> wrote:

I uploaded a screenshot of the financials on the sec site, and the XBRL facts csv to ChatGPT and asked it to produce a mapping

— Reply to this email directly, view it on GitHub https://github.com/dgunning/edgartools/issues/58#issuecomment-2157010566, or unsubscribe https://github.com/notifications/unsubscribe-auth/BIZJ6LL74CA4C4RNHW2HM3LZGT5XNAVCNFSM6AAAAABINUOKQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJXGAYTANJWGY . You are receiving this because you authored the thread.Message ID: @.***>