search

philipmulcahy / azad

amazon order history reporter chrome extension
Apache License 2.0
216 stars 72 forks source link

Items prices (item report) not showing up. Are they supposed to? (Amazon.ca) #162

Closed hgeorges closed 2 years ago

hgeorges commented 3 years ago

Hi, Just came across your tool, looking for a way to summarize & identify my purchases for tax exemption purposes. Amazon.ca doesn't have a consolidated downloadable report - so the task is entirely manual and tedious. Your tool did the job to grab all items, quite impressive! Thank you for making it available! The price column is showing however 0 for most of the items. In a list of 20-30 items, a couple only were showing some numbers. Is it something special to be done for getting those, or is that info rather unavailable (deconstructing reliably an order might be not possible)? Thank you

philipmulcahy commented 3 years ago

Please send me at least a couple of order dump json files as described in README.md and tell me which one(s) are being correctly parsed. Don’t post those files on this ticket! I am pretty overloaded at work at the moment so it might be a while before I get time to work on it, but multiple users are reporting something like this.

Yours,

Philip

On 4 Jun 2021, at 19:19, hgeorges @.***> wrote:

 Hi, Just came across your tool, looking for a way to summarize & identify my purchases for tax exemption purposes. Amazon.ca doesn't have a consolidated downloadable report - so the task is entirely manual and tedious. Your tool did the job to grab all items, quite impressive! Thank you for making it available! The price column is showing however 0 for most of the items. In a list of 20-30 items, a couple only were showing some numbers. Is it something special to be done for getting those, or is that info rather unavailable (deconstructing reliably an order might be not possible)? Thank you

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

philipmulcahy commented 3 years ago

Hi hgeorges,

I've got your email - thank you.

What I think I need to debug this (naturally it's showing prices for my amazon account - grr) is an order dump json file from one of your orders whose items are not being priced.

The procedures for getting the file are here: https://github.com/philipmulcahy/azad#how-to-save-and-send-an-order-debug-json-file DON'T attach it directly to the ticket unless you want the internet to see it. Within the json file is a frozen copy of the html amazon serves for that order, and will enable me to step through the scraping code as it parses the html, and test improvements.

Yours,

Philip

10REMSSeiller commented 2 years ago

I may have a similar issue – except it is with US Amazon.

All digital orders have item price, but all non-digital orders have 0 price.

Cleared cache, restarted Chrome, re-logged in.

Please let me know if you want me to file a new issue and what other info you need.

Thanks!

EricM81 commented 2 years ago

The bug is with "show items not orders"

philipmulcahy commented 2 years ago

Hi @hgeorges,

1) Sorry for being so massively unresponsive - I can see you wrote to me in June reporting that the order dump feature wasn't working for you.

2) I think that plan A is still to see if we can get a dump file. Other users have observed that the failure to produce a dump file can sometimes be resolved by clearing the azad cache and making sure that you're logged into Amazon - the dump feature can sometimes get stuck when the table you're clicking on is being served out of the local azad cache. Also, as @EricM81 suggests, maybe switching back to the "row per order" view may allow you to capture the dump file. So long as you get me a file for an order whose items are impacted in the items table we should be good - the underlying data is from the same html.

3) Plan B is to see if anyone else can deliver a viable dump file for an order that's not displaying the item prices @EricM81, @10REMSSeiller might be able to help us. Remember to email any json or html to the address in https://github.com/philipmulcahy/azad#how-to-save-and-send-an-order-debug-json-file, and not post them directly in this ticket. Please include this ticket's link in the subject so I can find it, and report that you've done it here - the gmail account is deluged by ridiculous requests, and proposals to buy the extension (presumably to spread malware), so I don't look at email there unless I am directed to it by a post on a github ticket.

EricM81 commented 2 years ago

@philipmulcahy , I just sent an email with dump and screenshots. Used this thread for the subject line:

image

philipmulcahy commented 2 years ago

Got it - thanks Eric!

philipmulcahy commented 2 years ago

I have a reproduction of the problem. It looks like I need to spend some quality time with xpath. I will report back when that's happened.

philipmulcahy commented 2 years ago

I've got what I believe to be some improved xpath, but am getting confused by url matching in the test code that may be indicative of something ugly going on in the caching code - I feel confident this is solvable when I have more brainpower.

philipmulcahy commented 2 years ago

@EricM81 @hgeorges azad_d0fc475b.zip might contain a fix for the display problem (not the broken debug menu item in items view). Would you mind taking it for a spin per how-to-try-out-an-experimental-version and letting me know how you get on?

philipmulcahy commented 2 years ago

hmm - I think azad_d0fc475b.zip might have regressed on digital items - I'd appreciate feedback on the non-digital items that were previously broken. This is like one of those tippy puzzles where you have to get all the little silver balls into their holes at the same time.

JGrantman commented 2 years ago

Long story short - the updated zip file posted above works for me. Details of what I experienced are provided below.

I had a very similar issue (with price information not displaying when "show items not orders" is selected). However, for me the price shows as 0 for all normal Amazon items, but the price does display for Amazon Fresh & Whole Foods orders. Example: For year 2021 I have 623 items - 56 items with price and 567 items with 0 for price. The price does show correctly for all orders when "show items not orders" is NOT selected.

Before coming here to post I did some troubleshooting/testing to rule out anything on my end:

HOWEVER... I saw the zip file posted above and installed it (and disabled regular Chrome store azad addon) and it appears to work. First run brought back price for every item and raw count of items is same for each of the years I have been using to compare results.

So I came to post for help but ended up posting to confirm the fix you proposed seems to work (for me). I will continue to use the fix version until you have fix read to propagate to Chrome Store.

THANK YOU for all the work. Chrome store is full of so much garbage and copy cat versions of minimally functional addons. This is one of 4-5 truly useful ones that solves a problem that is very difficult to solve -- extracting quality Amazon history.

FYI... I already have a 2021 dump connected (Excel) to the Microsoft Money in Excel sheet so I can automate categorization per items from the azad data to calculate splits on large multi-item Amazon orders that show up from my bank as a single large amount. ... and another copy of the data in a database for reconciling returns processing and deriving true history for items I purchase repeatedly. It has already helped identify several items with hugely variable price changes which should help target purchase timing when pricing is low. HUGE HUGE HUGE help having this addon instead of pulling the generic csv exports of order data and manually estimating prices for items on large multi-item orders.

THANK YOU!

EricM81 commented 2 years ago

I found a few problems that I've emailed you the details. I cannot download the order debug info for either the order or order item on these, so I'm including the CSV output for each plus the HTML of either the order invoice or order detail page.

The first issue is when using rewards points as payment. The order shows as if it cost nothing, but items include the price.

The second issue is with refunded items. They still show on the items page. I included a corner case that uses both rewards points and a partial refund.

philipmulcahy commented 2 years ago

azad_8225f1e7.zip fixes digital order item scraping (according to all the test examples I've got).

@EricM81 on "The first issue is when using rewards points as payment. The order shows as if it cost nothing, but items include the price.", I'm wondering if that isn't reasonable behaviour.

Unless we can reliably capture the json files, stuff we fix doesn't necessarily stay fixed, because we don't have a regression test. I've made a bug for this: #179

EricM81 commented 2 years ago

I'm wondering if that isn't reasonable behaviour.

If someone wants order totals to be zero for rewards purchases, then they have the option of filtering based on the 'payments' column. On the other hand, if someone needs order totals (expense reports for reimbursement in my case), they have no option.

I would prefer the order total, shipping, tax, etc be reported as the actual amount, and the payments reflect whether it was a credit card, gift card, rewards points, etc.

philipmulcahy commented 2 years ago

I think @EricM81's https://github.com/philipmulcahy/azad/issues/162#issuecomment-1012317213 and https://github.com/philipmulcahy/azad/issues/162#issuecomment-1013249189 are worthy of another ticket. On the basis no-one's claiming missing items from the original set, I'm going to merge to master. I'll close this ticket when I push a new version to the web store (unless someone gives me another example that fails on azad_8225f1e7.zip).

philipmulcahy commented 2 years ago

fixed in 1.9.23