search

madmachinations / home-assistant-alexa-shopping-list

A custom component for Home Assistant which synchronises your Alexa Shopping List
GNU General Public License v3.0
23 stars 2 forks source link

API list closure #7

Closed jouster1974 closed 3 days ago

jouster1974 commented 2 months ago

Will this integration stop working as off July 1st like all other list api accessing skills? I assume as such so wondering what if any workarounds there might be

madmachinations commented 2 months ago

Yep, this will stop working at that point unfortunately.

I have some ideas for how to get around the problem, but I need to see what still works and what doesn't after July 1st.

bwims commented 2 months ago

Bump. Any news? It's hard to live without a shopping list, and the Alexa App is pure bloatware.

Thanks!

madmachinations commented 2 months ago

So far the news is not looking good tbh.

I believe I know a way to make it work, but it's not a quick fix and has other problems. Essentially I'd have to sniff the alexa app requests which cover authentication and list management, and make the integration pretend to be the alexa app.

There used to be a web version of the alexa app, and I could have used it's API in the same way, but easier because you can just see the requests in any browser. But sadly they have also shut down their web app.

Sniffing the requests and replicating them should work. But, even if I did this, technically that's a "private" API which the terms will state I'm not allowed to use for any purpose amazon haven't said I can. So the fact I can get the information I need to make it work, doesn't mean Amazon would be okay with that. They would be within their rights to insist I take it down.

I am contemplating it still, but even so, between other things and depending on how much of a fight it gives me; It could be anything from a couple of weeks up to 3 months to have a working solution working on it by myself.

Todoist have given up on alexa lists full stop, they don't have any intentions to introduce even custom commands to alexa to add stuff to todoist lists specifically, so that's not even a plan B.

I've heard amazon are "overhauling lists" but that isn't anywhere near ready, and everything else has been cut off in the mean time. There's no guarantee they will allow integrations into whatever their next version of lists is. But the fact that they've cut everyone off suggests heavily they won't, they want people to have to open the alexa app to see their lists, probably so they can try and advertise where you can buy all that stuff from, or just to make it look like the alexa app is more liked and used than it really is.

It's an open secret that financially alexa and all the commerical smart speaker assistants are not generating the revenue their parent companies hoped they would. So I'm expecting to see more of this kind of behaviour from all of them. They will want to pump it for every penny and make it profitable. There's also soon likely to be the introduction of a subscription fee to access "premium" alexa features. Cutting off list API's could also be a step towards that, where you can have API access if you pay the subscription. But nobody knows for sure until something official is announced.

So, the spiteful goblin in me is saying I should seriously consider abandoning all the echos I have in the house and replacing them with a home grown solution, setting up satellites using HA voice.

I am still deciding what to do, but I will update when I've comitted to a direction.

madmachinations commented 2 months ago

There is another solution to making it work, but it's not pretty and would require installing an android simulator on the machine running HA, or at least on the same network.

I could make that work, but I don't consider it a good solution because that's quite a fat amount of software to run constantly for the sake of some lists. I could do it on my HA setup because I have a decent rig running it all. But people running it on yellows and pi's and such, they couldn't run something that heavy.

I'll have a definitive answer of what's happening in the next 2 weeks though

bwims commented 2 months ago

Thanks for the update. Amazon seem to want people to hate their stuff, don't they? The men in grey suits just cannot relate to the real world.

What I really need is a company to develop a hardware HA voice assistant that looks like the old echo dot (v3) That would supply my needs.

kellerassel007 commented 2 months ago

Looking forward for a new working solution!!

bwims commented 2 months ago

This might affect your efforts... It is no longer possible to delete completed items from the shopping list either by the Alexa app or the desktop web browser interface.

madmachinations commented 2 months ago

This might affect your efforts... It is no longer possible to delete completed items from the shopping list either by the Alexa app or the desktop web browser interface.

You have access to the desktop web interface? What URL are you using to access that?

bwims commented 2 months ago

sorry, I meant the web browser interface to Amazon. You hover over account & lists and click on "shopping list".

On Thu, 11 Jul 2024 at 13:26, Madmachinations @.***> wrote:

This might affect your efforts... It is no longer possible to delete completed items from the shopping list either by the Alexa app or the desktop web browser interface.

You have access to the desktop web interface? What URL are you using to access that?

— Reply to this email directly, view it on GitHub https://github.com/madmachinations/home-assistant-todoist-shopping-list/issues/7#issuecomment-2222802014, or unsubscribe https://github.com/notifications/unsubscribe-auth/AENIQ2LPVDN2SW77MKFW5QTZLZ2W7AVCNFSM6AAAAABKDSA2ESVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRSHAYDEMBRGQ . You are receiving this because you commented.Message ID: <madmachinations/home-assistant-todoist-shopping-list/issues/7/2222802014@ github.com>

madmachinations commented 2 months ago

Wow, why didn't I think of that before. That is so scrape-able! Which means I might be able to make it work via the amazon website. I know there are ways to delete items on a shopping list in different places on the Amazon site.

I will play with this concept over the coming days. If it works, the replacement integration can just login to your amazon account, and then pretend to be a web browser.

But this is the most optimistic path I've seen so far to making this work again.

Cross your fingers!

madmachinations commented 3 weeks ago

@jouster1974 @kellerassel007 @bwims and anyone else who is here.

So I am close to a working solution, I had the prototype running on my system yesterday.

However I need to put some final details on it so it can work on different people's setups. As it requires some extra steps to install so it can do the do.

With that in mind, please can you let me know how you are running HA and on what kind of architecture.

For example, I am running HA within a container on an intel mac mini.

Just so I can be sure I've covered as many bases as I can think of.

Thanks!

kellerassel007 commented 3 weeks ago

@jouster1974 @kellerassel007 @bwims and anyone else who is here.

So I am close to a working solution, I had the prototype running on my system yesterday.

However I need to put some final details on it so it can work on different people's setups. As it requires some extra steps to install so it can do the do.

With that in mind, please can you let me know how you are running HA and on what kind of architecture.

For example, I am running HA within a container on an intel mac mini.

Just so I can be sure I've covered as many bases as I can think of.

Thanks!

HA running as container on Proxmox host.

madmachinations commented 3 weeks ago

@jouster1974 @kellerassel007 @bwims and anyone else who is here. So I am close to a working solution, I had the prototype running on my system yesterday. However I need to put some final details on it so it can work on different people's setups. As it requires some extra steps to install so it can do the do. With that in mind, please can you let me know how you are running HA and on what kind of architecture. For example, I am running HA within a container on an intel mac mini. Just so I can be sure I've covered as many bases as I can think of. Thanks!

HA running as container on Proxmox host.

Cheers @kellerassel007, never heard of that haha, but I get the gist. Are you running it on x86_64 or arm?

bwims commented 3 weeks ago

I'm running HA in Docker container mode, which allows HACS but not add-ons. It sits on a Raspberry Pi in headless mode.

madmachinations commented 3 weeks ago

I'm running HA in Docker container mode, which allows HACS but not add-ons. It sits on a Raspberry Pi in headless mode.

Cool thanks

RocketKuhr commented 2 weeks ago

@jouster1974 @kellerassel007 @bwims and anyone else who is here.

So I am close to a working solution, I had the prototype running on my system yesterday.

However I need to put some final details on it so it can work on different people's setups. As it requires some extra steps to install so it can do the do.

With that in mind, please can you let me know how you are running HA and on what kind of architecture.

For example, I am running HA within a container on an intel mac mini.

Just so I can be sure I've covered as many bases as I can think of.

Thanks!

Hi, I´m running on Intel Nuc as x86_64

madmachinations commented 2 weeks ago

@jouster1974 @kellerassel007 @bwims and anyone else who is here. So I am close to a working solution, I had the prototype running on my system yesterday. However I need to put some final details on it so it can work on different people's setups. As it requires some extra steps to install so it can do the do. With that in mind, please can you let me know how you are running HA and on what kind of architecture. For example, I am running HA within a container on an intel mac mini. Just so I can be sure I've covered as many bases as I can think of. Thanks!

Hi, I´m running on Intel Nuc as x86_64

Are you running HA in a container environment? Or are you running HASS OS? Or have you done a manual install?

RocketKuhr commented 2 weeks ago

@jouster1974 @kellerassel007 @bwims and anyone else who is here. So I am close to a working solution, I had the prototype running on my system yesterday. However I need to put some final details on it so it can work on different people's setups. As it requires some extra steps to install so it can do the do. With that in mind, please can you let me know how you are running HA and on what kind of architecture. For example, I am running HA within a container on an intel mac mini. Just so I can be sure I've covered as many bases as I can think of. Thanks!

Hi, I´m running on Intel Nuc as x86_64

Are you running HA in a container environment? Or are you running HASS OS? Or have you done a manual install?

I´m running HASS OS

madmachinations commented 2 weeks ago

Okidokey, well I've done a quick google and the results suggest you can run extra docker containers on HASS OS, but it is highly discouraged.

So just FYI guys, the new solution is more involved than the old solution.

It is a selenium based approach which logs into the amazon website with your account, it also supports the MFA code step. No login details are saved, it just logs in once with you there, and then that browser session is stored and persists.

If you don't know, selenium is a testing framework where you can essentially remote control a web browser. In this case the web browser is headless, so no window appears.

Once logged in I have written code which accesses the shopping list and can see what is on it, add things to it, or remove things from it.

The whole thing is broken into three different parts:

I have tried everything I could to make this as painless and simple as possible.

I tried using just bare aiohttp with amazon but their whole system is a shit show of nonsense, which feels on purpose tbh.

I tried making a selenium based solution which could be bundled and just installed as a normal component. But selenium and/or chromium and/or chromium's driver told me to go fuck myself in different environments no matter what I did.

So then I tried deploying a vanilla standalone selenium grid instance which the custom component could just connect into to use the browser real quick. But, for reasons unknown to science, the browser just kept crashing repeatedly for no discernible reason.

Which is how I ended up here, where I didn't want to be. With an elaborate and entirely custom solution.

BUT... it does work. I'm just warning you all of what the future tastes like if you want these particular candies all up in your jazz.

It should run on most rigs. Older raspberry pi's might struggle, but could run the server in a non-containerized way perhaps. The only heavy part is when it spins up a headless browser, however it only does this briefly and when it has to.

HASS OS I don't think will play well with it, but I don't know much about it tbh. If you're comfortable running containers and/or setting up something new in systemd - you will be fine.

I'm nearly done and will release it this week at some point. I will write elaborate setup instructions for the different approaches. I'll cover as many things as I can and make it as accessible as I can. to go with it and I'll be knocking around if anyone gets stuck, just open an issue and I'll respond asap.

SavageCore commented 2 weeks ago

That sounds good, will look forward to seeing it.

This was my selenium based effort: https://github.com/SavageCore/HA_Addons/tree/main/alexa_shopping_list_sync

madmachinations commented 2 weeks ago

That sounds good, will look forward to seeing it.

This was my selenium based effort: https://github.com/SavageCore/HA_Addons/tree/main/alexa_shopping_list_sync

Looks neatly written, I like it. My main concern with it is the storage of login details and especially then the storage of the MFA secret. I considered doing something like this also for the sake of simplicity but it feels like an unnecessary risk vs persisting the session once authenticated and keeping it alive.

bwims commented 2 weeks ago

Sadly, I am in the middle of moving house across country (the UK) so will not be able to test it for some considerable while. (Happily, though, I will be close to my new(ish) grandson! )

FYI I had an alternative solution going a while back, because I use Node Red for automations, and the hooks exist.

It uses a contribution called node-red-contrib-alexa-remote2-applestrudel which among other things allows you to manipulate the Alexa shopping list directly. When I detect activity, I check to see if it involves adding to the list (via three patterns of response) then I take the details and use @foxleigh81/node-red-contrib-todoist-api to add it to todoist. I then delete the item off the shopping list.

Clunky but it works (for now).

B.

madmachinations commented 3 days ago

@SavageCore @jouster1974 @kellerassel007 @bwims

Hello everyone,

I have pushed the latest version, which I have had running on my setup for about a week now. It seems to be running as intended.

I'm going to close this issue, if you have any questions or problems then please open new issues.

Be sure to read the README file, there are several steps to getting this all setup. It's not as intimidating as it looks, it just ended up long because I wanted to write some instructions for different ways of going about it.