search

helto4real / custom_component_myfitnesspal

Custom component in Home Assistant for getting fitness datat from MyFitnessPal
MIT License
13 stars 13 forks source link

MFP authorisation method changed #25

Open Inferi0r opened 1 year ago

Inferi0r commented 1 year ago

Hi,

Since last night (after restart for update to HAS OS 8.5) MFP is unable to login with integration failed. Restart HAS/reload MFP/ reinstall MFP of integration also provides with a "Wrong username or password!" error. I did not change my account/password and can still login with those credentials over https://www.myfitnesspal.com/account/login

Edit: ahh its confirmed, login is broken due to hidden captcha: https://github.com/coddingtonbear/python-myfitnesspal/issues/144

This is the full debug log Logger: homeassistant.config_entries Source: custom_components/my_fitnesspal/init.py:49 Integration: myfitnesspal (documentation) First occurred: 13:07:28 (1 occurrences) Last logged: 13:07:28

Error setting up entry MyFitnessPal for my_fitnesspal

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 357, in async_setup
    result = await component.async_setup_entry(hass, self)
  File "/config/custom_components/my_fitnesspal/__init__.py", line 51, in async_setup_entry
    client = await hass.async_add_executor_job(wrap_client)
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/config/custom_components/my_fitnesspal/__init__.py", line 49, in wrap_client
    return ext_myfitnesspal.Client(username, password)
  File "/usr/local/lib/python3.10/site-packages/myfitnesspal/client.py", line 72, in __init__
    self._login()
  File "/usr/local/lib/python3.10/site-packages/myfitnesspal/client.py", line 121, in _login
    raise MyfitnesspalLoginError()
myfitnesspal.exceptions.MyfitnesspalLoginError
Inferi0r commented 1 year ago

Is it possible to update to the new 2.0.0 python package? We will have to login via browser since now, but I assume that's not an issue for HA.

rubenzori86 commented 1 year ago

I have the same issue too

kquinsland commented 1 year ago

Is it possible to update to the new 2.0.0 python package? We will have to login via browser since now, but I assume that's not an issue for HA.

Not likely. It looks like the author of the underlying library does not support a "you give me the cookie info" mechanism and opted to directly get it from your local browsers cookiejar. This is more user friendly but does assume that the cookiejar is a) accessible and b) on the same computer that the library will be used/running.

If the underlying library author added a "manually supply the cookie info" mechanism then it mighty be possible to manually configure the MFP integration after logging in on your local desktop / inspecting the cookiejar.

The real solution is MFP implement a proper API that allows for granting of JWT or oauth2 tokens or similar on "headless" devices or an oauth2 web call back strategy.

And while MFP is improving things, there's a laundry list of other things that are outright buggy/slow/broken with their API and mobile sync should be working on

Now that barcode scanning is behind a paywall, I would consider looking for alternatives to MFP. Yes, most alternatives are also paid, but if the choice is "free, buggy, limited API, no barcode scanning" or "paid, buggy, limited API but with barcode scanning" or "paid, api that supports headless, not slow or buggy, barcode scanning" i think the choice is pretty obvious.

I am also a +1 for this but I'll just uninstall the custom component as i'm not sure there's a way forward that MFP won't break again.

coddingtonbear commented 1 year ago

You can absolutely instantiate your own cookiejar and hand it directly to myfitnesspal.Client -- just import the class from http.cookiejar.CookieJar and set the cookies you need. You can then pass that cookiejar to the client directly; see https://python-myfitnesspal.readthedocs.io/en/latest/api/client.html#myfitnesspal.Client for the signature.

dkvdm commented 1 year ago

Also having similar issues, subscribing!

ColorfulQuark commented 1 year ago

You could use browser_cookie3 to get the cookies, then save to a file with pickle.dump or something like

cookiejar = http.cookiejar.LWPCookieJar(filename=filename)
for cookie in session.cookies: cookiejar.set_cookie(cookie) #indented cookiejar.save()

then retrieve the saved file and pass it to myfitnesspal.client. See the __init__ in MFP's client.py for some browser_cookie3 code.

Hope this helps.

helto4real commented 1 year ago

Looks like it is broken indeed. I have not the time to fix any solution since this needs to be developed in the underlying component. If anyone would consider contributing I would be great. I do not have time for this.

bachoo786 commented 1 year ago

Same issue here please can someone help resolve it. Thanks

dkvdm commented 1 year ago

I've created a proof of concept at https://github.com/dkvdm/custom_component_myfitnesspal/tree/cookiejar-poc It requires myfitnesspal==2.0.0 and includes a script that generates a cookiejar, which you copy over on to HASS. To generate cookiejar: python cookiejar_generate.py To test cookiejar: python cookiejar_test.py

Now, I'm no expert in HASS, and even though this should work, I haven't been able to get it to work in HASS due to my lack of knowledge in testing on HASS- I don't know how to run this interactively (via ssh? where is python located?).

Can someone help me with some basic test instructions for running this interactively on HASS (rpi)?

edit: seems like it cannot find myfitnesspal==2.0.0:

Logger: homeassistant.setup
Source: setup.py:184 
First occurred: 4:45:41 PM (1 occurrences) 
Last logged: 4:45:41 PM

Setup failed for custom integration my_fitnesspal: Requirements for my_fitnesspal not found: ['myfitnesspal==2.0.0'].

Any idea why it's not able to find this from PyPI? I'm more than happy to create a local copy of the myfitnesspal python package, but we should rather rely on package standards.

dkvdm commented 1 year ago

Seems like it has to do with gcc missing:

2022-10-10 12:22:18.373 ERROR (SyncWorker_3) [homeassistant.util.package] Unable to install package myfitnesspal==2.0.0: error: subprocess-exited-with-error
× Building wheel for lz4 (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [19 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-aarch64-cpython-310
creating build/lib.linux-aarch64-cpython-310/lz4
copying lz4/version.py -> build/lib.linux-aarch64-cpython-310/lz4
copying lz4/__init__.py -> build/lib.linux-aarch64-cpython-310/lz4
creating build/lib.linux-aarch64-cpython-310/lz4/frame
copying lz4/frame/__init__.py -> build/lib.linux-aarch64-cpython-310/lz4/frame
creating build/lib.linux-aarch64-cpython-310/lz4/block
copying lz4/block/__init__.py -> build/lib.linux-aarch64-cpython-310/lz4/block
running build_ext
building 'lz4._version' extension
creating build/temp.linux-aarch64-cpython-310
creating build/temp.linux-aarch64-cpython-310/lz4
creating build/temp.linux-aarch64-cpython-310/lz4libs
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fno-semantic-interposition -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -DTHREAD_STACK_SIZE=0x100000 -fPIC -Ilz4libs -I/usr/local/include/python3.10 -c lz4/_version.c -o build/temp.linux-aarch64-cpython-310/lz4/_version.o -O3 -Wall -Wundef
error: command 'gcc' failed: No such file or directory
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for lz4
ERROR: Could not build wheels for lz4, which is required to install pyproject.toml-based projects
2022-10-10 12:23:17.639 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: nzbget, raspberry_pi, androidtv
2022-10-10 12:24:02.191 ERROR (SyncWorker_3) [homeassistant.util.package] Unable to install package myfitnesspal==2.0.0: error: subprocess-exited-with-error
× Building wheel for lz4 (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [19 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-aarch64-cpython-310
creating build/lib.linux-aarch64-cpython-310/lz4
copying lz4/version.py -> build/lib.linux-aarch64-cpython-310/lz4
copying lz4/__init__.py -> build/lib.linux-aarch64-cpython-310/lz4
creating build/lib.linux-aarch64-cpython-310/lz4/frame
copying lz4/frame/__init__.py -> build/lib.linux-aarch64-cpython-310/lz4/frame
creating build/lib.linux-aarch64-cpython-310/lz4/block
copying lz4/block/__init__.py -> build/lib.linux-aarch64-cpython-310/lz4/block
running build_ext
building 'lz4._version' extension
creating build/temp.linux-aarch64-cpython-310
creating build/temp.linux-aarch64-cpython-310/lz4
creating build/temp.linux-aarch64-cpython-310/lz4libs
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fno-semantic-interposition -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -DTHREAD_STACK_SIZE=0x100000 -fPIC -Ilz4libs -I/usr/local/include/python3.10 -c lz4/_version.c -o build/temp.linux-aarch64-cpython-310/lz4/_version.o -O3 -Wall -Wundef
error: command 'gcc' failed: No such file or directory
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for lz4
ERROR: Could not build wheels for lz4, which is required to install pyproject.toml-based projects
2022-10-10 12:24:17.724 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: nzbget, raspberry_pi, androidtv
2022-10-10 12:25:17.808 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: nzbget, raspberry_pi, androidtv
2022-10-10 12:25:45.890 ERROR (SyncWorker_3) [homeassistant.util.package] Unable to install package myfitnesspal==2.0.0: error: subprocess-exited-with-error
× Building wheel for lz4 (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [19 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-aarch64-cpython-310
creating build/lib.linux-aarch64-cpython-310/lz4
copying lz4/version.py -> build/lib.linux-aarch64-cpython-310/lz4
copying lz4/__init__.py -> build/lib.linux-aarch64-cpython-310/lz4
creating build/lib.linux-aarch64-cpython-310/lz4/frame
copying lz4/frame/__init__.py -> build/lib.linux-aarch64-cpython-310/lz4/frame
creating build/lib.linux-aarch64-cpython-310/lz4/block
copying lz4/block/__init__.py -> build/lib.linux-aarch64-cpython-310/lz4/block
running build_ext
building 'lz4._version' extension
creating build/temp.linux-aarch64-cpython-310
creating build/temp.linux-aarch64-cpython-310/lz4
creating build/temp.linux-aarch64-cpython-310/lz4libs
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fno-semantic-interposition -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -DTHREAD_STACK_SIZE=0x100000 -fPIC -Ilz4libs -I/usr/local/include/python3.10 -c lz4/_version.c -o build/temp.linux-aarch64-cpython-310/lz4/_version.o -O3 -Wall -Wundef
error: command 'gcc' failed: No such file or directory
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.

What's the best way of making gcc included as part of the RPi install?

dkvdm commented 1 year ago

Can somebody help me figure out how to install myfitnesspal==2.0.0 (requires gcc) on raspberry pi, so I can build a fix for all of us? thanks.

sndri commented 1 year ago

Hi guys. This is the current workaround I have for getting the data without auth, be warned I cannot code & this won't be desirable for some with privacy concerns. MFP allows for a free username change at /account/change-username, so if it a concern one could switch to random generator text. Log into MFP & visit /account/diary-settings/, "Diary Sharing" can be set to public - allowing the last 365 days to be visible at /food/diary/user/. I then grabbed this into a csv using:

from datetime import date, timedelta
import requests
from bs4 import BeautifulSoup
import csv
today = date.today()
past = today-timedelta(days = 28)
url = f"https://www.myfitnesspal.com/reports/printable_diary/smartfleshlight?from={str(past)}&to={str(today)}" #yyyy-mm-dd
page = requests.get(url)
soup = BeautifulSoup(page.content, "html.parser")
head = soup.find('thead')
column_labels = ["Date"] +  [i.text for i in head.find_all('td')[1:]]
print(column_labels)
dates = soup.find_all('h2')
tables = soup.find_all('table')
if len(dates) != len(tables):
    print('OOPSIE WOOPSIE!! Uwu We made a fucky wucky!! A wittle fucko boingo! The code monkeys at our headquarters are working VEWY HAWD to fix this!')
out_to_csv = []
for i, j in zip(dates, tables):
    #out_to_csv += [[i.text]]
    for k in j.find_all('tfoot'): 
        out_to_csv += [[i.text]+[l.text for l in k.find_all('td')[1:]]]
[print(i) for i in out_to_csv]
header = column_labels
with open('homepal.csv','w', encoding = 'UTF8', newline = '') as f:
          writer = csv.writer(f)
          writer.writerow(header)
          writer.writerows(out_to_csv)

I'm very new to HA & coding but I'd like to at some point have this data pulled from MFP, and using my Withings smart scale bodyweight sensor, calculate real total daily energy expenditure (instead of equation-derived figures - high margin of error). This is typically done manually in sheets, shouldn't be too hard to drop the API and use this instead, and add a weight sensor option to generate a TDEE using the existing smart scale integration sensors - but I'm sure someone else could get to it much faster than myself.