search

tcgoetz / GarminDB

Download and parse data from Garmin Connect or a Garmin watch, FitBit CSV, and MS Health CSV files into and analyze data in Sqlite serverless databases with Jupyter notebooks.
GNU General Public License v2.0
1.18k stars 142 forks source link

make setup problem #40

Closed Kywalh closed 4 years ago

Kywalh commented 4 years ago

Hello, I have problems when it comes to run the make setup on Ubuntu 18 and Virtualbox. Here is the problem:

yann@yann-VirtualBox:~$ cd GarminDB/
yann@yann-VirtualBox:~/GarminDB$ make setup
make: /usr/local/bin/python3: Command not found
make: /usr/local/bin/python3: Command not found
git submodule init
git submodule update
Cloning into '/home/yann/GarminDB/Tcx'...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:tcgoetz/Tcx.git' into submodule path '/home/yann/GarminDB/Tcx' failed
Failed to clone 'Tcx'. Retry scheduled
Cloning into '/home/yann/GarminDB/utilities'...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:tcgoetz/utilities.git' into submodule path '/home/yann/GarminDB/utilities' failed
Failed to clone 'utilities'. Retry scheduled
Cloning into '/home/yann/GarminDB/Tcx'...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:tcgoetz/Tcx.git' into submodule path '/home/yann/GarminDB/Tcx' failed
Failed to clone 'Tcx' a second time, aborting
Makefile:43: recipe for target 'submodules_update' failed
make: *** [submodules_update] Error 1
yann@yann-VirtualBox:~/GarminDB$
tcgoetz commented 4 years ago

Do you have Python3 installed? /usr/local/bin/python3: Command not found is saying you don't. If not, Google for the instructions.

Do you have a GitHub login and a shared ssh key setup? `git@github.com: Permission denied (publickey)1 says you don't. https://help.github.com/en/github/authenticating-to-github/adding-a-new-ssh-key-to-your-github-account

Kywalh commented 4 years ago

Hi, thanks for the quick reply. I think I do have python3 installed

yann@yann-VirtualBox:~/GarminDB$ python3 --version
Python 3.6.9

Does it require a 3.7.x version ?

For the ssh keys, I have this:

yann@yann-VirtualBox:~/GarminDB$ ls -al ~/.ssh
total 12
drwx------  2 yann yann 4096 déc.  18 09:13 .
drwxr-xr-x 20 yann yann 4096 déc.  18 10:48 ..
-rw-r--r--  1 yann yann 1326 déc.  18 09:18 known_hosts
tcgoetz commented 4 years ago

If you run which python3 what doe sit say? Is it "/usr/local/bin/python3"

As far as the SSH keys, your listing isn't what's need. Did you setup a shared SSH key per in the instructions int he link I shared. How did you clone the GarminDB repo? With SSH or HTTPS?

tcgoetz commented 4 years ago

After you run which python3, you may need to edit the defines defines.mk to point to your python.

Kywalh commented 4 years ago

Which pythonreturns the following:

yann@yann-VirtualBox:~$ which python3
/usr/bin/python3

For the ssh keys, I don't know (haven't tried the link yet) but since I did a git clone https://..... I assume that I have kind of credentials, don't I ?

Kywalh commented 4 years ago

And after editing the python3 path in the defines.mk, I have the following:

yann@yann-VirtualBox:~/GarminDB$ make setup
Traceback (most recent call last):
  File "garmin.py", line 18, in <module>
    from version import print_version, python_version_check, log_version
  File "/home/yann/GarminDB/version.py", line 7, in <module>
    import utilities.version as uv
ModuleNotFoundError: No module named 'utilities.version'
Traceback (most recent call last):
  File "garmin.py", line 18, in <module>
    from version import print_version, python_version_check, log_version
  File "/home/yann/GarminDB/version.py", line 7, in <module>
    import utilities.version as uv
ModuleNotFoundError: No module named 'utilities.version'
git submodule init
git submodule update
Cloning into '/home/yann/GarminDB/Tcx'...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
tcgoetz commented 4 years ago

The submodules are still not cloning. The project is composed of the main repo GarminDB and two sub-repos Tcx and utilities. The submodule is clone is failing with "git@github.com: Permission denied (publickey)."

You originally cloned with HTTPS and the submodules won't work with that. You need to set up the shared keys and clone with SSH.

Kywalh commented 4 years ago

OK, understood. I create the ssh key and could clone the repo using SSH:

yann@yann-VirtualBox:~$ git clone git@github.com:tcgoetz/GarminDB.git
Cloning into 'GarminDB'...
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 2861 (delta 7), reused 9 (delta 3), pack-reused 2842
Receiving objects: 100% (2861/2861), 3.36 MiB | 432.00 KiB/s, done.
Resolving deltas: 100% (2064/2064), done.

Then I changed the defines.mk to change the path to Python3 and execute the make setup with the following output:

yann@yann-VirtualBox:~/GarminDB$ make setup
Traceback (most recent call last):
  File "garmin.py", line 18, in <module>
    from version import print_version, python_version_check, log_version
  File "/home/yann/GarminDB/version.py", line 7, in <module>
    import utilities.version as uv
ModuleNotFoundError: No module named 'utilities.version'
Traceback (most recent call last):
  File "garmin.py", line 18, in <module>
    from version import print_version, python_version_check, log_version
  File "/home/yann/GarminDB/version.py", line 7, in <module>
    import utilities.version as uv
ModuleNotFoundError: No module named 'utilities.version'
git submodule init
Submodule 'Fit' (git@github.com:tcgoetz/Fit.git) registered for path 'Fit'
Submodule 'Tcx' (git@github.com:tcgoetz/Tcx.git) registered for path 'Tcx'
Submodule 'utilities' (git@github.com:tcgoetz/utilities.git) registered for path 'utilities'
git submodule update
Cloning into '/home/yann/GarminDB/Fit'...
Cloning into '/home/yann/GarminDB/Tcx'...
Cloning into '/home/yann/GarminDB/utilities'...
Submodule path 'Fit': checked out '6bc6ca0cabefbc3d6feefb213cba67a5a0176d22'
Submodule path 'Tcx': checked out '9e528461a81979cafb918a90eadedf2d4b0ae159'
Submodule path 'utilities': checked out 'df337cde4a5fcab0c750e9896a18b3022ba602e2'
git pull --rebase
error: cannot pull with rebase: You have unstaged changes.
error: please commit or stash them.
Makefile:39: recipe for target 'update' failed
make: *** [update] Error 128
tcgoetz commented 4 years ago

This error: cannot pull with rebase: You have unstaged changes.1 means that you can't update your repo when you have changes in it (the edits to defines.mk). You should not need those anymore anyway due to changes I made yesterday. Remove the changes withgit checkout -f` and try the update again.

Kywalh commented 4 years ago

I did thegit checkout -f as you mentioned and rerun the make setup with the following output:

yann@yann-VirtualBox:~/GarminDB$ git checkout -f
Your branch is up to date with 'origin/master'.
yann@yann-VirtualBox:~/GarminDB$ make setup
Traceback (most recent call last):
  File "garmin.py", line 18, in <module>
    from version import print_version, python_version_check, log_version
  File "/home/yann/GarminDB/version.py", line 7, in <module>
    import utilities.version as uv
  File "/home/yann/GarminDB/utilities/__init__.py", line 11, in <module>
    from utilities.db import DbParams, DB, DBObject
  File "/home/yann/GarminDB/utilities/db.py", line 13, in <module>
    from sqlalchemy import create_engine, func, desc, extract, and_, literal_column
ModuleNotFoundError: No module named 'sqlalchemy'
Traceback (most recent call last):
  File "garmin.py", line 18, in <module>
    from version import print_version, python_version_check, log_version
  File "/home/yann/GarminDB/version.py", line 7, in <module>
    import utilities.version as uv
  File "/home/yann/GarminDB/utilities/__init__.py", line 11, in <module>
    from utilities.db import DbParams, DB, DBObject
  File "/home/yann/GarminDB/utilities/db.py", line 13, in <module>
    from sqlalchemy import create_engine, func, desc, extract, and_, literal_column
ModuleNotFoundError: No module named 'sqlalchemy'
git submodule init
git submodule update
git pull --rebase
Already up to date.
Current branch master is up to date.
make -C Fit deps
make[1]: Entering directory '/home/yann/GarminDB/Fit'
install --user --upgrade --requirement requirements.txt
install: unrecognized option '--user'
Try 'install --help' for more information.
Makefile:8: recipe for target 'deps' failed
make[1]: *** [deps] Error 1
make[1]: Leaving directory '/home/yann/GarminDB/Fit'
Makefile:46: recipe for target 'deps' failed
make: *** [deps] Error 2
tcgoetz commented 4 years ago

defines.mk generated empty strings for $PYTHON and $PIP. Not sure why and I don't have a matching setup to test it on.

Kywalh commented 4 years ago

I'm really grateful you take time to solve these issues... I cloned again in ssh and rerun the make setup, after I had to install PIP3:

yann@yann-VirtualBox:~/GarminDB$ make setup
Traceback (most recent call last):
  File "garmin.py", line 18, in <module>
    from version import print_version, python_version_check, log_version
  File "/home/yann/GarminDB/version.py", line 7, in <module>
    import utilities.version as uv
ModuleNotFoundError: No module named 'utilities.version'
Traceback (most recent call last):
  File "garmin.py", line 18, in <module>
    from version import print_version, python_version_check, log_version
  File "/home/yann/GarminDB/version.py", line 7, in <module>
    import utilities.version as uv
ModuleNotFoundError: No module named 'utilities.version'
git submodule init
Submodule 'Fit' (git@github.com:tcgoetz/Fit.git) registered for path 'Fit'
Submodule 'Tcx' (git@github.com:tcgoetz/Tcx.git) registered for path 'Tcx'
Submodule 'utilities' (git@github.com:tcgoetz/utilities.git) registered for path 'utilities'
git submodule update
Cloning into '/home/yann/GarminDB/Fit'...
Cloning into '/home/yann/GarminDB/Tcx'...
Cloning into '/home/yann/GarminDB/utilities'...
Submodule path 'Fit': checked out '6bc6ca0cabefbc3d6feefb213cba67a5a0176d22'
Submodule path 'Tcx': checked out '9e528461a81979cafb918a90eadedf2d4b0ae159'
Submodule path 'utilities': checked out 'df337cde4a5fcab0c750e9896a18b3022ba602e2'
git pull --rebase
Already up to date.
Current branch master is up to date.
make -C Fit deps
make[1]: Entering directory '/home/yann/GarminDB/Fit'
/usr/bin/pip3 install --user --upgrade --requirement requirements.txt
You must give at least one requirement to install (see "pip help install")
make[1]: Leaving directory '/home/yann/GarminDB/Fit'
make -C utilities deps
make[1]: Entering directory '/home/yann/GarminDB/utilities'
/usr/local/bin/pip3 install --user --upgrade --requirement requirements.txt
make[1]: /usr/local/bin/pip3: Command not found
Makefile:19: recipe for target 'deps' failed
make[1]: *** [deps] Error 127
make[1]: Leaving directory '/home/yann/GarminDB/utilities'
Makefile:46: recipe for target 'deps' failed
make: *** [deps] Error 2
yann@yann-VirtualBox:~/GarminDB$ which pip3
/usr/bin/pip3

in the following command line, I can see that the PIP3 path isn't correct: make[1]: /usr/local/bin/pip3: Command not found since : 

yann@yann-VirtualBox:~/GarminDB$ which pip3
/usr/bin/pip3
tcgoetz commented 4 years ago

In line 13 of defines.mk (since you are on Linux):

PIP3=$(shell which pip3)

should set it. You need to see why that isn't working.

Kywalh commented 4 years ago

I'm not an expert in Linux system so I don't really know where to look at... There is something strange :

make[1]: Entering directory '/home/yann/GarminDB/Fit'
**/usr/bin/pip3** install --user --upgrade --requirement requirements.txt
You must give at least one requirement to install (see "pip help install")
make[1]: Leaving directory '/home/yann/GarminDB/Fit'
make -C utilities deps
make[1]: Entering directory '/home/yann/GarminDB/utilities'
**/usr/local/bin/pip3** install --user --upgrade --requirement requirements.txt
make[1]: /usr/local/bin/pip3: Command not found

so

/usr/bin/pip3 install --user --upgrade --requirement requirements.txt ==> uses the right path /usr/local/bin/pip3 install --user --upgrade --requirement requirements.txt ==> uses the wrong path

awaisas commented 4 years ago

There is an Makefile in the Tcx folder as well. Changing the PIP3 variable to the correct pip3 path in it solves the problem.

tcgoetz commented 4 years ago

Try again with the latest code.

Kywalh commented 4 years ago

OK guys ! it seems to work now. I have followed the next instructions to pull the data. Now I need to understand how to read them :-)