Closed jasongfleischer closed 2 years ago
tktran11
commented
2 years ago Potentially unrelated, but it looks like your local config is missing fastparquet (which is required for panda's implementation of pd.read_parquet() to work with fastparquet as the selected engine).
jasongfleischer
commented
2 years ago I did a fast parquet install, still the same error message.
Per your request on Slack, here's the environment definition yml fleischer_base_env_panenka.yml.txt
tktran11
commented
2 years ago @jasongfleischer when you get the opportunity can you try running a local version of the docs?
to do so, CD into the main repo and type "make docs_serve"
this will only work if you already have Jekyll installed (I believe all the gem files and other requirements should already be present in the repo). Otherwise you can attempt to follow these steps in case local viewing doesn't work.
the link in the image: https://jekyllrb.com/docs/installation/ubuntu/
tktran11
commented
2 years ago i was having trouble getting the yaml file to create a working environment for me, but just cloning the repo and then running a local version of the docs worked perfectly fine
jasongfleischer
commented
2 years ago No dice on the docs. After the kids in bed tonight I'm going to try doing all this on my laptop instead of my desktop
(base) panenka:SALA jfleischer$ make docs_serve
nbdev_build_docs
Traceback (most recent call last):
File "/Users/jfleischer/opt/anaconda3/bin/nbdev_build_docs", line 5, in <module>
from nbdev.cli import nbdev_build_docs
ImportError: cannot import name 'nbdev_build_docs' from 'nbdev.cli' (/Users/jfleischer/opt/anaconda3/lib/python3.7/site-packages/nbdev/cli.py)
make: *** [docs] Error 1
(base) panenka:SALA jfleischer$ jekyll --version
jekyll 4.2.0
(base) panenka:SALA jfleischer$ Before I didn't mention that I am using current nbdev (v2) because I'd never installed on my desktop before. Perhaps @474benchen has some idea what's going on?
No dice with a brand new Conda environment, see attached screen grab if you're curious. TL;DR its a new error that is says that Astral's output, data['Sunrise'] is not a timestamp, so operations such as .dt.tz do not work. My new conda env is using astral 2.2, python 3.9.12, pandas 1.4.3. Full yml file attached again at the bottom of this message
Erm, nope. Using the new environment (and nbdev2 w/ quarto installed as root) I get this. Any comments Ben?
(SALA) panenka:SALA jfleischer$ nbdev_docs
pandoc -o README.md
to: gfm+footnotes+tex_math_dollars-yaml_metadata_block
standalone: true
default-image-extension: png
metadata
description: A suite of tools built to work with light data from a Philips Spectrum Plus.
title: SALA - Spectrum Actiwatch Light Analysis
Output created: docs/README.md
Traceback (most recent call last):
File "/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3398, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-1-5621ce88f242>", line 3, in <cell line: 3>
NameError: name 'SALAFrame' is not defined
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/jfleischer/opt/anaconda3/envs/SALA/bin/nbdev_filter", line 8, in <module>
sys.exit(nbdev_filter())
File "/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/fastcore/script.py", line 119, in _f
return tfunc(**merge(args, args_from_prog(func, xtra)))
File "/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/nbdev/cli.py", line 64, in nbdev_filter
with redirect_stdout(dn): filt.nb_proc(nb).process()
File "/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/nbdev/process.py", line 122, in process
for proc in self.procs: self._proc(proc)
File "/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/nbdev/process.py", line 115, in _proc
for cell in self.nb.cells: self._process_cell(proc, cell)
File "/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/nbdev/process.py", line 106, in _process_cell
if callable(proc) and not _is_direc(proc): cell = opt_set(cell, proc(cell))
File "/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/nbdev/processors.py", line 290, in __call__
if self.k.exc: raise Exception(f"Error{' in notebook: '+title if title else ''} in cell {cell.idx_} :\n{cell.source}") from self.k.exc[1]
Exception: Error in notebook: Processing in cell 9 :
#|output: asis
#| echo: false
show_doc(SALAFrame.__init__, title_level = 3)
fastparquet must be >0.5. That probably needs to be in settings.ini. Right now conda install fastparquet is using 0.5. Instead we must pip install it to get the current. If you already installed it with conda, do a pip install --upgrade fastparquet
After this change nbdev_export and nbdev_test had no errors. There are however some _test warnings that I suspect are not important
(SALA) panenka:SALA jfleischer$ nbdev_test
Found 1 csv files in data/v3/. Pass #1, raw data
.
.Found 1 csv files in data/v1/. Pass #1, raw data
.
.
Pass #2, data summary
.
.EOF without retrieving summary data: data/v3/user1234_v3sample.csv
Pass #2, data summary
.
.EOF without retrieving summary data: data/v1/user1234_v1sample.csv
Found 1 csv files in data/v1/. Pass #1, raw data
.
.Found 1 csv files in data/v3/. Pass #1, raw data
.
.
Pass #2, data summary
.
.EOF without retrieving summary data: data/v3/user1234_v3sample.csv
Pass #2, data summary
.
.EOF without retrieving summary data: data/v1/user1234_v1sample.csv
Success.
/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/joblib/externals/loky/backend/resource_tracker.py:318: UserWarning: resource_tracker: There appear to be 6 leaked folder objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/joblib/externals/loky/backend/resource_tracker.py:333: UserWarning: resource_tracker: /var/folders/zn/62137mk56n9bzpdvgmj4f4_w0000gn/T/joblib_memmapping_folder_15812_b623a73d255a4604ac1ebcdfef0f2519_de08fdf91303469f91a6120173804be7: FileNotFoundError(2, 'No such file or directory')
warnings.warn('resource_tracker: %s: %r' % (name, e))
/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/joblib/externals/loky/backend/resource_tracker.py:333: UserWarning: resource_tracker: /var/folders/zn/62137mk56n9bzpdvgmj4f4_w0000gn/T/joblib_memmapping_folder_15812_b623a73d255a4604ac1ebcdfef0f2519_e8f80ed3b78f4796a1b5ad97c92c2de7: FileNotFoundError(2, 'No such file or directory')
warnings.warn('resource_tracker: %s: %r' % (name, e))
/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/joblib/externals/loky/backend/resource_tracker.py:333: UserWarning: resource_tracker: /var/folders/zn/62137mk56n9bzpdvgmj4f4_w0000gn/T/joblib_memmapping_folder_15812_ae1676424b2a4a0f92aefbe3ef19205e_cc970b38a76e46d4a37aa2ecd3dd7acc: FileNotFoundError(2, 'No such file or directory')
warnings.warn('resource_tracker: %s: %r' % (name, e))
/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/joblib/externals/loky/backend/resource_tracker.py:333: UserWarning: resource_tracker: /var/folders/zn/62137mk56n9bzpdvgmj4f4_w0000gn/T/joblib_memmapping_folder_15812_98fab3cc445b49ecb407758e532f6d00_7df93c8a5535404c91e0e35be097092a: FileNotFoundError(2, 'No such file or directory')
warnings.warn('resource_tracker: %s: %r' % (name, e))
/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/joblib/externals/loky/backend/resource_tracker.py:333: UserWarning: resource_tracker: /var/folders/zn/62137mk56n9bzpdvgmj4f4_w0000gn/T/joblib_memmapping_folder_15812_98fab3cc445b49ecb407758e532f6d00_4d02505dfe994cc3a682ff035c4f4ad9: FileNotFoundError(2, 'No such file or directory')
warnings.warn('resource_tracker: %s: %r' % (name, e))
/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/joblib/externals/loky/backend/resource_tracker.py:333: UserWarning: resource_tracker: /var/folders/zn/62137mk56n9bzpdvgmj4f4_w0000gn/T/joblib_memmapping_folder_15812_9d6006fd7465417c9a25752ec4b447db_fa16bf84d82c4591b3c6d06eff6ed922: FileNotFoundError(2, 'No such file or directory')
warnings.warn('resource_tracker: %s: %r' % (name, e))To migrate to nbdev2 we need to migrate the notebooks documentation commands to a new format. See https://nbdev.fast.ai/migrating.html
This is a big PITA on MacOS. The default MacOS bash shell is very old, and does not include globstar which is part of the migration script from v1 to v2. A new bash can be installed via homebrew, follow the directions here https://dev.to/emcain/how-to-change-bash-versions-on-mac-with-homebrew-20o3
NB I have filed an issue for nbdev to modify their instructions https://github.com/fastai/nbdev/issues/1060
I have successfully updated my bash and migrated according to the recipe. However I still have the following errors... @tktran11 @samuelchu90 didn't we see something like this during the meeting, and it got resolved? How did we resolve it??? My brain is locked rn.
(SALA) panenka:SALA jfleischer$ nbdev_docs
pandoc -o README.md
to: gfm+footnotes+tex_math_dollars-yaml_metadata_block
standalone: true
default-image-extension: png
metadata
description: A suite of tools built to work with light data from a Philips Spectrum Plus.
title: SALA - Spectrum Actiwatch Light Analysis
Output created: _docs/README.md
Traceback (most recent call last):
File "/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3398, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-1-9e2041d8b22e>", line 10, in <cell line: 10>
ModuleNotFoundError: No module named 'SALA'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/jfleischer/opt/anaconda3/envs/SALA/bin/nbdev_filter", line 8, in <module>
sys.exit(nbdev_filter())
File "/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/fastcore/script.py", line 119, in _f
return tfunc(**merge(args, args_from_prog(func, xtra)))
File "/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/nbdev/cli.py", line 64, in nbdev_filter
with redirect_stdout(dn): filt.nb_proc(nb).process()
File "/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/nbdev/process.py", line 122, in process
for proc in self.procs: self._proc(proc)
File "/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/nbdev/process.py", line 115, in _proc
for cell in self.nb.cells: self._process_cell(proc, cell)
File "/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/nbdev/process.py", line 106, in _process_cell
if callable(proc) and not _is_direc(proc): cell = opt_set(cell, proc(cell))
File "/Users/jfleischer/opt/anaconda3/envs/SALA/lib/python3.9/site-packages/nbdev/processors.py", line 290, in __call__
if self.k.exc: raise Exception(f"Error{' in notebook: '+title if title else ''} in cell {cell.idx_} :\n{cell.source}") from self.k.exc[1]
Exception: Error in notebook: Plots in cell 4 :
from joblib import Parallel, delayed
import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns
import numpy as np
import matplotlib.dates as md
import matplotlib as mpl
from datetime import time
from SALA.processing import SALAFrame
(SALA) panenka:SALA jfleischer$ 
seeM
commented
2 years ago pip install -e '.[dev]' might fix your latest issue. As per https://nbdev.fast.ai/tutorials/tutorial.html#install-your-package
jasongfleischer
commented
2 years ago That worked, thanks @seeM !
jasongfleischer
commented
2 years ago Following the directions in https://nbdev.fast.ai/tutorials/tutorial.html#enabling-github-pages gave me most of the problem solved. In addition to what's in there I had to manually create a gh-pages branch... I suspect that the automatic nbdev stuff to do this didn't work because this started life as a private rather than public repo.
The repo's automatic push-and-it-rebuilds actions are working as intended. The docs are now (kinda) available at https://FleischerResearchLab.github.io/SALA/
So I'm closing this issue.
HOWEVER @tktran11 @474benchen @samuelchu90 @QiwenZz I need your help! Right now it's just the README.md file, not the rest of the docs! So seems like there's some config missing. Can you figure this out and make it go live properly?
seeM
commented
2 years ago @jasongfleischer https://fleischerresearchlab.github.io/SALA/ looks like the default website that GitHub renders from your README.md. This often happens when the repo isn't configured to use the gh-pages branch. Here are instructions on how to do that: https://nbdev.fast.ai/tutorials/tutorial.html#enabling-github-pages. The relevant part for convenience:
You can enable it for your repo by clicking on the “Settings” tab near the top-right of your repo page, then “Pages” on the left, then setting the “Branch” to “gh-pages”, and finally clicking “Save”.
Problem:
nbdev_test does not pass on my local computer after moving repo from Tyler's private repo to our lab org.
Context: may honestly have nothing to do with the move, and may just be my local config. In fact I suspect this is the case because when I pushed the repo to Github it built successfully! See https://github.com/FleischerResearchLab/SALA/actions/runs/3048411194
However, the build did not deploy a docs website at https://fleischerresearchlab.github.io/sala which is what I expected. What am I doing wrong?
Setup:
MacOS 10.15.7 Anaconda with the following packages
Error: