CommonClimate
commented
2 years ago Merci Laurent! I will play with that tomorrow as part of other upgrades to the notebook, which is now out of step with the latest Pyleoclim releases (0.8.x).
Open ltroussellier opened 2 years ago
CommonClimate
commented
2 years ago Merci Laurent! I will play with that tomorrow as part of other upgrades to the notebook, which is now out of step with the latest Pyleoclim releases (0.8.x).
CommonClimate
commented
2 years ago Hi Laurent, actually, with that separator I still encounter problems:
(top is your method loading the txt file, bottom is the csv I manually extracted)
What does your output look like? Perhaps it's a pandas version thing too. In this environment I am running 1.4.1
ltroussellier
commented
2 years ago Hi,
i have done it like this :
main differences i can spot : 1/ I used the "txt" file, and not the "csv" one. 2/ the encoding, but i think it's link to the file format (in csv, panda can actually tell) 3/ the starting row
But in fact, the top panda dataframe you showed is maybe good : there is a lot of NaN in the data before the age 38.37379
After dropping Nan:
Hope this helps toward a simpler notebook
CommonClimate
commented
2 years ago Hello @ltroussellier . I tried and failed your solution again with my version of pandas. Can you tell me which one you are using? For now, the notebook (updated to work with v0.8.1), still uses the hand-made csv file.
ltroussellier
commented
2 years ago In fact, I use a personnel kernel on a remote jupyter to test some librairies so, not the clearest pip list but here the complete list : Pandas : 1.2.4
Package Version
----------------------------- --------------------------
adal 1.2.7
adlfs 0.7.5
affine 2.3.0
aiobotocore 1.2.2
aiohttp 3.7.4
aioitertools 0.7.1
alembic 1.6.2
amqp 5.0.6
anyio 3.1.0
appdirs 1.4.4
argon2-cffi 20.1.0
asciitree 0.3.3
asn1crypto 1.4.0
astropy 4.2.1
async-generator 1.10
async-timeout 3.0.1
attrs 21.2.0
av 8.0.3
awscli 1.19.77
azure-core 1.14.0
azure-datalake-store 0.0.51
azure-identity 1.6.0
azure-storage-blob 12.8.1
Babel 2.9.1
backcall 0.2.0
backports.functools-lru-cache 1.6.4
batchspawner 1.1.0
bcrypt 3.2.0
beautifulsoup4 4.9.3
billiard 3.6.4.0
bleach 3.3.0
blinker 1.4
blosc 1.10.2
bokeh 2.3.2
boto3 1.17.77
botocore 1.20.77
Bottleneck 1.3.2
branca 0.4.2
brotlipy 0.7.0
cached-property 1.5.2
cachetools 4.2.2
cachey 0.2.1
Cartopy 0.19.0.post1
celery 5.0.5
certifi 2021.10.8
certipy 0.1.3
cf-xarray 0.5.2
cffi 1.14.5
cfgrib 0.9.9.0
cftime 1.5.0
cgen 2020.1
chardet 4.0.0
ciso 0.1.0
click 7.1.2
click-didyoumean 0.0.3
click-plugins 1.1.1
click-repl 0.1.6
cligj 0.7.1
cloudpickle 1.6.0
cmip6-preprocessing 0.0.0
cmocean 2.0
colorama 0.4.3
colorcet 2.0.6
colorspacious 1.1.2
configobj 5.0.6
croniter 0.3.36
cryptography 3.4.7
cycler 0.10.0
cytoolz 0.11.0
dask 2021.5.0
dask-gateway 0.9.0
dask-glm 0.2.0
dask-jobqueue 0.7.2
dask-kubernetes 2021.3.1
dask-labextension 5.0.1
dask-ml 1.9.0
datacube 1.8.3
datashader 0.12.1
datashape 0.5.4
decorator 5.0.9
defusedxml 0.7.1
descartes 1.1.0
distributed 2021.5.0
docker 5.0.0
docker-pycreds 0.4.0
docopt 0.6.2
docrep 0.2.7
docutils 0.15.2
donfig 0.6.0
eccodes 1.2.0
entrypoints 0.3
eofs 1.4.0
erddapy 1.0.0
ESMPy 8.1.0
et-xmlfile 1.1.0
fastapi 0.65.1
fasteners 0.16
fastjmd95 0.2.1
fastprogress 1.0.0
Fiona 1.8.18
fsspec 2021.5.0
future 0.18.2
gcsfs 2021.5.0
GDAL 3.2.1
geocube 0.0.16
geopandas 0.9.0
geoviews 0.0.0+g33876c88.gitarchive
gitdb 4.0.7
GitPython 3.1.17
google-auth 1.30.0
google-auth-oauthlib 0.4.4
graphviz 0.16
greenlet 1.1.0
gsw 3.4.0
h11 0.12.0
h5netcdf 0.11.0
h5py 3.2.1
HeapDict 1.0.1
holoviews 1.14.3
httptools 0.1.1
hvplot 0.7.1
idna 2.10
imagecodecs 2021.3.31
imageio 2.9.0
importlib-metadata 4.0.1
importlib-resources 5.1.3
intake 0.6.2
intake-esm 2021.1.15
intake-geopandas 0.2.4
intake-stac 0.0.0
intake-xarray 0.5.0
ipykernel 5.5.5
ipyleaflet 0.13.6
ipympl 0.8.8
ipyspin 0.1.5
ipython 7.23.1
ipython-genutils 0.2.0
ipywidgets 7.6.3
isodate 0.6.0
jdcal 1.4.1
jedi 0.18.0
Jinja2 3.0.1
jmespath 0.10.0
joblib 1.0.1
json5 0.9.5
jsonschema 3.2.0
jupyter-bokeh 3.0.0
jupyter-client 6.1.12
jupyter-core 4.7.1
jupyter-panel-proxy 0.1.0
jupyter-resource-usage 0.6.0
jupyter-server 1.7.0
jupyter-server-mathjax 0.2.2
jupyter-server-proxy 3.0.2
jupyter-telemetry 0.1.0
jupyterhub 1.4.1
jupyterlab 3.0.16
jupyterlab-execute-time 2.0.2
jupyterlab-git 0.30.1
jupyterlab-pygments 0.1.2
jupyterlab-server 2.5.2
jupyterlab-widgets 1.0.0
kiwisolver 1.3.1
kombu 5.0.2
kubernetes 17.17.0
kubernetes-asyncio 12.1.0
lark-parser 0.11.3
llvmlite 0.36.0
locket 0.2.0
lz4 3.1.3
Mako 1.1.4
Markdown 3.3.4
MarkupSafe 2.0.1
marshmallow 3.12.1
marshmallow-oneofschema 2.1.0
matplotlib 3.4.2
matplotlib-inline 0.1.2
mercantile 1.2.1
MetPy 1.0.1
mistune 0.8.4
monotonic 1.5
mpi4py 3.0.3
msal 1.12.0
msal-extensions 0.3.0
msgpack 1.0.2
msrest 0.6.21
multidict 5.1.0
multipledispatch 0.6.0
munch 2.5.0
mypy-extensions 0.4.3
natsort 7.1.1
nb-conda-kernels 2.3.1
nbclassic 0.2.8
nbclient 0.5.3
nbconvert 6.0.7
nbdime 3.0.0
nbformat 5.1.3
nbgitpuller 0.9.0
nbstripout 0.4.0
nc-time-axis 1.2.0
nest-asyncio 1.5.1
netCDF4 1.5.6
networkx 2.5
node 0.9.26
nodejs 0.1.1
notebook 6.4.0
npm 0.1.1
numba 0.53.1
numcodecs 0.7.3
numpy 1.20.3
oauthlib 3.0.1
odict 1.7.0
olefile 0.46
openpyxl 3.0.7
optional-django 0.1.0
packaging 20.9
pamela 1.0.0
pandas 1.2.4
pandocfilters 1.4.2
panel 0.11.3
param 1.10.1
paramiko 2.7.2
parcels 2.2.2
parso 0.8.2
partd 1.2.0
patsy 0.5.1
pendulum 2.1.2
pexpect 4.8.0
pickleshare 0.7.5
Pillow 8.2.0
Pint 0.17
pip 20.3.4
plumber 1.6
pooch 1.3.0
portalocker 1.7.0
prefect 0.14.19
progressbar2 3.53.1
prometheus-client 0.10.1
prompt-toolkit 3.0.18
psutil 5.8.0
psycopg2 2.8.6
ptyprocess 0.7.0
pyarrow 4.0.0
pyasn1 0.4.8
pyasn1-modules 0.2.7
pycamhd 0.7.0
pycparser 2.20
pyct 0.4.6
pycurl 7.43.0.6
pydantic 1.8.2
Pydap 3.2.2
pyerfa 1.7.3
Pygments 2.9.0
pyhdf 0.10.3
PyJWT 2.1.0
pykdtree 1.3.4
pymbolic 2021.1
PyNaCl 1.4.0
pyOpenSSL 20.0.1
pyorbital 1.6.1
pyparsing 2.4.7
pyproj 3.0.1
PyQt5 5.12.3
PyQt5-sip 4.19.18
PyQtChart 5.12
PyQtWebEngine 5.12.1
pyresample 1.19.0
pyrsistent 0.17.3
pyshp 2.1.3
PySocks 1.7.1
pyspectral 0.10.5
pystac 0.5.6
pystac-client 0.1.1
python-box 5.3.0
python-dateutil 2.7.5
python-dotenv 0.17.1
python-editor 1.0.4
python-geotiepoints 1.2.1
python-gist 0.10.4
python-gnupg 0.4.7
python-json-logger 2.0.1
python-slugify 5.0.2
python-utils 2.5.6
pytools 2021.2.6
pytz 2021.1
pytzdata 2020.1
pyviz-comms 2.0.1
PyWavelets 1.1.1
PyYAML 5.4.1
pyzmq 22.0.3
qgrid 1.3.1
rasterio 1.2.3
rechunker 0.4.2
redis 3.5.3
requests 2.25.1
requests-oauthlib 1.3.0
retrying 1.3.3
rio-cogeo 2.0.1
rioxarray 0.4.0
rsa 4.7.2
Rtree 0.9.7
ruamel.yaml 0.17.4
ruamel.yaml.clib 0.2.2
s3fs 2021.5.0
s3transfer 0.4.2
sat-search 0.3.0
sat-stac 0.4.1
satpy 0.28.1
scikit-image 0.18.1
scikit-learn 0.24.2
scipy 1.6.3
seaborn 0.11.2
Send2Trash 1.5.0
setuptools 49.6.0.post20210108
Shapely 1.7.1
simpervisor 0.4
six 1.16.0
smmap 3.0.5
sniffio 1.2.0
snuggs 1.4.7
sortedcontainers 2.4.0
soupsieve 2.0.1
sparse 0.12.0
spatialpandas 0.3.6
SQLAlchemy 1.4.15
sshtunnel 0.3.1
starlette 0.14.2
statsmodels 0.12.2
strict-rfc3339 0.7
supermercado 0.2.0
tabulate 0.8.9
tblib 1.7.0
terminado 0.10.0
testpath 0.5.0
text-unidecode 1.3
threadpoolctl 2.1.0
tifffile 2021.4.8
tiledb 0.8.9
toml 0.10.2
toolz 0.11.1
tornado 6.1
tqdm 4.60.0
traitlets 5.0.5
traittypes 0.2.1
trollimage 1.15.0
trollsift 0.3.5
typing-extensions 3.7.4.3
Unidecode 1.2.0
urllib3 1.26.4
uvicorn 0.13.4
uvloop 0.15.2
vine 5.0.0
voila 0.2.10
watchgod 0.6
wcwidth 0.2.5
webencodings 0.5.1
WebOb 1.8.7
websocket-client 0.57.0
websockets 8.1
wheel 0.36.2
widgetsnbextension 3.5.1
wrapt 1.12.1
xarray 0.18.2
xarray-leaflet 0.1.13
xarray-spatial 0.2.2
xarrayutils 0.1.3
xcape 0+unknown
xcube 0.8.1
xesmf 0.5.3
xgcm 0.5.1
xhistogram 0.2.0
xlayers 0.2.2
xmitgcm 0.5.1
xpublish 0.1.0
xrft 0.3.0
yarl 1.6.3
zarr 2.8.3
zict 2.0.0
zipp 3.4.1
zope.component 5.0.1
zope.deprecation 4.4.0
zope.event 4.5.0
zope.hookable 5.1.0
zope.interface 5.4.0
zope.lifecycleevent 4.3
playing with https://github.com/LinkedEarth/PaleoBooks/blob/master/notebooks/EDC_demo.ipynb
sep="/s+"seems to load the txt correctly