search

allenai / deep_qa

A deep NLP library, based on Keras / tf, focused on question answering (but useful for other NLP too)
Apache License 2.0
404 stars 133 forks source link

Investigate using cache better in CircleCI #329

Open matt-gardner opened 7 years ago

matt-gardner commented 7 years ago

Looks like our build scripts for CircleCI intentionally clear the cache before building the docs, which makes things take longer than they probably need to. It didn't use to matter much, because Travis was always way slower, but now sometimes CircleCI finishes after Travis. I don't know why clearing the cache is necessary; @nelson-liu, any particular reason you put that in there?

nelson-liu commented 7 years ago

yes.

we use a conda environment to build the docs, so rm -rf ~/.pyenv && rm -rf ~/virtualenvs is just to prevent any PATH conflicting things from inadvertently happening.

iirc to render math, I had to remove the binaries for tex-live and install it through that long command below; i'll look into alternatives.

to clarify: the line of code you linked does not remove anything that is cached that we use. I think the only thing we cache is the miniconda download, so caching apt-get stuff might help.

nelson-liu commented 7 years ago

in fact, 'texlive-binaries' isn't even installed by default in circle, so that command shouldn't really affect us; i'm guessing its the fact that we have to install those tex packages from apt-get...so caching those might help.

matt-gardner commented 7 years ago

Ok, thanks. This isn't super high priority, and we can fix it, so don't worry about it unless you really want to. I was just looking for some more context.