search

rivernews / review-scraper-java-development-environment

An environment to develop review scraper
0 stars 1 forks source link

Data Pipeline: compile java and only use jar in travis to improve spin up time #14

Closed rivernews closed 4 years ago

rivernews commented 4 years ago

Currently we install and compile in travis all over again. It might be great for early development.

But as things get stable, we want more performance. Currently it takes about 2 minutes to spin up the whole thing till the scraper run. Hopefully changing to JAR will improve the time.

rivernews commented 4 years ago

We were able to build and push to repo. But still not able to just install JRE instead of JDK in travisCI. But this indeed speeded up the time, e.g. originally 6 minutes task now only takes 5'30.