exception when strava fetch : java.lang.IndexOutOfBoundsException: 1

[b3b00]

i encountered an exception when trying to strava-pr strava fetch : ./strava-pr strava fetch Fetched run of 2014-07-17.

Exception in thread "main" java.lang.IndexOutOfBoundsException: 1
    at scala.collection.LinearSeqOptimized.apply(LinearSeqOptimized.scala:63)
    at scala.collection.LinearSeqOptimized.apply$(LinearSeqOptimized.scala:61)
    at scala.collection.immutable.List.apply(List.scala:86)
    at stravapr.Strava.activityToRun(Strava.scala:50)
    at stravapr.Strava.$anonfun$populateRunCache$6(Strava.scala:69)
    at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:234)
    at scala.collection.immutable.Set$Set1.foreach(Set.scala:95)
    at scala.collection.TraversableLike.map(TraversableLike.scala:234)
    at scala.collection.TraversableLike.map$(TraversableLike.scala:227)
    at scala.collection.AbstractSet.scala$collection$SetLike$$super$map(Set.scala:47)
    at scala.collection.SetLike.map(SetLike.scala:101)
    at scala.collection.SetLike.map$(SetLike.scala:101)
    at scala.collection.AbstractSet.map(Set.scala:47)
    at stravapr.Strava.$anonfun$populateRunCache$5(Strava.scala:69)
    at scala.Option.orElse(Option.scala:289)
    at stravapr.Strava.$anonfun$populateRunCache$4(Strava.scala:68)
    at stravapr.Strava.$anonfun$populateRunCache$4$adapted(Strava.scala:67)
    at scala.collection.immutable.List.flatMap(List.scala:335)
    at stravapr.Strava.populateRunCache(Strava.scala:67)
    at stravapr.Main$.stravaFetch(Main.scala:69)
    at stravapr.Main$.$anonfun$main$1(Main.scala:315)
    at stravapr.Main$.$anonfun$main$1$adapted(Main.scala:314)
    at scala.Option.foreach(Option.scala:257)
    at stravapr.Main$.main(Main.scala:314)
    at stravapr.Main.main(Main.scala)

I can send you my personnal access token if you'd like to debug it. My scala skills are too weak to debug it myself. Your project seems to be exactly what I was looking for , great job.

Olivier Duhart

[orium]

Hi Olivier,

Thanks for the bug report. I will look into it.

I can send you my personnal access token if you'd like to debug it.

That would be the easiest way. You can send it to my email (diogoremovemegsousa@gmail.com, but without the removeme part.).

[orium]

ping

[b3b00]

@orium , did you receive my strava token ? I sent it some days ago ?

[orium]

I didn't. I just received an email today.

[orium]

The problem seems to be that there is a run with no distance measurements. I have fixed the code to ignore those run, and will do a release soon.

[orium]

I'm releasing 0.2.1 now. It works, but some commands require a lot of time. For instance show takes about 9 minutes for your runs. history didn't even finish because I got out of memory.

I will open issues to address these limitations in the future.

[orium]

Release: https://github.com/orium/strava-pr/releases/tag/v0.2.1

[b3b00]

hello @orium ,

I 've just tried yur new release. strava fetch works better, not exiting with outofbound exception. It starts fetching but failed with a java.net.SocketTimeoutException: connect timed out Is there a way to workaround this failure downloading by hand all my GPX and storing them in the right place ?

At some point it states "Rate limit exceeded. Sleeping for 1 minute...." , it may be related.

a complete stacktrace :

Exception in thread "main" java.lang.RuntimeException: Could not parse personal activitySummary
    at kiambogo.scrava.ScravaClient.retrieveActivity(ScravaClient.scala:260)
    at stravapr.Strava.$anonfun$fetchRunActivity$1(Strava.scala:34)
    at stravapr.RateLimiter.run(RateLimiter.scala:25)
    at stravapr.RateLimiter.apply(RateLimiter.scala:38)
    at stravapr.Strava.fetchRunActivity(Strava.scala:34)
    at stravapr.Strava.$anonfun$populateRunCache$5(Strava.scala:80)
    at scala.Option.orElse(Option.scala:289)
    at stravapr.Strava.$anonfun$populateRunCache$4(Strava.scala:79)
    at stravapr.Strava.$anonfun$populateRunCache$4$adapted(Strava.scala:78)
    at scala.collection.immutable.List.flatMap(List.scala:335)
    at stravapr.Strava.populateRunCache(Strava.scala:78)
    at stravapr.Main$.stravaFetch(Main.scala:69)
    at stravapr.Main$.$anonfun$main$1(Main.scala:317)
    at stravapr.Main$.$anonfun$main$1$adapted(Main.scala:316)
    at scala.Option.foreach(Option.scala:257)
    at stravapr.Main$.main(Main.scala:316)
    at stravapr.Main.main(Main.scala)
Caused by: kiambogo.scrava.models.RateLimitException
    at kiambogo.scrava.ScravaClient.parseWithRateLimits(ScravaClient.scala:690)
    at kiambogo.scrava.ScravaClient.$anonfun$retrieveActivity$2(ScravaClient.scala:257)
    at scala.util.Try$.apply(Try.scala:209)
    at kiambogo.scrava.ScravaClient.retrieveActivity(ScravaClient.scala:257)
    ... 16 more

[orium]

Is there a way to workaround this failure downloading by hand all my GPX and storing them in the right place ?

Unfortunatly not. We don't process GPX, we process something more high-level informaittion that strava exposes.

At some point it states "Rate limit exceeded. Sleeping for 1 minute...." , it may be related.

It is. Apparently the way we detect a rate limit exceeded is not good enough. I will fix this soon, however, you should be able to run the fetch until there is nothing left to fetch.

[orium]

New release: https://github.com/orium/strava-pr/releases/download/v0.2.2/strava-pr-0.2.2.txz

[b3b00]

Thanks for your reactivity @orium . it works better. I guess half my activities are fetched sadly still an exception occured before the end of the fetch and no cache is saved in ~ /.config/strava-pr I am trying to set an intellijidea dev env to find if I can help you with this. But I am strugling with intellij to just be able to debug.

Fetched run of 2016-10-06. Rate limit exceeded. Sleeping for 1 minute... Rate limit exceeded. Sleeping for 2 minutes... Rate limit exceeded. Sleeping for 4 minutes... Rate limit exceeded. Sleeping for 8 minutes... Exception in thread "main" java.net.SocketTimeoutException: connect timed out at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:673) at sun.net.NetworkClient.doConnect(NetworkClient.java:175) at sun.net.www.http.HttpClient.openServer(HttpClient.java:463) at sun.net.www.http.HttpClient.openServer(HttpClient.java:558) at sun.net.www.protocol.https.HttpsClient.(HttpsClient.java:264) at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191) at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177) at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162) at scalaj.http.DefaultConnectFunc$.apply(Http.scala:471) at scalaj.http.DefaultConnectFunc$.apply(Http.scala:469) at scalaj.http.HttpRequest.doConnection(Http.scala:355) at scalaj.http.HttpRequest.exec(Http.scala:335) at scalaj.http.HttpRequest.asString(Http.scala:455) at kiambogo.scrava.ScravaClient.parseWithRateLimits(ScravaClient.scala:687) at kiambogo.scrava.ScravaClient.retrieveActivity(ScravaClient.scala:248) at stravapr.Strava.$anonfun$fetchRunActivity$1(Strava.scala:34) at stravapr.RateLimiter.run(RateLimiter.scala:26) at stravapr.RateLimiter.run(RateLimiter.scala:33) at stravapr.RateLimiter.run(RateLimiter.scala:33) at stravapr.RateLimiter.run(RateLimiter.scala:33) at stravapr.RateLimiter.run(RateLimiter.scala:33) at stravapr.RateLimiter.apply(RateLimiter.scala:39) at stravapr.Strava.fetchRunActivity(Strava.scala:34) at stravapr.Strava.$anonfun$populateRunCache$5(Strava.scala:80) at scala.Option.orElse(Option.scala:289) at stravapr.Strava.$anonfun$populateRunCache$4(Strava.scala:79) at stravapr.Strava.$anonfun$populateRunCache$4$adapted(Strava.scala:78) at scala.collection.immutable.List.flatMap(List.scala:335) at stravapr.Strava.populateRunCache(Strava.scala:78) at stravapr.Main$.stravaFetch(Main.scala:69) at stravapr.Main$.$anonfun$main$1(Main.scala:317) at stravapr.Main$.$anonfun$main$1$adapted(Main.scala:316) at scala.Option.foreach(Option.scala:257) at stravapr.Main$.main(Main.scala:316) at stravapr.Main.main(Main.scala)

[b3b00]

I get my intellij ok. As a first approach I forced strava-pr to save run-cache file at every activity fetch. of course it slows down the whole fetching but at list data is saved. after the 15min strava rate limit is reached we have to wait until the next 0 15 30 or 45 minutes in the hour to be able to restart the fetch. Works well by hand : now fetching another chunk of activities. I will try to work on a better strava ratelimiter to be able to conform to the strava limits. Will submit you a PR as soon as I feel ok with my work. I will rework the run-cache save to write only every 50 or so activities.

[orium]

So, aparently the ScravaClient will not recover from a timeout, which means that we should probably have logic so that when the ScravaClient throws a timeout exception we could recover by reconnecting.

Also, the "save activities every 50 runs" helps. I think we should also store what we have in case of an exception (printing an error saying that this is partial and the exception).

Also, I appreciate the initiative to go for a PR. Thanks for that! :)