Streaming API incomplete tweet

[bcambel]

Hi Adam,

Great library, thank you very much!

I am having an issue where the received tweet stream is incomplete. In some cases, the stream is properly JSON parsed though. Here is the code that I use

(defn parse-tweet
  [resp tweet]
  (try 
    (let [tw-obj (parse-string tweet true)]
      (when (:text tw-obj)
        (log/info (select-keys tw-obj [:id :text]))))
    (catch Throwable t 
      (log/error (.getMessage t) (format "Can't parse tweet\n%s" tweet)))))

(defn vacuum-twttr [tracking]
  (statuses-filter :params {:track tracking}
             :oauth-creds my-creds
             :callbacks (AsyncStreamingCallback. 
                            (fn [resp tweet]
                                (parse-tweet resp (str tweet)))
                          (comp println response-return-everything)
                          exception-print)))

My assumption is that somehow using str over ByteArrayOutputStream creates the problem. Do you have any suggestions ?

Thanks a lot, Bahadir

[adamwynne]

I think it being chunked by twitter - someone wrote some library (forgotten which) that sits on top of mine and does the correct message chunking - take a look at the issues and it should be there

On Thu, Dec 11, 2014 at 11:03 AM, Bahadir Cambel notifications@github.com wrote:

Hi Adam,

Great library, thank you very much!

I am having an issue where the received tweet stream is incomplete. In some cases, the stream is properly JSON parsed though. Here is the code that I use

(defn parse-tweet [resp tweet](try %28let [tw-obj %28parse-string tweet true%29] %28when %28:text tw-obj%29 %28log/info %28select-keys tw-obj [:id :text]%29%29%29%29 %28catch Throwable t %28log/error %28.getMessage t%29 %28format "Can't parse tweetn%s" tweet)))))

(defn vacuum-twttr [tracking](statuses-filter :params {:track tracking} :oauth-creds my-creds :callbacks %28AsyncStreamingCallback. %28fn [resp tweet] %28parse-tweet resp %28str tweet%29%29%29 %28comp println response-return-everything%29 exception-print%29))

My assumption is that somehow using str over ByteArrayOutputStream creates the problem. Do you have any suggestions ?

Thanks a lot, Bahadir

— Reply to this email directly or view it on GitHub https://github.com/adamwynne/twitter-api/issues/58.

[bcambel]

Thanks for the quick response Adam. For future reference, here is the library https://github.com/mccraigmccraig/twitter-streaming-client

[adamwynne]

thanks for pointing it out

On Thu, Dec 11, 2014 at 12:32 PM, Bahadir Cambel notifications@github.com wrote:

Thanks for the quick response Adam. For future reference, here is the library https://github.com/mccraigmccraig/twitter-streaming-client

— Reply to this email directly or view it on GitHub https://github.com/adamwynne/twitter-api/issues/58#issuecomment-66612626 .