http_error on every API call

[Fodoj]

Every API call I make gives me RouteError (http_error). After a bit of debugging I found out that the actual exception is #<Netrc::Error: unexpected EOF>. Tried re-generating API tokens but error is still there. Version 1.1.1.

[valeriansaliou]

\cc @baptistejamin

[baptistejamin]

Hello @Fodoj

I digged into your issue and I tried with the 1.1.1 release, using ruby v2.3.0 with the following sample:

require 'crisp-api'

client = Crisp::Client.new

client.authenticate("IDENTIFIER", "KEY")

client.website.list_conversations("WEBSITE_ID", 1)

puts(list)

I didnt have any issue using this sample. Could you try and let me know?

[Fodoj]

@baptistejamin that's exactly what I tried to do :( Could it be that my token or Crisp account has some issues? We are still on trial, but I could generate tokens just fine. Ruby 2.5 here.

[baptistejamin]

Is this using development? production?

Do you have a proxy? VPN? Firewall?

A token should return a completely different error.

[Fodoj]

No proxy, no firewall. I did not find any info in Crisp docs regarding development or production environments, it seems like I can only use tokens against my Crisp website.

[baptistejamin]

My question was more if you are running the code on a Heroku or on your own computer.

[Fodoj]

My personal laptop Am 28. März 2019, 18:48 +0100 schrieb Baptiste Jamin notifications@github.com:

My question was more if you are running the code on a Heroku or on your own computer. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[Fodoj]

I just tried generating new token and running code sample you provided above, same error. This is blocking as from switching to Crisp, as only step left is integration with out main backend :(

[valeriansaliou]

What's your server's / or / personal laptop outbound IP address? We can check our edges logs.

(in other words I need the outbound IP for which this does not work)

[Fodoj]

@valeriansaliou 185.17.207.5. That's my house IP, from ISP.

[valeriansaliou]

Thanks. When did you last try a request from the library? I'm not finding your IP in any edge logs. Do you also happen to run on IPv6? Check: http://ipv6-test.com

[Fodoj]

I do have ipv6 address too. Last attempt was around 2 hours ago

[valeriansaliou]

What's your IPv6 address so that I check?

[Fodoj]

Won't be able to provide it till the evening sadly. I will also try to do simple curl query tonight to check if it's network issue.

[Fodoj]

I tried with curl as well as Crisp Python SDK. Both attempts returned new error:

>>> client.website.list_conversations(website_id, 0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/fodoj/.local/lib/python2.7/site-packages/crisp_api/resources/website.py", line 61, in list_conversations
    return self.parent.get(self.__url_website(website_id, "/conversations/%d" % page_number))
  File "/home/fodoj/.local/lib/python2.7/site-packages/crisp_api/__init__.py", line 55, in get
    return self.__do_request("GET", resource, query=query)
  File "/home/fodoj/.local/lib/python2.7/site-packages/crisp_api/__init__.py", line 92, in __do_request
    raise RouteError(result["reason"] if ("reason" in result) else "error")
crisp_api.errors.route.RouteError: invalid_session

Why's session invalid?

[Fodoj]

As for IPs:

IPv6: 2001:a61:480f:2201:c9d3:549:37f6:ecf1
IPv4: 185.17.207.5

[valeriansaliou]

Do you happen to run a script, somewhere, that creates a lot of sessions? We use a sliding window mechanism to invalidate old sessions, if you create too many of them. We allow up to 15 simultaneous sessions tokens to be active at any given time.

[Fodoj]

@valeriansaliou funny thing: I just created a new token pair and I finally have a working API call via python sdk. But Ruby SDK still returns http_error with exactly same token and same API call.

[valeriansaliou]

I've just tried your code, and... it succeeded, even with your token.

[Fodoj]

I am out of ideas then. I just run:

2.5.0 :001 > require 'crisp-api'
 => true 
2.5.0 :002 > CRISP = Crisp::Client.new
 => #<Crisp::Client:0x00000000028b4a28 @auth={}, @bucket=#<Crisp::BucketResource:0x00000000028b45a0 @parent=#<Crisp::Client:0x00000000028b4a28 ...>>, @website=#<Crisp::WebsiteResource:0x00000000028b4550 @parent=#<Crisp::Client:0x00000000028b4a28 ...>>> 
2.5.0 :003 > CRISP.authenticate("a5305c79-5e6d-4afa-a549-6ca61d9ca03c","625bc42632c7c5dc36b1354dbd875f5d70c77ebc4501df9c12bd64d764f32561")
 => "625bc42632c7c5dc36b1354dbd875f5d70c77ebc4501df9c12bd64d764f32561" 
2.5.0 :004 > CRISP.website.list_conversations("03014c76-9a44-4192-8fea-2333bb8407f6", 0)
Traceback (most recent call last):
        5: from /home/fodoj/.rvm/rubies/ruby-2.5.0/bin/irb:11:in `<main>'
        4: from (irb):4
        3: from /home/fodoj/.rvm/gems/ruby-2.5.0/gems/crisp-api-1.1.1/lib/resources/website.rb:85:in `list_conversations'
        2: from /home/fodoj/.rvm/gems/ruby-2.5.0/gems/crisp-api-1.1.1/lib/crisp-api.rb:50:in `get'
        1: from /home/fodoj/.rvm/gems/ruby-2.5.0/gems/crisp-api-1.1.1/lib/crisp-api.rb:114:in `_do_request'
RouteError (http_error)
2.5.0 :005 >

With Python it works.

[Fodoj]

Ok, I just tried to run exactly same code on an EC2 instance and it worked. Only difference I can think of is lack of IPv6 on EC2 instance.

[valeriansaliou]

Can you try again now with the Ruby API? I've lowered our bot protection system security level.

[Fodoj]

@valeriansaliou same same. It really seems some lower level network quirk. Perhaps Ruby lib used for requests doesnt handle ipv6 well or something similar..

[valeriansaliou]

I guess, but it's all out of our control there. We use the rest-client Gem for HTTP calls.

[Fodoj]

So also nothing in the logs?

[valeriansaliou]

Yes, nothing. The request is blocked before it hits our edges.

[valeriansaliou]

Note that I'm also running IPv6, and I can hit our APIs w/o any issue either w/ my token or your token:

⋊> ~/C/ruby-crisp-api on master ◦ ruby --version                                                                               08:52:34
ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-darwin16]

gem list | grep rest
rest-client (2.0.2)

[Fodoj]

This gets even better: it only happens under my personal system user. Running same code as root works fine.

[valeriansaliou]

I think we can conclude that your system forbids you to access the IPv6 networking stack at the kernel level 😉

[Fodoj]

I figured what the issue was but I am hesitant to share it here. But I think it is still a bug in the library. Can we have a short 5-10min call so that I explain it to you?

[valeriansaliou]

You can email valerian@crisp.chat and I'll have a review later on today / tomorrow.

[valeriansaliou]

I've deleted your comment giving away your API tokens, just to be sure nothing bad can happen.

[Fodoj]

So, root cause is the bug in RestClient: https://github.com/rest-client/rest-client/issues/524.

Would you be open to switching to HTTParty? :)

[valeriansaliou]

I think we can wait for rest-client version 2.1 in which this seems to have been fixed. I want to stick to this library.

If however you do find a fix in our library to ensure rest-client does not read the netrc, I'll happily accept your PRs.

[Fodoj]

Up to you of course, but it seems like 2.1 is not coming any time soon, it's stuck as rc since 2017:

[Fodoj]

Anyways, many thanks for prompt support. For time being I will update my .netrc file when working on Crips-related project. :)

[atinder]

I am facing the exact same issue. what is required in the .netrc file?