Session is closed while creating link, target: test_purge

[KevinGossentCap]

Hi, I'm actually trying to make use of RabbitMQ with a "Informatica Cloud Real Time" iPaaS plateform. This product is implementing AMQP 1-0 through Java QPid (I don't know the version yet but I'll know soon).

I'm able to have a listener getting messages without any problem. The problem is when it tries to write to a queue (any queue).

If it can help here are what gives the 2 Rabbit log files. log:

=INFO REPORT==== 21-Jul-2016::10:42:07 ===
accepting AMQP connection <0.6179.0> (127.0.0.1:56312 -> 127.0.0.1:5672)

=ERROR REPORT==== 21-Jul-2016::10:42:07 ===
** Generic server <0.6185.0> terminating
** Last message in was {'$gen_cast',
                        {frame,
                         {'v1_0.attach',
                          {utf8,
                           <<"test_purge<-79b9fd92-00f1-4a43-849e-7cd71e66c11a">>},
                          {uint,0},
                          false,
                          {ubyte,0},
                          {ubyte,0},
                          {'v1_0.source',
                           {utf8,<<"79b9fd92-00f1-4a43-849e-7cd71e66c11a">>},
                           undefined,undefined,undefined,undefined,undefined,
                           undefined,undefined,
                           {'v1_0.accepted'},
                           {list,
                            [{symbol,<<"amqp:accepted:list">>},
                             {symbol,<<"amqp:rejected:list">>}]},
                           undefined},
                          {'v1_0.target',
                           {utf8,<<"test_purge">>},
                           undefined,undefined,undefined,undefined,undefined,
                           undefined},
                          undefined,undefined,
                          {uint,0},
                          undefined,undefined,undefined,undefined},
                         <0.6179.0>}}
** When Server state == {state,<0.6188.0>,<0.6194.0>,65528,<0.6179.0>,
                            <0.6184.0>,
                            {[],[]},
                            {session,0,2048,2048,0,65535,65535,0,65535,65535,
                                0,0,
                                {0,nil},
                                {0,nil}}}
** Reason for termination == 
** {function_clause,
       [{rabbit_amqp1_0_link_util,'-outcomes/1-lc$^0/1-0-',
            [{list,
                 [{symbol,<<"amqp:accepted:list">>},
                  {symbol,<<"amqp:rejected:list">>}]}],
            []},
        {rabbit_amqp1_0_link_util,outcomes,1,[]},
        {rabbit_amqp1_0_incoming_link,attach,3,[]},
        {rabbit_amqp1_0_session_process,with_disposable_channel,2,[]},
        {rabbit_amqp1_0_session_process,handle_control,2,[]},
        {rabbit_amqp1_0_session_process,handle_cast,2,[]},
        {gen_server2,handle_msg,2,[]},
        {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}

=ERROR REPORT==== 21-Jul-2016::10:42:10 ===
closing AMQP connection <0.6179.0> (127.0.0.1:56312 -> 127.0.0.1:5672):
{handshake_error,running,<0.6185.0>,
    {{symbol,<<"amqp:internal-error">>},
     "Session error: ~p~n~p~n",
     [function_clause,
      [{rabbit_amqp1_0_link_util,'-outcomes/1-lc$^0/1-0-',
           [{list,
                [{symbol,<<"amqp:accepted:list">>},
                 {symbol,<<"amqp:rejected:list">>}]}],
           []},
       {rabbit_amqp1_0_link_util,outcomes,1,[]},
       {rabbit_amqp1_0_incoming_link,attach,3,[]},
       {rabbit_amqp1_0_session_process,with_disposable_channel,2,[]},
       {rabbit_amqp1_0_session_process,handle_control,2,[]},
       {rabbit_amqp1_0_session_process,handle_cast,2,[]},
       {gen_server2,handle_msg,2,[]},
       {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]]}}

sasl.log

=CRASH REPORT==== 21-Jul-2016::10:42:07 ===
  crasher:
    initial call: gen:init_it/6
    pid: <0.6185.0>
    registered_name: []
    exception exit: {function_clause,
                        [{rabbit_amqp1_0_link_util,'-outcomes/1-lc$^0/1-0-',
                             [{list,
                                  [{symbol,<<"amqp:accepted:list">>},
                                   {symbol,<<"amqp:rejected:list">>}]}],
                             []},
                         {rabbit_amqp1_0_link_util,outcomes,1,[]},
                         {rabbit_amqp1_0_incoming_link,attach,3,[]},
                         {rabbit_amqp1_0_session_process,
                             with_disposable_channel,2,[]},
                         {rabbit_amqp1_0_session_process,handle_control,2,[]},
                         {rabbit_amqp1_0_session_process,handle_cast,2,[]},
                         {gen_server2,handle_msg,2,[]},
                         {proc_lib,init_p_do_apply,3,
                             [{file,"proc_lib.erl"},{line,240}]}]}
      in function  gen_server2:terminate/3 
    ancestors: [<0.6183.0>,<0.6180.0>,<0.6178.0>,<0.6177.0>,
                  rabbit_tcp_client_sup,rabbit_sup,<0.159.0>]
    messages: []
    links: [<0.6183.0>]
    dictionary: [{credit_flow_default_credit,{200,50}},
                  {{credit_to,<0.6179.0>},48}]
    trap_exit: true
    status: running
    heap_size: 987
    stack_size: 27
    reductions: 545
  neighbours:

=SUPERVISOR REPORT==== 21-Jul-2016::10:42:07 ===
     Supervisor: {<0.6183.0>,rabbit_amqp1_0_session_sup}
     Context:    child_terminated
     Reason:     {function_clause,
                     [{rabbit_amqp1_0_link_util,'-outcomes/1-lc$^0/1-0-',
                          [{list,
                               [{symbol,<<"amqp:accepted:list">>},
                                {symbol,<<"amqp:rejected:list">>}]}],
                          []},
                      {rabbit_amqp1_0_link_util,outcomes,1,[]},
                      {rabbit_amqp1_0_incoming_link,attach,3,[]},
                      {rabbit_amqp1_0_session_process,
                          with_disposable_channel,2,[]},
                      {rabbit_amqp1_0_session_process,handle_control,2,[]},
                      {rabbit_amqp1_0_session_process,handle_cast,2,[]},
                      {gen_server2,handle_msg,2,[]},
                      {proc_lib,init_p_do_apply,3,
                          [{file,"proc_lib.erl"},{line,240}]}]}
     Offender:   [{pid,<0.6185.0>},
                  {name,channel},
                  {mfargs,
                      {rabbit_amqp1_0_session_process,start_link,
                          [{0,<0.6179.0>,<0.6184.0>,
                            {user,<<"sfdc">>,
                                [administrator],
                                [{rabbit_auth_backend_internal,none}]},
                            <<"/">>,65528,
                            {amqp_adapter_info,
                                {127,0,0,1},
                                5672,
                                {127,0,0,1},
                                56312,<<"127.0.0.1:56312 -> 127.0.0.1:5672">>,
                                {'AMQP',"1.0"},
                                [{ssl,false}]},
                            <0.6181.0>}]}},
                  {restart_type,intrinsic},
                  {shutdown,4294967295},
                  {child_type,worker}]

=SUPERVISOR REPORT==== 21-Jul-2016::10:42:07 ===
     Supervisor: {<0.6183.0>,rabbit_amqp1_0_session_sup}
     Context:    shutdown
     Reason:     reached_max_restart_intensity
     Offender:   [{pid,<0.6185.0>},
                  {name,channel},
                  {mfargs,
                      {rabbit_amqp1_0_session_process,start_link,
                          [{0,<0.6179.0>,<0.6184.0>,
                            {user,<<"sfdc">>,
                                [administrator],
                                [{rabbit_auth_backend_internal,none}]},
                            <<"/">>,65528,
                            {amqp_adapter_info,
                                {127,0,0,1},
                                5672,
                                {127,0,0,1},
                                56312,<<"127.0.0.1:56312 -> 127.0.0.1:5672">>,
                                {'AMQP',"1.0"},
                                [{ssl,false}]},
                            <0.6181.0>}]}},
                  {restart_type,intrinsic},
                  {shutdown,4294967295},
                  {child_type,worker}]

Hoping to find a way to make it work. rabbitmq.config.txt

[michaelklishin]

How can we reproduce with a released Qpid client version?

[KevinGossentCap]

I know for sure it's Qpid Java client and only the version 1.0 of the AMQP protocol. As soon as I have the needed additional information from Informatica (should be today) I'll report here or have them report directly.

[KevinGossentCap]

I've gotten some information. Here is the exact platform used by their implementation : https://mvnrepository.com/artifact/org.apache.camel/camel-amqp/2.14.3

To test/debug, I've gone down to doing the following scenario in their tool:

• in RabbitMQ (3.5.6 & ERLang 18.1) creating a new user admin
• in RabbitMQ not using ssl, using standard ports, 1-0 pluging enabled
• in tool programming following scenario: . 1 listener on a first queue expeting a JSON . taking the message . sending the message to a second queue
• in RabbitMQ from management website: inserting a JSON message inside the first queue

What I see is:

• the message is taken
• the platform errors on sending to second queue
• the platform ACK the message inputted nevertheless

I'll continue to fill the gap with any information you could ask for which I have/receive answer hoping one of their dev team will afterwards take the lead if/when needed.

[michaelklishin]

Can you please try 3.6.3? There weren't many changes in this plugin since 3.5.6 but there were some, e.g. https://github.com/rabbitmq/rabbitmq-amqp1.0/commit/d265b45f8ba7ce71c6768f30e78dd2604d035a22.

[KevinGossentCap]

Ok, I'll test with 3.6.3 (but I have tested 2 days ago with 3.6.2 and had same issue).

An other possibility would be that I create an account on the Informatica plateform for you to take/install an agent to run on you side and we could make the change for the scenario to be able to run on your side at will. (and then see maybe more information on RabbitMQ's side that what I can see myself)

[michaelklishin]

Thank you for the offer, we will have to find a way to reproduce with straight Qpid, so I'd focus on that.

On Thu, Jul 21, 2016 at 2:53 PM, KevinGossentCap notifications@github.com wrote:

Ok, I'll test with 3.6.3 (but I have tested 2 days ago with 3.6.2 and had same issue).

An other possibility would be that I create an account on the Informatica plateform for you to take/install an agent to run on you side and we could make the change for the scenario to be able to run on your side at will. (and then see maybe more information on RabbitMQ's side that what I can see myself)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rabbitmq/rabbitmq-amqp1.0/issues/31#issuecomment-234232064, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAEQs25v7qfMeFnuumHzdj4EUFndWapks5qX123gaJpZM4JRmNj .

MK

Staff Software Engineer, Pivotal/RabbitMQ

[KevinGossentCap]

I transmitted the request regarding the need to reproduce it in pure Qpid. In the meantime if it can help, the product itself is a BPMN product and here is the BPEL definition of the process. test-Queue-to-Queue.zip

[KevinGossentCap]

Hi @michaelklishin , Here is @tjohnsoninformatica from Informatica with which I'm interracting. he'll be going with you on the subject to have a process working so that we can all find the thing which is not doing as expected and in the end making the connection work. I'll let him express how we propose to build the testing scenario, it will imply some actions on both sides.

Hoping to have some incites soon :)

[tjohnsoninformatica]

Hi @michaelklishin , In order for you guys to test what we're sending, which is causing an issue, I'll need to setup a webservice which you can call, which posts the payload to a RabbitMQ queue which you provide. Please could you set up a queue which we can access from the internet to post to. Once I have that, I can create the required process and send you the details.

Regards Trevor

[tjohnsoninformatica]

Hi @michaelklishin , Any update on the queue needed to reproduce the issue?

Regards Trevor

[dumbbell]

Hi! It looks like the attach method's source specifies a list of outcomes and it hits a bug in the plugin. I could reproduce it by manually calling the function. Now I'm trying to reproduce with a real connection.

[tjohnsoninformatica]

Hey, let me know if you need me to set something up to reproduce the issue using our systems. I can easily create a process that would be exposed of ReST which could put a message on your queue. All I'd need is the queue to be made public on the internet.

[michaelklishin]

I believe we have a good enough understanding of the problem to come up with a test case. A new test suite that was merged last week, which also helps.

[dumbbell]

Hi @KevinGossentCap and @tjohnsoninformatica! I pushed a patch for this issue in the branch linked above this comment.

Could you please try it and tell me if it fixes your issue?

[michaelklishin]

Note that we can produce a one-off plugin or even server build if necessary, let us know. Thank you.

On Tue, Jul 26, 2016 at 1:19 PM, Jean-Sébastien Pédron < notifications@github.com> wrote:

Hi @KevinGossentCap https://github.com/KevinGossentCap and @tjohnsoninformatica https://github.com/tjohnsoninformatica! I pushed a patch for this issue in the branch linked above this comment.

Could you please try it and tell me if it fixes your issue?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rabbitmq/rabbitmq-amqp1.0/issues/31#issuecomment-235227997, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAEQqC9Xjw_Lg60Z9wsRAqs-hcnsS-Eks5qZd83gaJpZM4JRmNj .

MK

Staff Software Engineer, Pivotal/RabbitMQ

[KevinGossentCap]

Hi @michaelklishin and @dumbbell , Yes, having a one-off plugin to test how it works on our end would help. With it I'll see how exactly to replace the included plugin with the one-off one and test if it solves the whole problem. regards

[dumbbell]

GitHub doesn't let me attach the plugin, so I followed up by email.

You can drop the plugin in the RabbitMQ plugins directory. You must also remove the existing AMQP 1.0 plugin. Note that the existing plugin will have a version number in its filename, unlike the one I sent by email.

[KevinGossentCap]

Hi @michaelklishin , @dumbbell , I tested the the plugin with Rabbit 3.6.3 and we are now able to push messages in Queues :+1: . For testing purpose I'll also test it with Rabbit 3.5.6 to see if it's Rabbit version indépendant (which would also be a great news). Thanks for your work.

[michaelklishin]

3.6.x plugins are not generally expected to work with 3.5.x but it might work in this case.

[bmantha]

Hi ,

I downloaded RabbitMQ 3.6.12 and I have the same issue when it writes to a queue through 1.0 plugin.

Could someone please send me the patch . Thank you.