search

rabbitmq / rabbitmq-delayed-message-exchange

Delayed Messaging for RabbitMQ
Other
2.06k stars 150 forks source link

Compatibility with RabbitMQ 3.4.4? #11

Closed RmMsr closed 9 years ago

RmMsr commented 9 years ago

Hi, I am receiving an internal error whenever I publish a message to an x-delayed-message exchange. Server is RabbitMQ 3.4.4 on Erlang 17.5.

From /var/log/rabbitmq/rabbit\@vagrant-sasl.log

=CRASH REPORT==== 3-Jun-2015::09:00:39 ===
  crasher:
    initial call: gen:init_it/6
    pid: <0.970.0>
    registered_name: []
    exception exit: {function_clause,
                     [{rabbit_delayed_message_utils,get_msg,
                       [{delivery,true,true,<0.970.0>,
                         {basic_message,
                          {resource,<<"/$my_app">>,exchange,
                           <<"$my_exchange">>},
                          [<<>>],
                          {content,60,
                           {'P_basic',undefined,undefined,[],1,undefined,
                            undefined,undefined,undefined,undefined,undefined,
                            undefined,undefined,undefined,undefined},
                           none,none,
                           [<<"test">>]},
                          <<74,33,34,175,15,53,141,68,50,225,139,248,181,254,
                            94,133>>,
                          false},
                         1}],
                       []},
                      {lists,foldl,3,[{file,"lists.erl"},{line,1261}]},
                      {rabbit_delayed_message_utils,get_delay,1,[]},
                      {rabbit_exchange_type_delayed_message,delay_message,2,
                       []},
                      {rabbit_exchange_type_delayed_message,route,2,[]},
                      {rabbit_exchange,route1,3,[]},
                      {rabbit_exchange,route,2,[]},
                      {rabbit_channel,handle_method,3,[]}]}
      in function  gen_server2:terminate/3 
    ancestors: [<0.968.0>,rabbit_direct_client_sup,rabbit_sup,<0.150.0>]
    messages: []
    links: [<0.968.0>,<0.967.0>]
    dictionary: [{msg_size_for_gc,4},
                  {process_name,
                      {rabbit_channel,{<<"<rabbit@vagrant.2.961.0>">>,1}}},
                  {permission_cache,
                      [{{resource,<<"/$my_app">>,exchange,
                            <<"$my_exchange">>},
                        write}]},
                  {{xtype_to_module,'x-delayed-message'},
                   rabbit_exchange_type_delayed_message},
                  {guid,{{1243685551,255167812,853642232,3053346437},0}}]
    trap_exit: true
    status: running
    heap_size: 1598
    stack_size: 27
    reductions: 1715
  neighbours:
    neighbour: [{pid,<0.967.0>},
                  {registered_name,[]},
                  {initial_call,{amqp_channel,init,['Argument__1']}},
                  {current_function,{gen_server,loop,6}},
                  {ancestors,
                      [<0.965.0>,<0.962.0>,<0.960.0>,<0.959.0>,amqp_sup,
                       <0.53.0>]},
                  {messages,[]},
                  {links,[<0.965.0>,<0.970.0>]},
                  {dictionary,
                      [{process_name,
                           {amqp_channel,
                               {<<"<rabbit@vagrant.2.961.0>">>,1}}}]},
                  {trap_exit,false},
                  {status,waiting},
                  {heap_size,376},
                  {stack_size,9},
                  {reductions,215}]

=SUPERVISOR REPORT==== 3-Jun-2015::09:00:39 ===
     Supervisor: {<0.968.0>,rabbit_channel_sup}
     Context:    child_terminated
     Reason:     {function_clause,
                  [{rabbit_delayed_message_utils,get_msg,
                    [{delivery,true,true,<0.970.0>,
                      {basic_message,
                       {resource,<<"/$my_app">>,exchange,
                        <<"$my_exchange">>},
                       [<<>>],
                       {content,60,
                        {'P_basic',undefined,undefined,[],1,undefined,
                         undefined,undefined,undefined,undefined,undefined,
                         undefined,undefined,undefined,undefined},
                        none,none,
                        [<<"test">>]},
                       <<74,33,34,175,15,53,141,68,50,225,139,248,181,254,94,
                         133>>,
                       false},
                      1}],
                    []},
                   {lists,foldl,3,[{file,"lists.erl"},{line,1261}]},
                   {rabbit_delayed_message_utils,get_delay,1,[]},
                   {rabbit_exchange_type_delayed_message,delay_message,2,[]},
                   {rabbit_exchange_type_delayed_message,route,2,[]},
                   {rabbit_exchange,route1,3,[]},
                   {rabbit_exchange,route,2,[]},
                   {rabbit_channel,handle_method,3,[]}]}
     Offender:   [{pid,<0.970.0>},
                  {name,channel},
                  {mfargs,
                      {rabbit_channel,start_link,
                          [1,<0.967.0>,<0.967.0>,<0.961.0>,
                           <<"<rabbit@vagrant.2.961.0>">>,
                           rabbit_framing_amqp_0_9_1,
                           {user,<<"$my_app">>,
                               [administrator],
                               rabbit_auth_backend_internal,
                               {internal_user,<<"$my_app">>,
                                   <<98,157,153,175,250,176,64,34,246,54,121,
                                     205,158,209,254,205,51,33,24,177>>,
                                   [administrator]}},
                           <<"/$my_app">>,
                           [{<<"publisher_confirms">>,bool,true},
                            {<<"exchange_exchange_bindings">>,bool,true},
                            {<<"basic.nack">>,bool,true},
                            {<<"consumer_cancel_notify">>,bool,true},
                            {<<"connection.blocked">>,bool,true},
                            {<<"authentication_failure_close">>,bool,true}],
                           <0.964.0>,<0.969.0>]}},
                  {restart_type,intrinsic},
                  {shutdown,4294967295},
                  {child_type,worker}]

=SUPERVISOR REPORT==== 3-Jun-2015::09:00:39 ===
     Supervisor: {<0.968.0>,rabbit_channel_sup}
     Context:    shutdown
     Reason:     reached_max_restart_intensity
     Offender:   [{pid,<0.970.0>},
                  {name,channel},
                  {mfargs,
                      {rabbit_channel,start_link,
                          [1,<0.967.0>,<0.967.0>,<0.961.0>,
                           <<"<rabbit@vagrant.2.961.0>">>,
                           rabbit_framing_amqp_0_9_1,
                           {user,<<"$my_app">>,
                               [administrator],
                               rabbit_auth_backend_internal,
                               {internal_user,<<"$my_app">>,
                                   <<98,157,153,175,250,176,64,34,246,54,121,
                                     205,158,209,254,205,51,33,24,177>>,
                                   [administrator]}},
                           <<"/$my_app">>,
                           [{<<"publisher_confirms">>,bool,true},
                            {<<"exchange_exchange_bindings">>,bool,true},
                            {<<"basic.nack">>,bool,true},
                            {<<"consumer_cancel_notify">>,bool,true},
                            {<<"connection.blocked">>,bool,true},
                            {<<"authentication_failure_close">>,bool,true}],
                           <0.964.0>,<0.969.0>]}},
                  {restart_type,intrinsic},
                  {shutdown,4294967295},
                  {child_type,worker}]

=SUPERVISOR REPORT==== 3-Jun-2015::09:00:39 ===
     Supervisor: {<0.965.0>,amqp_channel_sup}
     Context:    child_terminated
     Reason:     {function_clause,
                  [{rabbit_delayed_message_utils,get_msg,
                    [{delivery,true,true,<0.970.0>,
                      {basic_message,
                       {resource,<<"/$my_app">>,exchange,
                        <<"$my_exchange">>},
                       [<<>>],
                       {content,60,
                        {'P_basic',undefined,undefined,[],1,undefined,
                         undefined,undefined,undefined,undefined,undefined,
                         undefined,undefined,undefined,undefined},
                        none,none,
                        [<<"test">>]},
                       <<74,33,34,175,15,53,141,68,50,225,139,248,181,254,94,
                         133>>,
                       false},
                      1}],
                    []},
                   {lists,foldl,3,[{file,"lists.erl"},{line,1261}]},
                   {rabbit_delayed_message_utils,get_delay,1,[]},
                   {rabbit_exchange_type_delayed_message,delay_message,2,[]},
                   {rabbit_exchange_type_delayed_message,route,2,[]},
                   {rabbit_exchange,route1,3,[]},
                   {rabbit_exchange,route,2,[]},
                   {rabbit_channel,handle_method,3,[]}]}
     Offender:   [{pid,<0.967.0>},
                  {name,channel},
                  {mfargs,
                      {amqp_channel,start_link,
                          [direct,<0.961.0>,1,<0.966.0>,
                           {<<"<rabbit@vagrant.2.961.0>">>,1}]}},
                  {restart_type,intrinsic},
                  {shutdown,4294967295},
                  {child_type,worker}]

=SUPERVISOR REPORT==== 3-Jun-2015::09:00:39 ===
     Supervisor: {<0.965.0>,amqp_channel_sup}
     Context:    shutdown
     Reason:     reached_max_restart_intensity
     Offender:   [{pid,<0.967.0>},
                  {name,channel},
                  {mfargs,
                      {amqp_channel,start_link,
                          [direct,<0.961.0>,1,<0.966.0>,
                           {<<"<rabbit@vagrant.2.961.0>">>,1}]}},
                  {restart_type,intrinsic},
                  {shutdown,4294967295},
                  {child_type,worker}]

From /var/log/rabbitmq/rabbit\@vagrant.log

=ERROR REPORT==== 3-Jun-2015::09:00:39 ===
** Generic server <0.970.0> terminating
** Last message in was {'$gen_cast',
                           {method,
                               {'basic.publish',0,
                                   <<"$my_exchange">>,<<>>,
                                   true,false},
                               {content,60,
                                   {'P_basic',undefined,undefined,[],1,
                                       undefined,undefined,undefined,
                                       undefined,undefined,undefined,
                                       undefined,undefined,undefined,
                                       undefined},
                                   none,none,
                                   [<<"test">>]},
                               noflow}}
** When Server state == {ch,running,rabbit_framing_amqp_0_9_1,1,<0.967.0>,
                            <0.967.0>,<0.961.0>,
                            <<"<rabbit@vagrant.2.961.0>">>,
                            {lstate,<0.969.0>,false},
                            none,1,
                            {[],[]},
                            {user,<<"$my_app">>,
                                [administrator],
                                rabbit_auth_backend_internal,
                                {internal_user,<<"$my_app">>,
                                    <<98,157,153,175,250,176,64,34,246,54,121,
                                      205,158,209,254,205,51,33,24,177>>,
                                    [administrator]}},
                            <<"/$my_app">>,<<>>,
                            {dict,0,16,16,8,80,48,
                                {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                 []},
                                {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                  [],[]}}},
                            {state,
                                {dict,0,16,16,8,80,48,
                                    {[],[],[],[],[],[],[],[],[],[],[],[],[],
                                     [],[],[]},
                                    {{[],[],[],[],[],[],[],[],[],[],[],[],[],
                                      [],[],[]}}},
                                erlang},
                            {dict,0,16,16,8,80,48,
                                {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                 []},
                                {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                  [],[]}}},
                            {dict,0,16,16,8,80,48,
                                {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                 []},
                                {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                  [],[]}}},
                            {set,0,16,16,8,80,48,
                                {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                 []},
                                {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                  [],[]}}},
                            <0.964.0>,
                            {state,fine,5000,#Ref<0.0.0.21372>},
                            true,1,
                            {{0,nil},{0,nil}},
                            [],
                            {{0,nil},{0,nil}},
                            [{<<"publisher_confirms">>,bool,true},
                             {<<"exchange_exchange_bindings">>,bool,true},
                             {<<"basic.nack">>,bool,true},
                             {<<"consumer_cancel_notify">>,bool,true},
                             {<<"connection.blocked">>,bool,true},
                             {<<"authentication_failure_close">>,bool,true}],
                            none,0,none}
** Reason for termination == 
** {function_clause,
       [{rabbit_delayed_message_utils,get_msg,
            [{delivery,true,true,<0.970.0>,
                 {basic_message,
                     {resource,<<"/$my_app">>,exchange,
                         <<"$my_exchange">>},
                     [<<>>],
                     {content,60,
                         {'P_basic',undefined,undefined,[],1,undefined,
                             undefined,undefined,undefined,undefined,
                             undefined,undefined,undefined,undefined,
                             undefined},
                         none,none,
                         [<<"test">>]},
                     <<74,33,34,175,15,53,141,68,50,225,139,248,181,254,94,133>>,
                     false},
                 1}],
            []},
        {lists,foldl,3,[{file,"lists.erl"},{line,1261}]},
        {rabbit_delayed_message_utils,get_delay,1,[]},
        {rabbit_exchange_type_delayed_message,delay_message,2,[]},
        {rabbit_exchange_type_delayed_message,route,2,[]},
        {rabbit_exchange,route1,3,[]},
        {rabbit_exchange,route,2,[]},
        {rabbit_channel,handle_method,3,[]}]}

=WARNING REPORT==== 3-Jun-2015::09:00:39 ===
Connection (<0.961.0>) closing: internal error in channel (<0.967.0>): {function_clause,
                                                                        [{rabbit_delayed_message_utils,
                                                                          get_msg,
                                                                          [{delivery,
                                                                            true,
                                                                            true,
                                                                            <0.970.0>,
                                                                            {basic_message,
                                                                             {resource,
                                                                              <<"/$my_app">>,
                                                                              exchange,
                                                                              <<"$my_exchange">>},
                                                                             [<<>>],
                                                                             {content,
                                                                              60,
                                                                              {'P_basic',
                                                                               undefined,
                                                                               undefined,
                                                                               [],
                                                                               1,
                                                                               undefined,
                                                                               undefined,
                                                                               undefined,
                                                                               undefined,
                                                                               undefined,
                                                                               undefined,
                                                                               undefined,
                                                                               undefined,
                                                                               undefined,
                                                                               undefined},
                                                                              none,
                                                                              none,
                                                                              [<<"test">>]},
                                                                             <<74,33,
                                                                               34,175,
                                                                               15,53,
                                                                               141,68,
                                                                               50,225,
                                                                               139,248,
                                                                               181,254,
                                                                               94,133>>,
                                                                             false},
                                                                            1}],
                                                                          []},
                                                                         {lists,
                                                                          foldl,
                                                                          3,
                                                                          [{file,
                                                                            "lists.erl"},
                                                                           {line,
                                                                            1261}]},
                                                                         {rabbit_delayed_message_utils,
                                                                          get_delay,
                                                                          1,
                                                                          []},
                                                                         {rabbit_exchange_type_delayed_message,
                                                                          delay_message,
                                                                          2,
                                                                          []},
                                                                         {rabbit_exchange_type_delayed_message,
                                                                          route,
                                                                          2,
                                                                          []},
                                                                         {rabbit_exchange,
                                                                          route1,
                                                                          3,
                                                                          []},
                                                                         {rabbit_exchange,
                                                                          route,
                                                                          2,
                                                                          []},
                                                                         {rabbit_channel,
                                                                          handle_method,
                                                                          3,
                                                                          []}]}

=ERROR REPORT==== 3-Jun-2015::09:00:39 ===
webmachine error: path="/api/exchanges/%2F$my_app/$my_exchange/publish"
{error,
 {error,function_clause,
  [{rabbit_mgmt_wm_exchange_publish,bad,
    [{function_clause,
      [{rabbit_delayed_message_utils,get_msg,
        [{delivery,true,true,<0.970.0>,
          {basic_message,
           {resource,<<"/$my_app">>,exchange,
            <<"$my_exchange">>},
           [<<>>],
           {content,60,
            {'P_basic',undefined,undefined,[],1,undefined,undefined,
             undefined,undefined,undefined,undefined,undefined,undefined,
             undefined,undefined},
            none,none,
            [<<"test">>]},
           <<74,33,34,175,15,53,141,68,50,225,139,248,181,254,94,133>>,
           false},
          1}],
        []},
       {lists,foldl,3,[{file,"lists.erl"},{line,1261}]},
       {rabbit_delayed_message_utils,get_delay,1,[]},
       {rabbit_exchange_type_delayed_message,delay_message,2,[]},
       {rabbit_exchange_type_delayed_message,route,2,[]},
       {rabbit_exchange,route1,3,[]},
       {rabbit_exchange,route,2,[]},
       {rabbit_channel,handle_method,3,[]}]},
     {wm_reqdata,'POST',http,
      {1,1},
      "192.168.222.1",
      {wm_reqstate,#Port<0.8907>,
       [{'content-encoding',"identity"},
        {'content-type',"text/html"},
        {resource_module,rabbit_mgmt_wm_exchange_publish}],
       undefined,"192.168.222.1",
       {wm_reqdata,'POST',http,
        {1,1},
        "192.168.222.1",undefined,[],"/api/exchanges/%2F$my_app/com.meltw...",
        "/api/exchanges/%2F$my_app/com.melt...",
        [{exchange,"com.meltwater.$my_app...."},{vhost,"%2F$my_app"}],
        [],"../../../../..",500,1073741824,67108864,
        [{"_$my_app_session","BAh7CkkiEF9jc3J..."},{"__profilin","p%3Dt"}],
        [],
        {13,
         {"host",
          {'Host',"192.16..."},
          {"connec...",
           {'Connection',"..."},
           {"...",{'Authorization','...'},{'...'},nil},
           {"...",{'...'},nil,{'...'}}},
          {"user-...",{'User-Agent',"..."},{"...",{'...'},nil,{'...'}},nil}}},
        not_fetched_yet,false,
        {1,{"content-ty...",{"Cont...","tex..."},nil,nil}},
        <<>>,follow_request,[],80,[]},
       undefined,undefined,
       {wm_log_data,undefined,
        {1433,322039,55807},
        'POST',
        {13,
         {"host",
          {'Host',"192.168.222.2:15..."},
          {"connection",
           {'Connection',"keep-aliv..."},
           {"authoriza...",
            {'Authorization',"Ba..."},
            {"ac...",{'Accept',"..."},nil,{"...",{'...'},'...'}},
            nil},
           {"content-...",
            {'Content-Length',"2..."},
            nil,
            {"...",{'Content-Type',"...",'...'},nil,{'...'}}}},
          {"user-agent",
           {'User-Agent',"Mozilla/..."},
           {"origin",
            {"Or...","h..."},
            nil,
            {"...",{'Referer',"...",'...'},nil,nil}},
           nil}}},
        "192.168.222.1","/api/exchanges/%2F$my_app/com.mel...",
        {1,1},
        404,0,undefined,undefined,undefined}},
      [],"/api/exchanges/%2F$my_app/$my_exchange...",
      "/api/exchanges/%2F$my_app/com.meltwater.$my_app.interna...",
      [{exchange,"$my_exchange"},{vhost,"%2F$my_app"}],
      [],"../../../../..",500,1073741824,67108864,
      [{"_$my_app_session","BAh7CkkiEF9jc3JmX3Rva2VuBjoGRUZJIjF..."},
       {"__profilin","p%3Dt"}],
      [],
      {13,
       {"host",
        {'Host',"192.168.222.2:15672"},
        {"connection",
         {'Connection',"keep-alive"},
         {"authorization",
          {'Authorization',"Basic a2VybW..."},
          {"accept",
           {'Accept',"*/*"},
           nil,
           {"acce...",
            {'Accept-Encoding',"..."},
            nil,
            {"...",{'...'},nil,nil,'...'}}},
          nil},
         {"content-length",
          {'Content-Length',"210"},
          nil,
          {"content-ty...",
           {'Content-Type',"tex..."},
           nil,
           {"co...",{'Cookie',"..."},nil,{"...",{'...'},'...'}}}}},
        {"user-agent",
         {'User-Agent',"Mozilla/5.0 (Macin..."},
         {"origin",
          {"Origin","http://192...."},
          nil,
          {"referer",{'Referer',"htt..."},nil,nil}},
         nil}}},
      not_fetched_yet,false,
      {1,{"content-type",{"Content-Type","text/html"},nil,nil}},
      <<>>,follow_request,[],80,[]},
     {context,
      {user,<<"$my_app">>,
       [administrator],
       rabbit_auth_backend_internal,
       {internal_user,<<"$my_app">>,
        <<98,157,153,175,250,176,64,34,246,54,121,205,158,209,254,205,51,33,24,
          177>>,
        [administrator]}},
      <<"$my_app">>}],
    []},
   {rabbit_mgmt_util,with_channel,5,[]},
   {rabbit_mgmt_util,with_decode,5,[]},
   {rabbit_mgmt_wm_exchange_publish,process_post,2,[]},
   {webmachine_resource,resource_call,3,[]},
   {webmachine_resource,do,3,[]},
   {webmachine_decision_core,resource_call,1,[]},
   {webmachine_decision_core,decision,1,[]}]}}

This happens with and without an x-delay message header.

Is the plugin expected to work wit RabbitMQ 3.4?

tobias74 commented 9 years ago

I had similar issues with the version of rabbitmq that is currently the default version within Ubuntu 14.04 LTS

After I upgrade to RabbitMQ 3.5.3 and the most recent Erlang everything worked fine.

michaelklishin commented 9 years ago

I'm afraid we don't have the resources to support 3.4.x. Move to 3.5.3.

RmMsr commented 9 years ago

Ok, I see. Unfortunately we cannot move to RabbitMQ 3.5 because it is not available in stock CentOS. We go back to the old 3 exchange approach. But good that you offer a better way from now on. Thanks.