search

nestjs / nest

A progressive Node.js framework for building efficient, scalable, and enterprise-grade server-side applications with TypeScript/JavaScript 🚀
https://nestjs.com
MIT License
66.89k stars 7.56k forks source link

(microservice)There is a concurrent bug in microservice. #505

Closed whtiehack closed 6 years ago

whtiehack commented 6 years ago

I'm submitting a...


[ ] Regression 
[X] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

Current behavior

[Nest] 17131   - 2018-3-13 18:56:06   [NestFactory] Starting Nest application...
[Nest] 17131   - 2018-3-13 18:56:06   [InstanceLoader] ApplicationModule dependencies initialized +6ms
[Nest] 17131   - 2018-3-13 18:56:06   [InstanceLoader] CatsModule dependencies initialized +2ms
[Nest] 17131   - 2018-3-13 18:56:06   [InstanceLoader] MicroClientModule dependencies initialized +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [RoutesResolver] CatsController {/cats}: +27ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/, POST} route +1ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/, GET} route +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/micro, GET} route +1ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/micro2, GET} route +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/microBuffer, GET} route +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/:id, GET} route +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [NestApplication] Nest application successfully started +1ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:196,102,138,162,96,result:531 +28628ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:149,191,43,199,184,result:531 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:172,173,129,100,34,result:531 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:78,73,184,99,46,result:531 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [ClientTCP] Error: write after end
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:139,101,64,195,156,result:632 +9ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:153,109,96,9,105,result:632 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] count:50 +1ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:31,156,77,50,185,result:632 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:128,22,188,27,2,result:632 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:152,69,56,3,181,result:448 +11ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:73,197,170,75,105,result:448 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:38,4,169,178,1,result:656 +6ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:137,174,56,115,25,result:401 +9ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:12,99,37,139,113,result:401 +1ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:191,41,152,122,132,result:401 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:186,106,14,27,186,result:401 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:142,60,16,127,165,result:472 +11ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:136,179,67,184,79,result:472 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:117,89,133,67,95,result:472 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:165,179,192,9,85,result:472 +1ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:132,191,146,85,128,result:567 +11ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:91,34,66,18,29,result:567 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:82,67,30,52,161,result:567 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:189,94,104,137,82,result:567 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:89,143,95,74,126,result:236 +6ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:150,136,38,131,87,result:437 +4ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:77,34,156,55,97,result:437 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:95,197,162,80,178,result:369 +5ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:11,194,108,198,106,result:369 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [ClientTCP] Error: write after end
[Nest] 17131   - 2018-3-13 18:56:34   [ClientTCP] Error: write after end
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:190,94,154,195,148,result:617 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:196,41,193,170,151,result:617 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [ClientTCP] Error: write after end
[Nest] 17131   - 2018-3-13 18:56:34   [ClientTCP] Error: write after end
[Nest] 17131   - 2018-3-13 18:56:34   [ClientTCP] Error: write after end
[Nest] 17131   - 2018-3-13 18:56:34   [ClientTCP] Error: write after end
[Nest] 17131   - 2018-3-13 18:56:34   [ClientTCP] Error: write after end

Expected behavior

[Nest] 17131   - 2018-3-13 18:56:06   [NestFactory] Starting Nest application...
[Nest] 17131   - 2018-3-13 18:56:06   [InstanceLoader] ApplicationModule dependencies initialized +6ms
[Nest] 17131   - 2018-3-13 18:56:06   [InstanceLoader] CatsModule dependencies initialized +2ms
[Nest] 17131   - 2018-3-13 18:56:06   [InstanceLoader] MicroClientModule dependencies initialized +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [RoutesResolver] CatsController {/cats}: +27ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/, POST} route +1ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/, GET} route +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/micro, GET} route +1ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/micro2, GET} route +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/microBuffer, GET} route +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/:id, GET} route +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [NestApplication] Nest application successfully started +1ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] count:50 +1ms

Minimal reproduction of the problem with instructions

Client

@Component()
export class MicroClientService {
    @Client({ transport: Transport.TCP, port: 3006 })
    public testMicroClient: ClientProxy

}

@Controller('cats')
export class CatsController {
    logger:Logger;
    count:number;
    constructor(private readonly catsService: CatsService, private readonly microClientService: MicroClientService) {
        this.logger = new Logger("CatsController",true);
        this.count = 0;
    }

    static getRandom():number{
        return Math.floor(Math.random()*200);
    }
    @Get('/micro')
    async testMicroService() {
        const data = [CatsController.getRandom(),CatsController.getRandom(),CatsController.getRandom(),CatsController.getRandom(),CatsController.getRandom()];
        const result = await this.microClientService.testMicroClient.send<number>({cmd: 'sum'}, data).toPromise();
        if(result!=data.reduce((a,b)=>a+b)){
            this.logger.error(`data:${data.toString()},result:${result}`);
        }
        this.count++;
        if(this.count%50==0){
            this.logger.log(`count:${this.count}`);
        }
        return result;
    }
}

Server

  const app = await NestFactory.createMicroservice(ApplicationModule,{
    transport: Transport.TCP,
      port:3006
  });
@Controller()
export class MathController {
  @MessagePattern({ cmd: 'sum' })
  sum(data: number[]): number {
  //  throw new RpcException('~~server error~');
    return (data || []).reduce((a, b) => a + b);
  }
}

test code

function fun(){
    $.ajax({
    url:'http://127.0.0.1:3000/cats/micro',
    method:'GET',
    success: function(res){

    },
    error: function(err){
        console.error('err',err);
  }});
}

// !!!!!!
for(var i=0;i<60;i++){fun()}

What is the motivation / use case for changing the behavior?

i think the problem is .. https://github.com/nestjs/nest/blob/7bb35c0b387b46a49ce6d5cdaf039bbfb2da7762/src/microservices/server/server.ts#L25-L36 https://github.com/nestjs/nest/blob/7bb35c0b387b46a49ce6d5cdaf039bbfb2da7762/src/microservices/server/server.ts#L34

client: https://github.com/nestjs/nest/blob/7bb35c0b387b46a49ce6d5cdaf039bbfb2da7762/src/microservices/client/client-tcp.ts#L40-L63

https://github.com/nestjs/nest/blob/7bb35c0b387b46a49ce6d5cdaf039bbfb2da7762/src/microservices/client/client-tcp.ts#L58-L59

If i want to re-implement ClientProxy and CustomTransportStrategy will modify server.ts and client-proxy.ts.

i want use my own implement , how can i do? https://github.com/whtiehack/pinus/blob/master/packages/pinus-rpc/lib/rpc-server/acceptors/tcp-acceptor.ts

https://github.com/whtiehack/pinus/blob/master/packages/pinus-rpc/lib/rpc-client/mailboxes/tcp-mailbox.ts

https://github.com/whtiehack/pinus/blob/master/packages/pinus-rpc/lib/util/composer.ts

Environment


Nest version:4.6.6


For Tooling issues:
- Node version: 8.9.4 
- Platform:  Mac 

Others:
whtiehack commented 6 years ago

We are looking for a new framework to start a new project. I would like to know if this framework has been stress tested. Because of the large amount of traffic in our project, it is very cautious to choose what framework to use. Is there any data to look at? Our focus is on gateway(websocket) and microservices(tcp).😭

kamilmysliwiec commented 6 years ago

Thanks for reporting, I'm on it.

However, Nest microservices strategies are good for prototyping. Still, I'd recommend using a mature solution like, for example, NATS and plug it into Nest as a Custom Transport Strategy (which should be very easy). It will change in the incoming release (v5.0.0) where microservices will offer much more than before.

whtiehack commented 6 years ago

@kamilmysliwiec Thanks for the answer, I thought of a solution.Thanks.

kamilmysliwiec commented 6 years ago

The microservices package has been completely rewritten under the hood. With v5.0.0 this bug is fixed (v5 is available already as 5.0.0-beta.1 in NPM, docs are here https://docs.nestjs.com/v5/)

zuohuadong commented 6 years ago

image like it: https://github.com/Netflix/Hystrix

When the load is too high, the VIP access is guaranteed. What am I supposed to do?

@kamilmysliwiec

lock[bot] commented 4 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.