Apollo gateway - file upload "missing operations" error on micro-service !!

[tkssharma]

always please provide examples so that devs don't have to struggle when they use some external 3rd party library

First of all thanks for providing this solution I am a bit stuck with this solution

I saw this Blog https://medium.com/profusion-engineering/file-uploads-graphql-and-apollo-federation-c5a878707f4c the tried this solution but it does not work

I have nest js apollo federation gateway and microservices to support file upload, my use case is the same as mentioned in this blog buildService: ({ url }) => new FileUploadDataSource({ url, useChunkedTransfer: true }), My gateway

@Module({
  imports: [
    GraphQLModule.forRoot<ApolloGatewayDriverConfig>({
      server: {
        context: handleAuth,
      },
      driver: ApolloGatewayDriver,
      gateway: {
        buildService: ({ url }) => new FileUploadDataSource({ url, useChunkedTransfer: true }),
        /* buildService: ({ name, url }) => {
          return new RemoteGraphQLDataSource({
            url,
            willSendRequest({ request, context }: any) {
              request.http.headers.set('userId', context.userId);
              // for now pass authorization also
              request.http.headers.set('authorization', context.authorization);
              request.http.headers.set('permissions', context.permissions);
            },
          });
        }, */
        supergraphSdl: new IntrospectAndCompose({
          subgraphs: [
            { name: 'User', url: 'http://localhost:5006/graphql' },
            { name: 'Home', url: 'http://localhost:5003/graphql' },
            { name: 'Booking', url: 'http://localhost:5004/graphql' },
          ],
        }),
      },
    }),
  ],
})

earlier I was trying this based on some existing issues on this repo

 gateway: {
        buildService: ({ url }) =>
          new FileUploadDataSource({
            url,
            useChunkedTransfer: true,
            willSendRequest({ request, context }: any) {
              if (context.req) {
                const { cookie, authorization, referer, as, userId, permissions } = context.req.headers;

                request.http.headers.set('userId', userId);
                // for now pass authorization also
                request.http.headers.set('authorization', authorization);
                request.http.headers.set('permissions', permissions);
              }
            },
          }),
        supergraphSdl: new IntrospectAndCompose({
          subgraphs: [
            { name: 'User', url: 'http://localhost:5006/graphql' },
            { name: 'Home', url: 'http://localhost:5003/graphql' },
            { name: 'Booking', url: 'http://localhost:5004/graphql' },
          ],
        }),
      },

This is my curl request, i have created this curl request from the example app you guys have shared

curl 'http://localhost:5002/graphql' \
  -H 'Accept-Language: en-GB,en-US;q=0.9,en;q=0.8' \
  -H 'Connection: keep-alive' \
  -H 'Content-Type: multipart/form-data; boundary=----WebKitFormBoundarydTXvKTxEAVztL3Cl' \
  -H 'Origin: http://localhost:3008' \
  -H 'Referer: http://localhost:3008/' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Site: same-site' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36' \
  -H 'accept: */*' \
  -H 'sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="101", "Google Chrome";v="101"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "macOS"' \
  --data-raw $'------WebKitFormBoundarydTXvKTxEAVztL3Cl\r\nContent-Disposition: form-data; name="operations"\r\n\r\n{"operationName":"uploadHomePhoto","variables":{"file":null},"query":"mutation uploadHomePhoto($file: Upload\u0021) {\\n  uploadHomePhoto(file: $file) {\\n    id\\n    __typename\\n  }\\n}"}\r\n------WebKitFormBoundarydTXvKTxEAVztL3Cl\r\nContent-Disposition: form-data; name="map"\r\n\r\n{"1":["variables.file"]}\r\n------WebKitFormBoundarydTXvKTxEAVztL3Cl\r\nContent-Disposition: form-data; name="1"; filename="qwdeqd.txt"\r\nContent-Type: text/plain\r\n\r\nqwedqdeq\r\n------WebKitFormBoundarydTXvKTxEAVztL3Cl--\r\n' \
  --compressed

Github Example Client-side https://github.com/jaydenseric/apollo-upload-examples

at the microservices end I am getting the same always

{"correlationId":"e4464af9-0af4-472d-882f-82de9034d7aa","level":"error","message":"[Fri May 27 15:32:43 2022] [error] Missing multipart field ‘operations’ (https://github.com/jaydenseric/graphql-multipart-request-spec)."}
{"correlationId":"e4464af9-0af4-472d-882f-82de9034d7aa","level":"error","message":"[Fri May 27 15:32:43 2022] [error] Missing multipart field ‘operations’ (https://github.com/jaydenseric/graphql-multipart-request-spec)."}
{"correlationId":"e4464af9-0af4-472d-882f-82de9034d7aa","level":"error","message":"[Fri May 27 15:32:44 2022] [error] Missing multipart field ‘operations’ (https://github.com/jaydenseric/graphql-multipart-request-spec)."}
{"correlationId":"e4464af9-0af4-472d-882f-82de9034d7aa","level":"error","message":"[Fri May 27 15:32:44 2022] [error] Missing multipart field ‘operations’ (https://github.com/jaydenseric/graphql-multipart-request-spec)."}

Please let me know your inputs on this.

Thx,

[barbieri]

hi @tkssharma, we'll take a look and come back asap, but it may happen only during next week, ok?

[tkssharma]

i am good,

Thanks for the support.

[DDDKnightmare]

{"correlationId":"e4464af9-0af4-472d-882f-82de9034d7aa","level":"error","message":"[Fri May 27 15:32:43 2022] [error] Missing multipart field ‘operations’ (https://github.com/jaydenseric/graphql-multipart-request-spec)."} {"correlationId":"e4464af9-0af4-472d-882f-82de9034d7aa","level":"error","message":"[Fri May 27 15:32:43 2022] [error] Missing multipart field ‘operations’ (https://github.com/jaydenseric/graphql-multipart-request-spec)."} {"correlationId":"e4464af9-0af4-472d-882f-82de9034d7aa","level":"error","message":"[Fri May 27 15:32:44 2022] [error] Missing multipart field ‘operations’ (https://github.com/jaydenseric/graphql-multipart-request-spec)."} {"correlationId":"e4464af9-0af4-472d-882f-82de9034d7aa","level":"error","message":"[Fri May 27 15:32:44 2022] [error] Missing multipart field ‘operations’ (https://github.com/jaydenseric/graphql-multipart-request-spec)."}

I've seen similar errors when the services weren't parsing the received request body.Could you check if your microservices are parsing the request body? Since you've got errors on them, it means they are receiving requests. Another point: Could you add -X POST curl to the request ? curl defaults the method to GET, and most servers ignore the body on GET requests.

[tkssharma]

This is nestjs app, it manages body parsing itself

added -X POST it didn't change anything I am checking this, may be I will give you the whole example with the federation to try this

[tkssharma]

Just for example here is my microservice code 👍
https://github.com/mrsauravsahu/blog-graphql-nestjs-fileupload-apollo-federation

I am able to upload files using

curl --location --request POST 'http://localhost:8080/graphql' \
--form 'operations="{\"query\": \"mutation updateProfilePhoto($file: Upload!) {  coverPhoto(file: $file)} \", \"variables\": {\"file\": null}}"' \
--form 'map="{\"0\": [\"variables.file\"]}"' \
--form '0=@"./assets/grand-palais-mrsauravsahu.jpg"'

and this is how my gateway looks like Now when I plug the gateway to this service I am not able to get the upload working TypeError: Cannot destructure property 'createReadStream' of 'file' as it is undefined.

I hope this example is enough to re-produce this issue

 import { RemoteGraphQLDataSource } from '@apollo/gateway';
import {
  Module,
  BadRequestException,
  HttpStatus,
  HttpException,
  UnauthorizedException,
  MiddlewareConsumer,
} from '@nestjs/common';
import { IntrospectAndCompose } from '@apollo/gateway';
import { ApolloGatewayDriver, ApolloGatewayDriverConfig } from '@nestjs/apollo';
import { GraphQLModule } from '@nestjs/graphql';
import { verify, decode } from 'jsonwebtoken';
import { INVALID_AUTH_TOKEN, INVALID_BEARER_TOKEN } from './app.constants';
import { graphqlUploadExpress } from 'graphql-upload';
import FileUploadDataSource from '@profusion/apollo-federation-upload';

const getToken = (authToken: string): string => {
  const match = authToken.match(/^Bearer (.*)$/);
  if (!match || match.length < 2) {
    throw new HttpException(
      { message: INVALID_BEARER_TOKEN },
      HttpStatus.UNAUTHORIZED,
    );
  }
  return match[1];
};

const decodeToken = (tokenString: string) => {
  const decoded = verify(tokenString, process.env.SECRET_KEY);
  if (!decoded) {
    throw new HttpException(
      { message: INVALID_AUTH_TOKEN },
      HttpStatus.UNAUTHORIZED,
    );
  }
  return decoded;
};
const handleAuth = ({ req }) => {
  try {
    if (req.headers.authorization) {
      const token = getToken(req.headers.authorization);
      const decoded: any = decodeToken(token);
      console.log(
        `userId: ${decoded.userId} permissions: ${decoded.permissions}`,
      );
      return {
        userId: decoded.userId,
        permissions: decoded.permissions,
        authorization: `${req.headers.authorization}`,
      };
    }
  } catch (err) {
    throw new UnauthorizedException(
      'User unauthorized with invalid authorization Headers',
    );
  }
};
@Module({
  imports: [
    GraphQLModule.forRoot<ApolloGatewayDriverConfig>({
      server: {
        context: handleAuth,
      },
      driver: ApolloGatewayDriver,
      gateway: {
        buildService: ({ url }) =>
          new FileUploadDataSource({
            url,

            useChunkedTransfer: true,
            willSendRequest({ request, context }: any) {
              request.http.headers.set('userId', context.userId);
              // for now pass authorization also
              request.http.headers.set('authorization', context.authorization);
              request.http.headers.set('permissions', context.permissions);
            },
          }),

        supergraphSdl: new IntrospectAndCompose({
          subgraphs: [
            { name: 'User', url: process.env.AUTH_API },
            { name: 'Home', url: process.env.HOME_MANAGER_API },
            { name: 'Booking', url: process.env.BOOKING_MANAGER_API },
            { name: 'File', url: process.env.FILE_MANAGER_API },
          ],
        }),
      },
    }),
  ],
})
export class AppModule {
  configure(consumer: MiddlewareConsumer) {
    //consumer.apply(graphqlUploadExpress()).forRoutes('graphql');
  }
}

require('dotenv').config();
import { NestFactory } from '@nestjs/core';
import { graphqlUploadExpress } from 'graphql-upload';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  app.use(graphqlUploadExpress({ maxFileSize: 2 * 1000 * 1000 }));
  await app.listen(process.env.PORT || 4000);
}
bootstrap();

[DDDKnightmare]

@tkssharma , using the example application you provided, it worked without using chunked transfer. Maybe some additional configuration is needed to use chunked transfer on Nest ?

[cabelitos]

I saw these kind of errors happening if you forget to add the Upload resolver your project. Did you add it?

[tkssharma]

Yes, My microservice works fine and i am able to upload files it just while I send same request through a gateway

[oliveirarleo]

@tkssharma Have you fixed this problem? we have a similar titled issue that it looks like it was solved by correctly adding the headers. Please let me know or else we can close this issue.

[tkssharma]

i am not able to fix it

[oliveirarleo]

Okay, then we'll put someone on to help you here.

[gabriel4k2]

Hi @tkssharma I'll take a look on what is going on.

[frederic11]

I have the same problem as @tkssharma. I wasn't able to pass the file to the microservice through the Gateway. The Microservice works perfectly by itself. Any luck in finding what's wrong?

[tkssharma]

Thanks, finally someone is able to re-produce, This package provides this basic feature but if it doesn't work then ...

[frederic11]

Replying to myself here more than anyone; Explicitly setting the useChunkedTransfer: false works fine for me. @tkssharma can you try that on your end? @DDDKnightmare already mentioned that it does work without Chunked Transfer.

@Module({
  imports: [
    AppConfigModule.forRoot(),
    GraphQLModule.forRootAsync<ApolloGatewayDriverConfig>({
      inject: [ConfigService],
      driver: ApolloGatewayDriver,
      useFactory: async (config: ConfigService) => ({
        server: {
          context: async ({ req }) => {
            const auth = new Auth(config)
            await auth.handle(req)
          },
          debug: true,
          playground: true,
          sortSchema: true,
          introspection: true,
          cors: ['*'],
          path: '/api/graphql',
        },
        gateway: {
          buildService: ({ url }) => new FileUploadDataSource({ url, useChunkedTransfer: false }),
          supergraphSdl: readFileSync(process.env.MLP_GATEWAY_GRAPH || './supergraph.graphql') //TO Validate
            .toString()
            .replace(/HOST/g, process.env.MLP_HOST),
        },
      }),
    }),
  ],
  controllers: [],
  providers: [],
})
export class AppModule {}

[brunorosano]

@tkssharma, I've run the example you sent and the file upload worked correctly with and without chunked transfer. Maybe something changed in the most recent versions of nest that fixed this issue. These are the package versions I used during the tests:

  "dependencies": {
    "@apollo/federation": "^0.38.1",
    "@apollo/gateway": "^2.0.0",
    "@apollo/subgraph": "^2.3.2",
    "@nestjs/apollo": "^10.2.0",
    "@nestjs/common": "^8.0.0",
    "@nestjs/core": "^8.0.0",
    "@nestjs/graphql": "^10.2.0",
    "@nestjs/platform-express": "^8.0.0",
    "@profusion/apollo-federation-upload": "^4.0.0",
    "apollo-server-core": "^3.11.1",
    "apollo-server-express": "^3.11.1",
    "graphql-upload": "^11.0.0",
    "install": "^0.13.0",
    "jsonwebtoken": "^9.0.0",
    "nestjs": "^0.0.1",
    "npm": "^9.5.1",
    "typescript": "^4.6.3"
  },
  "devDependencies": {
    "@types/graphql-upload": "^8.0.7",
    "@types/jsonwebtoken": "^9.0.1"
  }

Which package versions have you used? I can also do some tests with them to check if this problem is caused by any of these packages