Invalid request format

[aquiladev]

Issue

I have an issue with the pinning of file/directory. When I use Postman, everything works fine, but with SDK I always receive an error Invalid request format

Code example:

const readableStreamForFile = fs.createReadStream('./data/1.dat');
pinata.pinFileToIPFS(readableStreamForFile)
    .then((result) => {
        //handle results here
        console.log(result);
    }).catch((err) => {
        //handle error here
        console.log(err);
    });

Version

@pinata/sdk@1.1.10

[obo20]

@aquiladev would you be able to upload your data folder that you're working with?

[aquiladev]

no, the code throws the same error

pinata.pinFromFS(source)
    .then((result) => {
        return result.IpfsHash;
    })
    .catch((err) => {
        throw new Error(err);
    });

[obo20]

@aquiladev This is likely a problem with the data you're attempting to upload. Is 1.dat a folder or a file?

[aquiladev]

it is a file, but I also tried folder. Through Postman I upload the file successfully. the file is simple text file

[aquiladev]

I've changed the extension to TXT, but did not help 1.txt

[obo20]

Can I ask what version of npm / node you're running? Also what operating system are you running?

I can't seem to reproduce this on my end.

[aquiladev]

OS: Windows 10 1903 node: v10.17.0 npm: 6.14.4

[obo20]

@aquiladev would you be able to upgrade to node 12.14.1 by any chance?

[aquiladev]

updated to LTS 12.18.1, but it did not help

[obo20]

@aquiladev could you copy and paste your entire file you're running? I'm specifically looking at the imports you're using. I'd also recommend deleting your node_modules and re-installing them with npm install

Unfortunately I just tried this on a windows machine with the same node / npm version and am not able to reproduce. If you're able to provide a .zip folder of your project that would be super helpful to me in attempting to reproduce.

[obo20]

You may also want to make sure that any files you're attempting to upload are readable by your project. I've seen people run into issues before because their projects read / write permissions were very strange on windows machines.

[aquiladev]

I've tried to remove node_modules, but it did not help. The file/folder is part of the project, I don't think there is a problem with permissions.

I'm going to push the project and will share a link

[aquiladev]

https://github.com/aquiladev/ipfs-action/blob/master/uploader/pinata/index.js

The project is a GitHub Action. When I run the Action on a GitHub runner, everything works fine, but it does not work locally.

[obo20]

The only thing I can think of is it sounds like there's something strange going on with your local environment on your machine.

I'm really sorry that I can't be of more help right now.

I'm going to keep this ticket alive for awhile in case anybody else is able to reproduce this issue and can hopefully provide extra details of what's happening.

[ltfschoen]

I've encountered the same error when I run pinata.pinFromFS in my production environment on Heroku, whereas when I was running it locally on my macOS it didn't produce any error. I'm in the process of trying to resolve it.

[ltfschoen]

@aquiladev have you tried doing this?

const path = require('path');
const PATH_FILE = `${path.join(__dirname, '.', 'data')}/1.dat`;
const readableStreamForFile = fs.createReadStream(PATH_FILE);
...

[ltfschoen]

FYI, I resolved my error. It was occurring in my code here https://github.com/ltfschoen/ethquad/blob/master/scripts/pinataUploadIpfs.js#L66, but only when running it on Heroku, and the cause of the error was because I was because I needed to change a line in my package.json to https://github.com/ltfschoen/ethquad/blob/master/package.json#L24 with SKIP_PREFLIGHT_CHECK=true before running the webpack command, otherwise Heroku would whinge as follows (because Heroku also installs a version of Webpack).

remote: One CLI for webpack must be installed. These are recommended choices, delivered as separate packages:
remote:  - webpack-cli (https://github.com/webpack/webpack-cli)
remote:    The original webpack full-featured CLI.
remote: We will use "yarn" to install the CLI via "yarn add -D".
remote: Do you want to install 'webpack-cli' (yes/no):        $ node scripts/pinataUploadIpfs.js
remote:        Successfully authenticated with Pinata
remote:        $ mkdir -p /tmp/build_44ff3cd9/client/build/ipfs
remote:        Generating Pin...
remote: Error: Invalid request format.
remote:     at /tmp/build_44ff3cd9/node_modules/@pinata/sdk/lib/pinata-sdk.js:22336:22
remote:     at processTicksAndRejections (internal/process/task_queues.js:93:5)

And as you can see in these Heroku logs it's prompting me to answer Do you want to install 'webpack-cli' (yes/no):, but as it's production I can't respond, and it just continues with the next script in package.json of running node scripts/pinataUploadIpfs.js (whereas if I ran this locally with yarn dev:ipfs:preview it'd stops when it gets to the prompt).

So because the prompt isn't answered, it doesn't finish running the end of the "build:release:webpack" script in package.json, so it doesn't actually populate my PATH_SOURCE_CODE with the built code https://github.com/ltfschoen/ethquad/blob/master/scripts/pinataUploadIpfs.js#L66, hence the error Error: Invalid request format.

[isaadabbasi]

I am running in same issue

System Specs: MacOS Catalina 10.15.5 Nodejs: 14.4 Npm: 16.14.4

Scenario: I have a graphQL server which also does the file upload. When i do pinFromFS and pass a filePath to it, it works like a charm. But when i do pinFileFromIPFS and pass a steram = readableStream() to it (Which is instanceOf stream.Readable native module) it throws Invalid Request Format Further debugging i added a console.log in @pinata/sdk's pinFileFromIPFS function and i can see the headers have pinata_api_key and pinata_secret_key (SO IT CANNOT BE ENV ISSUE)

[eightyfive]

I'm facing the same problem.

  async pinImage(dataURL) {
    const data = dataURL.replace(/^data:image\/png;base64,/, "");
    const buff = Buffer.from(data, "base64");
    const stream = Readable.from(buff);

    const res = await pinata.pinFileToIPFS(stream);

    return res.IpfsHash;
  },

No problem when reading a local file (filesystem).

[eightyfive]

Temporary solution (but far from ideal):

  async pinImage(dataURL) {
    const data = dataURL.replace(/^data:image\/png;base64,/, "");
    const buff = Buffer.from(data, "base64");
    const tmp = path.resolve(os.tmpdir(), `${new Date().getTime()}.png`);

    await fs.writeFile(tmp, buff);

    const stream = fs.createReadStream(tmp);

    const res = await pinata.pinFileToIPFS(stream);

    return res.IpfsHash;
  },

Note: Using fs-extra.

[eightyfive]

Found it !

  async pinImage(dataURL) {
    const data = dataURL.replace(/^data:image\/png;base64,/, "");
    const buff = Buffer.from(data, "base64");
    const stream = Readable.from(buff);

    // ¡¡ THE HACK !!
    stream.path = "some_filename.png";

    const res = await pinata.pinFileToIPFS(stream);

    return res.IpfsHash;
  },

Basically the Pinata API complains if you don't provide a filename for the file you want to pin. The hint was to look at the "Pin Manager" where file names are displayed for every upload.

@obo20 I believe the documentation / recipe book should point out this edge case or the API should return a better error message.

[austinkline]

Confirming that the stream.path workaround works for me as well:

    const readable = Readable.from("your string here");
    readable.path = "some_name.png";

    const options = { ... };

    try {
        const uploadRes = await pinata.pinFileToIPFS(readable, options);
        console.log(uploadRes); <-- this is hit if you set readable.path
    } catch (err) {
        console.log(err); <-- this is hit if you do not.
    }

[tomonari-t]

@austinkline 's post solved my issue.

[isaadabbasi]

@eightyfive Man, I can't thank you enough <3 I approve his solutions works the best

[Ian-Bright]

@eightyfive Was your solution reliably pinning images? I am using very similar code but when the images get pinned and I view them in the pin manager I am just seeing a buffer and not the actual image. Thanks

[eightyfive]

Hey @Ian-Bright no problem pinning images in my case

[Ian-Bright]

Hmmm ok interesting. I am trying to do this with a lambda function and for some reason the buffer is not being decoded correctly. I may switch to a regular node server

On Mon, Jul 12, 2021, 4:45 PM 85 @.***> wrote:

Hey @Ian-Bright https://github.com/Ian-Bright no problem pinning images in my case

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/PinataCloud/Pinata-SDK/issues/28#issuecomment-878583193, or unsubscribe https://github.com/notifications/unsubscribe-auth/APYUAHGVMMR65EVL6WDJPX3TXNH77ANCNFSM4OCBKHEQ .

[talentlessguy]

IMO would be nice if by default it sets a UUID to a file if path is not

x.path = '...' hack still works, even though ReadStream or Readable don't have "path" property

[rusticphilosopher]

.path hack worked for me also - examples and documentation needs to be updated to represent this.

[ahujasid]

Readable throws the error for me "Readable.from is not available in the browser". Is this common for everyone??

[blastshielddown]

@austinkline 's post solved for me as well. Thank you

[kingjerod]

The SDK should really add a function for uploading Buffers. Ran into this same problem.

[silasdavis]

Yeah this is a really obvious thing to want to do

[Willyham]

Had the same issue. We're generating image data from an API not from files. Buffer upload or at least a mention in the docs would be nice.

[cbrpnk]

That hack solved the issue for me, thanks!

[HappySean2845]

Nice Hack, It did helped me from the Invalid request!

[brucexu-eth]

dude, you just saved my day, appreciate!! @eightyfive

[hillmark]

Hack worked perfectly, thanks!

[niraj-khatiwada]

@eightyfive hack worked for me as well. But I would really really prefer uploading Buffer directly.

[xpluscal]

@eightyfive any idea how your hack would apply for uploading a folder as outlined here: https://docs.pinata.cloud/pinata-api/pinning/pin-file-or-directory#uploading-and-pinning-a-directory

[eightyfive]

Hey unfortunately I don't know.

I use NFT Storage now.

[molejnik-mergebit]

Found it !

  async pinImage(dataURL) {
    const data = dataURL.replace(/^data:image\/png;base64,/, "");
    const buff = Buffer.from(data, "base64");
    const stream = Readable.from(buff);

    // ¡¡ THE HACK !!
    stream.path = "some_filename.png";

    const res = await pinata.pinFileToIPFS(stream);

    return res.IpfsHash;
  },

Basically the Pinata API complains if you don't provide a filename for the file you want to pin. The hint was to look at the "Pin Manager" where file names are displayed for every upload.

@obo20 I believe the documentation / recipe book should point out this edge case or the API should return a better error message.

Same issue and same solution here, thank you a million! It's a shame it's nowhere documented...

[rommel-pinata]

Yeah it seems most of the issue faced in this thread as related to the name of the file, this is now mandatory in the new release.

[SimRunBot]

should also update the API doc here https://docs.pinata.cloud/pinata-api/pinning/pin-file-or-directory with the .path solution

[SimRunBot]

Was your solution reliably pinning images? I am using very similar code but when the images get pinned and I view them in the pin manager I am just seeing a buffer and not the actual image. Thanks

For anybody having this particular issue: make sure the file you are pinning is of the correct iterable image type ( likely uint8) instead of string, which happened to me and is hard to notice since it looks like an array but actually is one long string.

[ruuuruiya]

Found it !

  async pinImage(dataURL) {
    const data = dataURL.replace(/^data:image\/png;base64,/, "");
    const buff = Buffer.from(data, "base64");
    const stream = Readable.from(buff);

    // ¡¡ THE HACK !!
    stream.path = "some_filename.png";

    const res = await pinata.pinFileToIPFS(stream);

    return res.IpfsHash;
  },

Basically the Pinata API complains if you don't provide a filename for the file you want to pin. The hint was to look at the "Pin Manager" where file names are displayed for every upload.

@obo20 I believe the documentation / recipe book should point out this edge case or the API should return a better error message.

2024 UPDATE! For those who are new to this, now you dont need to specify the stream.path = "filename.png" you have to provide the name inside options like this:

    const res = await pinata.pinFileToIPFS(stream, {
        pinataMetadata: {
            name: "tes.png"
        }
    })

even tho pinata docs said its optional: pinata.pinFileToIPFS(readableStream, options) Params

• readableStream - A readableStream of the file to be added
• options (optional): A JSON object that can contain the following keyvalues:
  1. pinataMetadata (optional): A JSON object with metadata for the file being pinned
  2. pinataOptions (optional): A JSON object with additional options for the file being pinned

but the error will occur like this: ⨯ Error: filename was not provide, make sure to provide options.pinataMetadata.name

So make sure to provide the pinataMetadata.name

+================+ but the thing is that I dont know how to make it happen using the API /pinFileToIPFS I have tried stream.path = "filename.png" and formData.append("pinataMetadata", JSON.stringify({ name: "filename.png" })); (docs said default to filename) but still got { error: 'Invalid request format.' }

[stevedylandev]

Hey all! Thanks for your comments on this one as it has been certainly confusing to say the least.

In summation, the Pinata API accepts blobs as long as they are appended to form data like so:

const JWT = 'YOUR_PINATA_JWT';

async function pinFileToIPFS() {
  try {
    const text = "Hello World!";
    const blob = new Blob([text], { type: "text/plain" });
    const data = new FormData();
    data.append("file", blob);

    const res = await fetch("https://api.pinata.cloud/pinning/pinFileToIPFS",
      method: "POST",
      headers: {
        Authorization: `Bearer ${JWT}`,
      },
      body: data,
    );
    const resData = await res.json();
    console.log(resData);
  } catch (error) {
    console.log(error);
  }
};

Link to docs

We are going to be rewriting the SDK to make this easier to adapt and use in scenarios like this where you are not using a file source from node.