Touffy / client-zip

A client-side streaming ZIP generator
MIT License
349 stars 22 forks source link

TypeError: Failed to fetch // Question #80

Open hwyatt opened 10 months ago

hwyatt commented 10 months ago

Hello! I am using client-zip to package up audio tracks (probably ~75-100 at a time) in my NextJS app. The first few times I ran this, it was excitingly performant considering my input. However, after those initial test attempts I am now getting TypeError: Failed to fetch. Is there a better way to handle this based on my use case? I just copy pasta'd the example usage code so I very well could be missing something that would help me out here.

Thank you!

const createZip = async (tracks) => {
  console.log("started");
  let files: any = [];
  tracks.map((track) => {
    track.clips.forEach((clip) => {
      files.push({ name: `${track.name}/${clip.name}`, input: clip.content });
    });
  });
  try {
    // get the ZIP stream in a Blob
    const blob = await downloadZip(files).blob();

    // make and click a temporary link to download the Blob
    const link = document.createElement("a");
    link.href = URL.createObjectURL(blob);
    link.download = "MultiTracks.zip";
    link.click();
    link.remove();
    console.log("finished");
  } catch (err) {
    console.log(err);
  }
};
hwyatt commented 10 months ago

Also, occasionally getting results like this even when the zipping seems successful

Screenshot 2024-01-06 at 6 58 22 AM

Touffy commented 10 months ago

TypeError: Failed to fetch cannot be thrown by client-zip. Can't guess where it comes from without seeing more code.

Don't forget to call URL.revokeObjectURL(link.href) if you're sure the same download won't be needed after the click. If you do not, the Blob will never be garbage-collected while you remain on the page.

The error shown in your Archive Utility dialog might be caused by your filename logic. client-zip does not sanitize filenames beyond removing or adding a trailing slash (depending on whether the entry is a folder with no content). If you tell client-zip to store a file with an impossible filename — such as an empty string, or just a slash —, it will do that, but it's not terribly surprising that macOS later refuses to extract to that filename.

Touffy commented 10 months ago

Correction : TypeError: Failed to fetch cannot be thrown by the current version of client-zip. Until 1.6.0 or 2.4.0, there was a fetch to get the WebAssembly module for CRC32, and it may very well fail, for a number of reasons. Make sure you are using the latest version.

hwyatt commented 10 months ago

Thanks for the quick response!

The catch (err) was logging the TypeError: Failed to fetch, so I just assumed that was the fault of await downloadZip(files).blob(); or my input. I should be on the latest version, but I will double check.

The file naming logic I am using is working at times. It may just be the machine I am using at the moment. I'm gonna do some testing to ensure it's not happening across different machines.

freedomsky11 commented 10 months ago

I have met the same problem recently, and I'm use in nextjs too.

Touffy commented 10 months ago

Wait… do you get the error on the server-side of nextJS ? Did you bundle client-zip with the nextJS toolchain ?

hwyatt commented 10 months ago

I am not using client-zip in a NextJS API route, but rather in a component. It seemed that after explicitly putting "use client" at the top of my component that the error had ceased, but I'm still getting it sadly. I am on ^2.4.4

Touffy commented 10 months ago

Could still be the bundling ; there was a bug report a while ago, where nextJS was trying to transpile client-zip to es2016 or something… not ≥ es2020 anyway. But the symptoms were quite different.

I can't help much without access to a page where you have included client-zip with nextJS. Like I said, that error message doesn't exist in client-zip's code — and it has no dependencies that might throw the error — nor any call to fetch. So it has to come from something added by nextJS, or maybe your own code when you get the input Responses.

hwyatt commented 10 months ago

@Touffy I can drop you the Netlify or the repo, if either of those would be helpful. I cannot see currently where Next is attempting to transpile client-zip, but maybe I am missing something obvious.

hwyatt commented 9 months ago

I also tried adding client-zip in my next.config.js like so:

const nextConfig = {
  transpilePackages: ["client-zip"],
};

module.exports = nextConfig;

This also made no difference

davidjunger-oa commented 9 months ago

The netlify app would help. If I have time to look at it closely enough, anyway.

Touffy commented 9 months ago

Sorry, forgot to switch back to my main account. The previous message was also me.

hwyatt commented 9 months ago

https://sunday-set-builder.netlify.app/

Let me know if I need to setup a way for you to manually trigger the downloadZip().

Touffy commented 9 months ago

I haven't triggered a download (a set of valid files would help, I guess) but I can already tell you that client-zip is indeed bundled (into _next/static/chunks/208-0156ce2acf62063b.js) and that it's not transpiled (which is good).

hwyatt commented 9 months ago

I was able to reproduce the error by dropping this Test Song 1 into the drop area 4/5 times, then hitting Build Ableton Set. I am logging the error to the console as well.

Link to Zip (just unzip and drop it in the droparea 5 or so times) https://drive.google.com/file/d/1nSNB5Ca-NWjG-luhltSk4McKWXw4ZpBH/view?usp=drive_link

Angel-Ponce commented 4 months ago

Same problem, I generate a simple .zip file to wrap various images into only file When I work on my local environments everythings runs ok, but when I deploy my application to development/production environments the final .zip result generates the following output:

image

There is a process executed using the terminal:

image

Touffy commented 3 months ago

The lack of the End of Central Directory suggests that the download was interrupted. There is no condition in client-zip that would allow it to skip that part otherwise.

Also the fact that it works consistently on localhost… how long does your download take locally and in your dev/prod environments ? Are you using Firefox, perhaps ? (there is a known issue where Firefox can kill a Service Worker even while it's downloading from it)

mavogler commented 3 months ago

I have the same problem. After some research I found out, that it must have something to do with rollupjs and the treeshaking it does. Specifically, the normalizeName-function in metadata.ts is affected. If I disable treeshaking in the rollup-config, everything works fine. If I enable it, the fixFilename-function throws an error "The file must have a name." because encodedName is undefined. This also causes the other error mentioned above.

This is the normalizeMetadata-function with treeshaking enabled.

function s(e2, i2, r2) {
  let [f2, o2] = /* @__PURE__ */ function(e3) {
    return [void 0, 0];
  }();
  if (e2 instanceof File) return { o: d(f2 || t(e2.name)), u: BigInt(e2.size), l: o2 };
  if (e2 instanceof Response) {
    const n2 = e2.headers.get("content-disposition"), i3 = n2 && n2.match(/;\s*filename\*?=["']?(.*?)["']?$/i), a2 = i3 && i3[1] || e2.url && new URL(e2.url).pathname.split("/").findLast(Boolean), s2 = a2 && decodeURIComponent(a2), u2 = +e2.headers.get("content-length");
    return { o: d(f2 || t(s2)), u: BigInt(u2), l: o2 };
  }
  return f2 = d(f2, void 0 !== e2 || void 0 !== r2), "string" == typeof e2 ? { o: f2, u: BigInt(t(e2).length), l: o2 } : e2 instanceof Blob ? { o: f2, u: BigInt(e2.size), l: o2 } : e2 instanceof ArrayBuffer || ArrayBuffer.isView(e2) ? { o: f2, u: BigInt(e2.byteLength), l: o2 } : { o: f2, u: u(e2, r2), l: o2 };
}

This is the normalizeMetadata-function with treeshaking disabled.

function s(e2, i2, r2) {
  let [f2, o2] = function(e3) {
    return e3 ? e3 instanceof Uint8Array ? [e3, 1] : ArrayBuffer.isView(e3) || e3 instanceof ArrayBuffer ? [n(e3), 1] : [t(e3), 0] : [void 0, 0];
  }(i2);
  if (e2 instanceof File) return { o: d(f2 || t(e2.name)), u: BigInt(e2.size), l: o2 };
  if (e2 instanceof Response) {
    const n2 = e2.headers.get("content-disposition"), i3 = n2 && n2.match(/;\s*filename\*?=["']?(.*?)["']?$/i), a2 = i3 && i3[1] || e2.url && new URL(e2.url).pathname.split("/").findLast(Boolean), s2 = a2 && decodeURIComponent(a2), u2 = r2 || +e2.headers.get("content-length");
    return { o: d(f2 || t(s2)), u: BigInt(u2), l: o2 };
  }
  return f2 = d(f2, void 0 !== e2 || void 0 !== r2), "string" == typeof e2 ? { o: f2, u: BigInt(t(e2).length), l: o2 } : e2 instanceof Blob ? { o: f2, u: BigInt(e2.size), l: o2 } : e2 instanceof ArrayBuffer || ArrayBuffer.isView(e2) ? { o: f2, u: BigInt(e2.byteLength), l: o2 } : { o: f2, u: u(e2, r2), l: o2 };
}

I don't know why rollup does this and I don't know if this problem can be fixed with this Issue.

EDIT After updating rollupjs, which in my case comes with vite and maybe is also used by nextjs, everything works fine again.

Touffy commented 3 months ago

So it was a bug in rollup, and they fixed it ? I'm glad to hear it. Thanks for investigating. I was really scratching my head wondering how rollup could believe that name could only ever be falsy.

@Angel-Ponce are you also using rollup ? can you check if updating rollup also fixes the issue for you ?

I don't think it's the same bug for @hwyatt, as I checked your deployed code in january and didn't spot any transformations there.

Angel-Ponce commented 3 months ago

Unfortunately I'm currently working on a Legacy project and the frontend infraestructure it's builded over Webpack 4.X.X

I'll try to investigate if webpack fires something similar to rollupjs?

Thanks for everyone.

Angel-Ponce commented 3 months ago

After checking a littel bit the behaviour of my implementation I confirm that client-zip works properly only if I wrap more than 1 file into the final zip

It means, if I compress only 1 file the resulting .zip breaks.

Angel-Ponce commented 3 months ago

So, it seems that my problem it does not part here, because Webpack 4.x.x works fine to build my project

Angel-Ponce commented 3 months ago

omg, I discover of my implementation checks berfore compress (downloadZip) if the amount of files are more than 1, and, if only exists 1 file I don't perform a compression, but I forget to change the final extension of this process.

It means that there are not problems with client-zip, it's directly a bug with my implementation

Sorry.

Touffy commented 3 months ago

Thanks. I am relieved to see an explanation and it's not even a bug in client-zip.

With any luck, @hwyatt and @freedomsky11 will find similar problems in their code or bundler…

Angel-Ponce commented 3 months ago

Yeah, I think it's possible, there not seem like a bug in client-zip