search

EqualMa / gitpkg

use a sub directory of a github repo as yarn / npm dependency directly
https://gitpkg.vercel.app
315 stars 19 forks source link

500 Internal Server Error #25

Open mansona opened 3 years ago

mansona commented 3 years ago

I have just used this tool to install one sub-package and it worked great! I am trying again with another package from the same monorepo but now I'm getting a 500 error 🤔

I'm assuming that this is just a bit of a blip on the serverside (since it's a 500 error) or a rate-limiting thing somewhere but in the off chance that it's an actual problem I thought I should open an issue to let you know.

Here is a snippet from the full verbose log so you can see what I'm doing: 

32 verbose stack Error: 500 Internal Server Error - GET https://gitpkg.now.sh/ember-fastboot/ember-cli-fastboot/packages/fastboot?rehydrate-config
32 verbose stack     at /usr/local/lib/node_modules/npm/node_modules/npm-registry-fetch/check-response.js:123:15
32 verbose stack     at runMicrotasks (<anonymous>)
32 verbose stack     at processTicksAndRejections (internal/process/task_queues.js:97:5)
33 verbose statusCode 500
34 verbose pkgid https://gitpkg.now.sh/ember-fastboot/ember-cli-fastboot/packages/fastboot?rehydrate-config
35 verbose cwd /Users/mansona/git/opensource/ember/ember-website
36 verbose Darwin 19.5.0
37 verbose argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "-D" "https://gitpkg.now.sh/ember-fastboot/ember-cli-fastboot/packages/fastboot?rehydrate-config"
38 verbose node v12.18.1
39 verbose npm  v7.7.5
40 error code E500
41 error 500 Internal Server Error - GET https://gitpkg.now.sh/ember-fastboot/ember-cli-fastboot/packages/fastboot?rehydrate-config

Let me know if there is anything else that I can provide you with to help debug 👍

mansona commented 3 years ago

So I thought this might be related to the other reports of the pkg being too big so I cloned the repo and tried to run it locally.

And, turns out, it just works when running locally 🙃 I don't know if you have any logs on the server that would give you more information as to why it was failing but I don't think I'm able to debug this from my side any more 🤔

royduin commented 2 years ago

I'm also getting a 500 error, for example with: https://gitpkg.now.sh/royduin/reactivesearch/packages/vue?patch-4&scripts.postinstall=yarn%20install%20--ignore-scripts%20%26%26%20yarn%20run%20build-es&scripts.build-es=nps%20build.es any thoughts?

EqualMa commented 2 years ago

@royduin Hi! It seems that there are some bugs with the simple format url. You can try out the formal api: https://gitpkg.now.sh/api/pkg?url=royduin/reactivesearch/packages/vue&commit=patch-4&scripts.postinstall=yarn%20install%20--ignore-scripts%20%26%26%20yarn%20run%20build-es&scripts.build-es=nps%20build.es

Really sorry for leaving this issue unresolved for so long. I've been working on my dissertation for a long time. Once it is fininshed, I will be back for my open source projects. ♥

royduin commented 2 years ago

Awesome! Thank you 💯

ziogaschr commented 2 years ago

I confirm this doesn't work for me too. I tried with both simple and advanced format URLs. I receive the 500 Internal Server Error.

I am trying to install this https://gitpkg.now.sh/api/pkg?url=etclabscore/safe-core-sdk/packages/safe-core-sdk&commit=feat/etc

EqualMa commented 2 years ago

@ziogaschr Hi, I just tried the url and it worked. Could you please provide more information about the error, like the detailed response body or a screenshot?

ziogaschr commented 2 years ago

@EqualMa thanks for the fast reply, it works when running yarn install on the host machine, but not from within docker.

EqualMa commented 2 years ago

@ziogaschr Can you docker exec into that container and run curl or wget to check out the detailed message?

ziogaschr commented 2 years ago

It fails during the build. So I have tried to add it as a step in my Dockerfile and the outputs are:

EqualMa commented 2 years ago

@ziogaschr You can replace all gitpkg.now.sh to gitpkg.vercel.app and try again. (The yarn.lock would change)

ziogaschr commented 2 years ago

This seems to have worked. Thanks a lot @EqualMa

aolchik commented 1 year ago

I'm also getting 504 GATEWAY TIMEOUT error when installing https://gitpkg.now.sh/api/pkg?url=aolchik/netlify-cms/packages/netlify-cms&commit=master

May it be to the repo size?

EqualMa commented 1 year ago

@aolchik Yes, large repo might lead to long processing time. You can try to create a branch which removes unused large files in a new commit.

subzero10 commented 1 year ago

Hello, thanks for this amazing tool!

I am facing an HTTP 500 error and stumbled on this issue. Is it still an issue or am I not using the API correctly? Context:

I've tried calling the gitpkg API in different formats, both with npm and yarn:

FFdhorkin commented 1 year ago

@aolchik Yes, large repo might lead to long processing time. You can try to create a branch which removes unused large files in a new commit.

This isn't an option when you don't own a repo. It would require you to fork a repo, and at that point, you might as well bypass using GitPkg in the first place.

estyrke commented 1 year ago

@FFdhorkin could you elaborate? I can think of two ways to bypass gitpkg in an own fork: one is to publish a forked NPM package, the other is to rewrite the repo so the subfolder you need at the root. Both seem kind of tedious compared to gitpkg...

FFdhorkin commented 1 year ago

@estyrke I was working on a Mattermost plugin. They're consolidating several packages into a very large monorepo. There's package.jsons inside folders, but no way to just install the individual dependency I needed.

The packages aren't published to NPM, either.

Attempting to use gitpkg gave me 500s. So I ended up having to clone the entire repository.

That was my whole point: making a fork to install a dependency is tedious. That's why your comment about "just make a branch" isn't a viable way to use gitpkg on large repos. You can't make a branch in a repo you don't own without making a fork.

The whole reason I wanted to use gitpkg was to avoid that, but if gitpkg can't support large repos, what's the point of only using a subfolder as a dependency rather than the whole repository?

estyrke commented 1 year ago

@FFdhorkin Yes, I agree totally! (it was someone else who made the comment about "just make a branch")

You said "It would require you to fork a repo, and at that point, you might as well bypass using GitPkg in the first place.", but even with an own fork, I can't think of a good way to bypass GitPkg. I can think of a few not so good ways and was just wondering if you knew of any better methods (given that I already have a fork). I ended up publishing a forked package to npm, but it feels less than ideal. :)

Super-Chama commented 1 year ago

I faced the same issue of 500 server error when i added few static assets to one of the packages we use via gitpkg. must be due to increased file size. so i created a simpler self hostable version without relying on vercel. it's very simple and have fraction of flexibility offered by gitpkg but sharing it if anyone is interested in using for their own projects. https://github.com/Super-Chama/gh2npm

@EqualMa thanks for creating gitpkg and tar-transform it was very inspiring

coyotte508 commented 7 months ago

Hi!

This url is crashing with a 500 as well: https://gitpkg.vercel.app/api/pkg?url=huggingface/hub-docs/tasks&commit=9c675770ca275899f36c4e489b36009cab4effff