search

FreeCAD / FreeCAD-library

A library of Parts for FreeCAD. WARNING - This library is huge. It might take a long time to download and make the addons manager unresponsive for many minutes.
1.27k stars 1.06k forks source link

FreeCAD parts library is corrupted #331

Open lfrnndz opened 3 years ago

lfrnndz commented 3 years ago

Hi, devs!

I downloaded the zip file, but I get this message upon extraction of the file.

image

I also can't get the parts library installed using the addon manager. I let it run for over two hours, but FreeCAD just seemed to hang. For comparison, I downloaded the zip file here in github for about 30 minutes.

luzpaz commented 3 years ago

We need to do something about the size of the repo it's enormous and GH maybe throttling it.

CC @yorikvanhavre @chennes

chennes commented 3 years ago

If the download worked fine direct from GitHub, it's probably the Python url lib request that's borking. A quick look at the code makes me think it's actually trying to keep the whole thing in RAM instead of streaming it to disk. That's a pretty easy change, and I'm working in AddonManager now anyway. I'll go ahead and implement that.

bitfarmer commented 2 years ago

Some more insights on this problem. I twice tried a direct git clone in a terminal session overnight, protected from logout by screen and on both occasions it aborted at some stage during the night. I then tried a direct zip download from GitHub and that is indicating, as above, to be corrupted: unexpected end of archive. I am now trying a shallow-clone to see if there is any hope of obtaining the library.

For what it is worth, I suspect that something is overflowing its number field width, either 32 bit (blocks) or 64 bit (bytes). If the shallow clone works, I will report later

bitfarmer commented 2 years ago

That didn't work either. I suspect that the shallow clone is nearly as big as the normal one. In case anybody is analysing this, here is the terminal log:

Cloning into 'parts_library'...
remote: Enumerating objects: 11809, done.
remote: Counting objects: 100% (11809/11809), done.
remote: Compressing objects: 100% (8146/8146), done.
client_loop: send disconnect: Broken pipe0 GiB | 354.00 KiB/s
fetch-pack: unexpected disconnect while reading sideband packet
fatal: early EOF
fatal: fetch-pack: invalid index-pack output

The total downloaded amount was 4103GiB (4302741503 bytes) before aborting. And whats interesting is none of these numbers is near a 32 bit or 64 bit integer limit. Maybe something does overflow internally.

chennes commented 2 years ago

I would very much like to fix this "for real" and not just find a temporary workaround. I've started a discussion in the forums here: https://forum.freecadweb.org/viewtopic.php?f=8&t=63140

ghost commented 2 years ago

One of the best way to reduce the size of the repo is to squash all the commits messages. With a git client it freezzes when im seeing the commits