Closed ghost closed 5 years ago
@lgierth what is the meaning of the first column? is it size in bytes?
Yeah roughly, it's CumulativeSize from ipfs object stat
These are pins logged on the old pinbot host (deimos), they've already been been moved into cluster, just putting them here for safekeeping as I'm tearing down that old pinbot host.
The following are reported as broken by pin verify
and won't be imported to new cluster (they get stuck)
Edit @lgierth: These aren't actually broken, it just turns out they weren't being reprovided for whatever reason. I'm copying them to another, faster host now (biham is flatfs on ext4). The only one that can't be transferred is QmbdLBA51HsQ9PpcED1epXAxLfHgrd2PDZ3ktmjhFTjg94 because its root node is too big for Bitswap.
The following pins are not getting pinned from archives.ipfs.io because archives.ipfs.io is not pinning them:
QmfXH9XtP7xmoTH8WAp4HNSduqWMwLTH8B8TvbTkdgzNAa
Qmaskk1Egq5zmZsGTd7dwNiiK1cwfmx7k1StG1WJQjwGDm
QmRrnfFUgx81KZR9ibEcxDXgevoj9e5DydB5v168yembnX
QmbXX6jkJSx1aH41nfBqPyZAgxe1m4CzMMVnhYz1xRyFDQ
QmTC8ADX28gEWH68SsuduWoropCad9g9bfK2b2WWwYggPv
Qmbs2DxMBraF3U8F7vLAarGmZaSFry3vVY5zytuN3BxwaY
seems archives.ipfs.io is just an nginx with forwards to 2 hosts running ipfs. Both are only pinning 1 thing:
root@gateway0:~# curl localhost:5001/api/v0/pin/ls?recursive=true
{"Keys":{"QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn":{"Type":"recursive"}}}
After months I have been unable to pin 8078545828 QmaJGL73SdbjUnZPEsA8ytzPQUHzYtR7UBebEkfCLj2rCB dev tr.wikipedia-on-ipfs.org
from auva. pin verify
throws some errors there. I'm leaving this one as impossible.
I'm gonna call nihal lost -- a disk silently died and that's what very likely trashed the ipfs/badger repo. About its data:
tl;dr it's fine
Try the pins from biham once more -- they're on a way faster filesystem now, and are also properly provided to the network now.
(Except 20118812895 QmbdLBA51HsQ9PpcED1epXAxLfHgrd2PDZ3ktmjhFTjg94
whose root node is too big for Bitswap.)
ack @lgierth , but I think biham was fully imported already, twice. I'll re-do, doesn't hurt
ack @lgierth , but I think biham was fully imported already, twice. I'll re-do, doesn't hurt
The broken pins aren't broken anymore :) It just didn't announce/provide them for some reason... and QmbdLBA51HsQ9PpcED1epXAxLfHgrd2PDZ3ktmjhFTjg94's root node is too big for Bitswap.
@lgierth most pins have dissapeared from biham though:
root@biham ~ # docker exec -ti ipfs ipfs pin ls --type=recursive
QmT5NvUtoM5nWFfrQdVrFtvGfKFmG7AHE8P34isapyhCxX recursive
QmbdLBA51HsQ9PpcED1epXAxLfHgrd2PDZ3ktmjhFTjg94 recursive
QmcvfB6pAqUfTnuAK8zFKVxbdhopnBPveJrDcy1JAA7HX5 recursive
QmezTdYeKyjPFoREStJQQbvATUP8yRJdHMMZx2rZ86p9g9 recursive
QmR1gzPYUwxEUWHbeRggZzfYy5Fxsd8Qc7hXUUnJQwxrZq recursive
QmR2R6iCKY8KGdvDEkuv2K9CL5tusAi74qKftdrTTdHQTn recursive
This intended?
I'm pinning those (they weren't pinned), except *94 which you say wont work.
most pins have dissapeared from biham though:
Yes I replicated them to scrappy and chappy, and then unpinned them on biham. Biham was acting really weirdly regarding providing those pins...
So do pin those too :)
Hi, I think all the pins in the servers posted here are migrated to the storage cluster (those that are not broken somehow).
The only things pinning
now (since a long while ago) are:
Qmaskk1Egq5zmZsGTd7dwNiiK1cwfmx7k1StG1WJQjwGDm | "scholarpedia.org"
Qmbs2DxMBraF3U8F7vLAarGmZaSFry3vVY5zytuN3BxwaY | "pgp public key database"
QmbXX6jkJSx1aH41nfBqPyZAgxe1m4CzMMVnhYz1xRyFDQ | "online encyclopedia of integer sequences"
QmfXH9XtP7xmoTH8WAp4HNSduqWMwLTH8B8TvbTkdgzNAa | "arxiv"
QmTC8ADX28gEWH68SsuduWoropCad9g9bfK2b2WWwYggPv | "no-intro collection"
QmZrL4YkmgC7x3ZAicHx9s3fkoNgVmg87zHMSsmjRnhM23 | dweb
@hsanjuan is this work finished now?
These pins are still pinning. I don't think there are providers...
Okay, we're done here, I'm updating the no-intro collection dataset on archives.ipfs.io, the dweb thing was actually an /ipns key, and the other four datasets are in various stages of being lost.
724 GB from auva.i.ipfs.io (click to expand)
``` - zdpuAqN6LUsggXfSWDhAzmnLAkWFztsR1w1QhKr7mBp2wfBMK tr.wikipedia-on-ipfs.org search index 4 QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn empty unixfs dir 6 QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH empty unixfs file 181 QmZtHVrH3p9iE39K1j4u2uxfzvzBKUqQa3LbUafLwHurBi dev distributed-wikipedia-mirror 187 QmU9UuSv5uEHyqvAoRtKdupLp5jCFCurKaq2H8vz5Rs4rK dev distributed-wikipedia-mirror 191 QmNqbYogAxH4mmt5WhuKN7NFEUDZ9V3Scxh7QbLwTKBJDk dev distributed-wikipedia-mirror 2854 QmaqE51tmbT5vRtZTzzvcnLPyA7XqAhE9PS51xtysD66h8 ipfs.io dns zone file 5979 Qmc4aqtsr8ysMo3TYPg8tPD5iAhrawTCR4ytfe5SxYA77w dev distributed-wikipedia-mirror 5980 QmRTDua6jcAjTQMbEjBVVVv5KM9wamy7afKiaLXLHZaxk6 dev distributed-wikipedia-mirror 5981 Qmb1zF66Q13GD7UJMgZfkeJ1bgFsGFPhYvp1ASaXhRurhA dev distributed-wikipedia-mirror 5987 QmXGVeLvNv1b3rvEigQrjuoiFUwULczeCAhqVgzgMCpmDL dev distributed-wikipedia-mirror 6023 Qmc5hoLqpkgesbT1pEuRiHYHUuGGmbVpRsKeXWjWW4wqzm dev distributed-wikipedia-mirror 6077 QmS3CJ6hKh29pZLEoCgGKdTc8Kpximkc13u585dP3PHwmF dev distributed-wikipedia-mirror 6079 QmRCtJrmqxL47hDh6kMF9cKDhTGSL4maATUnoNfu6TmNyS dev distributed-wikipedia-mirror 6083 QmUxHJHvgS4xzRSPZ5987BDLCFqLSMtUu8PFoUsDnUqbT6 dev distributed-wikipedia-mirror 6125 QmcwcmFeCtC7vddbvSbof8dwtrPogG9HJnL5kYVnkFAUtg dev distributed-wikipedia-mirror 6238 QmfGabjT8ygoXvhkBNPgehmkHVaT2PxEhSbrNkypZeqaTA dev distributed-wikipedia-mirror 6376 QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG ipfs init files 6580 QmVLDAhCY3X9P2uRudKAryuQFPM5zqA3Yij1dY8FpGbL7T ipfs init files 6831 QmSSu91q7sU2qTBVt7X3KbA7dHbuR1RiNLuS4RNBv9APU1 dev distributed-wikipedia-mirror 6833 QmXBLgtqjpBd5FQTzofsruBkPQomrwPRD8jDGQQY2ET24Q dev distributed-wikipedia-mirror 6840 Qmbsap9dckwKbBcEydpTv2rzYnGzHoax2Qjc5KSvkRAM6B dev distributed-wikipedia-mirror 7022 QmTbXZ3YqNDBAQTQkifinWymu4y3kcHZVQcxC16Pf7R7Fq dev distributed-wikipedia-mirror 7653 QmQUMkhkqnSD25wu1rMhk9P1EkCVAF7pCojXvywW9iQnh6 dev distributed-wikipedia-mirror 7654 QmPfsE3agVx4pkBYbpAp8C9kagKXDy15xTu8QN82w9jKbu dev distributed-wikipedia-mirror 7756 Qmdk5tF2x8Mdb85puC4WRWTJDBF3wTGLwDDKqy1x4nJgee dev distributed-wikipedia-mirror 7765 QmSCnFG9oewVz6SAhW26cvTmhXPnUjrvA6XUqYFc4kPeJ8 dev distributed-wikipedia-mirror 7766 QmcAtSs3w3L6yzYiNWA9C5d8Goet2q33t1pogdbi4uuVcV dev distributed-wikipedia-mirror 7791 QmTEsjwJMetntQkb91vpx3KWH2vBHqmKjYdvuycX772vjj dev distributed-wikipedia-mirror 7926 Qme4SpUEEa2kMUE7WpuCmMvdTMxTekpFTSHXmwpTpYKMws dev distributed-wikipedia-mirror 7928 QmcYU4iaULU5arUwbnnH422CWc4nRswRbztDENCjGS9gmf dev distributed-wikipedia-mirror 7975 QmcmKEq4tqZ33QuRzAZRgZK96jBa69Dc4KyTBMKVZ1rZDC dev distributed-wikipedia-mirror 8029 Qme7cUZxpCqHvtt8s2JS1RyEjsMX4btc6SDBiGZyr1dm6f dev distributed-wikipedia-mirror 8172 QmcQFYiLcBKSaVfFk4spmcSEpXYP7hnmcW4tiwb6cDEFQq dev distributed-wikipedia-mirror 8210 QmVTYu5MqBiEcCMKZpXwXViXLvUL3UMo5ErRMgAHULpSFu dev distributed-wikipedia-mirror 8255 QmSwckkTrktiPbacQBhox4KHwSdNEyGn62HRipt42Gvbf4 dev distributed-wikipedia-mirror 8276 QmbN1zPdSW1FNG7sf17oUxLF5Lfid5YW8NKSwbEZ75B9v7 dev distributed-wikipedia-mirror 8314 QmXFG1QJSkpPtdv9NqUFdvW6VNDhxZpo3L58kqXojfAfyR dev distributed-wikipedia-mirror 8316 QmV3ifvKkuhQauQMyoXiji4oD5cMzB5YGqcFmMBzcxrMje dev distributed-wikipedia-mirror 8465 QmVyP28H1r9DutKBEo7hpeV3ccPDuz2sJ4F4dcQwvsjMtd dev distributed-wikipedia-mirror 8498 QmVUEbPixiy7cei7h69jEj2Yax6XgYASzEWY7BSuif7n4r dev distributed-wikipedia-mirror 8656 QmT1UKaAkBqsa93jqu842XHqgSv9zM5aMLfeWgmMPpwsJg dev distributed-wikipedia-mirror 8667 QmcfqnwA3YZ3p8SAHNVpzbm2wL9snTo8BcEPPa6f569Bkn dev distributed-wikipedia-mirror 8688 QmckRLJ4pZKkPxhAb9JdazdjJnT88fz2rmT48bJN594EDC dev distributed-wikipedia-mirror 8703 QmU5kc9LYawmmFWNq8rSkd9jMTLMsL5WwzxYHC5HZFAfPG dev distributed-wikipedia-mirror 8721 QmYGAV8D6WrBeXfV8HF9R2nVVD3i544CtWZMQQYnVHRGeP dev distributed-wikipedia-mirror 8743 QmRzuiwABRk2e91nmYngNapPQXcse5q8ZhSc4aiYzrmne2 dev distributed-wikipedia-mirror 8745 QmV3ZvLp7S1GtEi8iRL8CdDcCDKchAK9epYbVs21AyYa5H dev distributed-wikipedia-mirror 8758 QmcUqphZkD3HzS4HvvYYQCYF7xwZHYPh43N6BnTN7RucFa dev distributed-wikipedia-mirror 9045 QmeRtLRE9khyGPpmbZWTMdTiknWtFvbHxzrweDhvk5kGs9 dev distributed-wikipedia-mirror 9154 QmNmgcnJNU12MyE4wJ7jTKJASt6XJzrwLbCi3KyT1uPtx5 dev distributed-wikipedia-mirror 9155 QmYbaJ7BHg8isRnEDJzXYZYbKr46ckdZ97qXNpfbm1ihdP dev distributed-wikipedia-mirror 9194 QmWazt2Fsw3yQ2EX1AaocfD59oLfAF98397jML3TAUsvFX dev distributed-wikipedia-mirror 9194 QmXp2AAt37esDqtXFtuLf83Pc4qZxHcDpvzANWibPi9f1q dev distributed-wikipedia-mirror 9221 QmdgDFvLPyLkDV8zR7r92Ceerx8u5hf968PNxjU51yRZL6 dev distributed-wikipedia-mirror 9238 QmWLqd5AY48igAqJ1x8Rn7UJqJY7P2xZkfDZZ2EFTMWTsY dev distributed-wikipedia-mirror 9244 QmY5n66gUWuFW4KhyoFmPnPbVFzyEG1y9ym3gMK4u7nHwC dev distributed-wikipedia-mirror 9257 QmfKPd6VDKV4ygnsPhzVS3A4UbNNWM2j7Hy2xMNXHdK1Tg dev distributed-wikipedia-mirror 9339 QmRza6ZgeBAmWKqLUmQew99MmNSmx8AnVGnQxK4FPS2u6k dev distributed-wikipedia-mirror 9376 QmaK8ViM9L6odTcUnKTbURm6csA3d5UaDbdbq3Pk7ZkY9i dev distributed-wikipedia-mirror 9395 QmQqMCxp6NLkQQ8kz6GFQgpTqw9QAewBWHqj8Y4YWK3kq6 dev distributed-wikipedia-mirror 9403 QmNuPjYJdYMUCGpdErRufaGP276FZ6vXTatGKnd45pdcvF dev distributed-wikipedia-mirror 9433 QmTodwLVxko8zjsK5bkDq7MWH8f2CSNr5XmS5q7p1csWXG dev distributed-wikipedia-mirror 28755 QmU4h18xe8J4v1mbft4dYzfVywhUPFV6SXz7NLJspgWayb dev distributed-wikipedia-mirror 151113 QmYRDivso51zQC4R7TtK5w9cwUqNg4GN5dug1bu7NadVQq dev distributed-wikipedia-mirror 151114 Qmbh7wGZ87gZsEmnNiTqGXgwmBRT9fA12QoTRTfAAxEQ1h dev distributed-wikipedia-mirror 151115 QmNZXNgB8X2hc63C7RBGDdwctWYC9LkNis14V4qWgwFRV2 dev distributed-wikipedia-mirror 328653 QmZBy4sQgN6jGS1RFtSpkeKopzxQzM4udMhYu9VNLiaBaq dev distributed-wikipedia-mirror 791765 QmeQiE6P8JC7hwBN8Tn92ZA3DTVwmP9CtjGedpQSgpRukU cyphernomicon 987456 Qmd87gkaTn4Gd9qfB3dwLEY2MYKmXqadzoyVTddiBnfsTY random js code 1487077 QmQ2Nt5X95s6gcYqsoF81Pbmu9p7iJqDpccDPpF3VGKYn6 cyphernomicon 3009143 QmSavwBHBEaXwje4QYRSqE3xZaswfoAsqixPyp8wEsuk8Z random js-ipfs 3199323 QmczqxpYyy7qHzxGQo7fPrtdZNKfdS2VekemmB3XT4y8tr dev distributed-wikipedia-mirror 3478553 QmaWnPUwrd4DjG2zsGMrjgttqGHcTJrEBCPvM47PLSLZ9A mixmap https://github.com/substack/mixmap 4907757 QmVkDZ3Xy6tKBYr6zFprM1BDBx32NUL2Hf27ipLW6H5CyE dweb-primer.ipfs.io 5446033 QmbkwXXqNNZgLhHrfhAQXF6jyvNPmEfFwKVTNuyrFFcvq9 ipld slides 5446079 QmeDjE6akebMn4P9Gp9nhMnL9PowRjMDLEvMk3RL2vTfqG ipld slides 7440641 QmUo3J6saWmt7eZiY36iAVuqiuzCbwKPU1MBL28mnc9YeZ book site reliability engineering 7441982 QmNzTPpy9yMDsSyD55Mrq5C2Bh9AgCFRTcUC4oS1g2XVmU book site reliability engineering 8383890 QmVTVoUdYib3D8gbkUcv5Ur4yTxzMrk2Sx9kAHdCuC39ZX go-ipfs debug dump 11218744 QmfHfH2wR3rndD4Sb53Fj9uGtmxKxTbxPTh72Bhk61jcjX book network programming with go 11221154 QmZEcu45NKWyrx2JSn2e7nW1E6H4daEd22QNZDmmXZFGo3 book network programming with go 11510132 QmZQ6tsjn4GLVjvw6L7LCNeLb9istuKT1bTThwd9SPgQEE dev distributed-wikipedia-mirror 12095619 QmYmZeazPp2ZpUA2UBV2jeSLhT5GyAHpMmUN3a51KPDugo ipfs slides 12095687 QmfAVVv6T4tWDovhGnSDoMhu1d51EpAjjXxts3s8Xm5hfs ipfs slides 12444685 QmVb727aLjRy9iW5nHfjtiPo7YiaxicJdp96YEmDvF3fp3 dev distributed-wikipedia-mirror 15459575 QmS9RnpHnduRvV1TFHcXBGffRy6BBGmxHVZqjeta98yXAg blog.ipfs.io 18059115 QmV5PLazMsBk8bRhRAyDhNuJt9N19cjayUSDvw8DKxSmFz mixmap https://github.com/substack/mixmap 18061773 Qme5hTmyRku1m3sjwPoMmsf65msWbiFc5MVKPcP19jj3Kr mixmap https://github.com/substack/mixmap 19925396 QmQZcdXPvpBEzyELDiN2ynXxXAorXXMTJfYKTa47dy7cF6 go-ipfs debug dump 22052715 Qmf1aw4rTrRptNa8zU58hkgC6hBiFxdbtocK9ZWEm9arkh ipld slides 22052771 QmNaWuhU9QHkRQUjnFJxGSvqbeEgrZVrovxFqFUma7LWkn ipld slides 26136384 QmfFLdjQX2xm9A5c3a2HtkpdUgCCcSiM2grUNeyWwGUsDZ ipld slides 26136432 QmTYJEZr5AX5oHv9DNuS4Pm8Tu9zfeYD1bJKi9avJx5gGn ipld slides 28309282 Qmau85zafHEA7mXteh79heMPKTCVQjRtYM8gPgzKxTCNdc go-ipfs debug dump 30191816 QmWYpVRqMrW7QEpqa728CzrBgQa85x5xfhZdzzBcRQ1K77 book the nature of code 30197063 QmcpvuutMYq1UByDQTtwNa4X3rw4N8ZrXfecjHdHZoHUFf book the nature of code 34474961 QmWogkyvoX1n8MAebk2XbDQFU9BTz5Z6PcmmFvD9ce9PZP ipld slides 34475812 QmfUPTKrY3NDJWCvvLUSf4n7uQR1Mbge8Bi7SxapU7Lt6p ipld slides 35428287 QmcChTQVRaY9igSejrhR5DbT9HVQABPuwArpJsrzsFUWDg dev distributed-wikipedia-mirror 35965871 QmcVsJ5v1ivDALYy4AgiqBFFuwuErjkRDXmg648SpUG74s dev distributed-wikipedia-mirror 36276760 QmSvWRvGEYUTfsmNq2BgQ6XXpZhD7ADu4GA8E4Xv9GV9vn ipld slides forest.jpg 61482165 QmZdGfVQvmdKvuj1gJzmdKAifecSw5WQugA9CiGRmuRhqi mars long distance replication software 61491006 QmYZZE7qFyAh6ogbvB2WWYytqYyM4n8UirEn8GYZGKGetp mars long distance replication software 79973627 QmNeLBdWJy73sQ3thgpHKapMHoZrVR9MACZU4KA78136yH ipfs all hands 2017-12-18 93572958 QmNQ1TQzC3D4MMJFN5LVCKPBryRg968QTCvQ29miamxQq8 old fahrplan 33c3 93733093 QmSyTtJLJT1vRtYvrop4oNaZaWugg9f3ndAU4z1coSUDVy broken fahrplan 33c3 109796940 Qmb8wsGZNXt5VXZh1pEmYynjB6Euqpq3HYyeAdw2vScTkQ xkcd 109893912 QmWSp2z15A1xAToFevDo7WmRCGHJHsCu8GvLs5ySTgbAaH xkcd 112193146 QmV2j9qLHYaoMhZ6GjrphPCygAeLxHys5qPTYpHXZwKbHb xkcd 275902964 QmSqKztyUkNYmZkmPQGyMR6xM85oK9zvna8xspHCehW11q auschwitz-prozess.de 275922514 QmZyEz9ZRxrAheCLZBYxVKHgfNs7FsRFyKN5MHzjrhqBoK auschwitz-prozess.de 286437850 QmTK1mFMW1D7qVcDMVv1Vf32A1WfwhbyrktNYGZqHbDYPS random magik6k 321971871 QmNui7ZhbNkjLaYoZEKXgtDi3wWcY4HknpfpHKz2xZMNnD old fahrplan 33c3 377214141 QmSHYyXVcH5QxzhdTVUsJYU576xDfj51L4GfSQi7DeeJtP fahrplan 33c3 377248924 QmdV79vscMxXrNdDa6S6kaRsvoGJKNFiypAD512aNhfvpz fahrplan 33c3 480845779 QmcHZGoZkv11Ntynqxj5HjqaKAgLUvHSwTeREbDsVFESq4 presidential daily briefs 509706943 QmNda8YogQrqzWbbvQbrzpaMjKkkBvMF3pedr7JWkmAuTX ipfs all hands 2017-07-17 3538941892 QmU2TE8ym8rBAMwzaanZ3SbVtBJY4R7oX2uN1fca4JAnrD simple.wikipedia-on-ipfs.org 3538942500 QmY1jXSeNbZAU6iDunjUoQNJSpm8mR9AYckW5u7KzC4B2Z simple.wikipedia-on-ipfs.org 3881621114 QmVL6L6BY3uvoLzuqErFcKDDy916Jg4cQ6tJ8yQC9GoUmi blender.org videos 8078545828 QmaJGL73SdbjUnZPEsA8ytzPQUHzYtR7UBebEkfCLj2rCB dev tr.wikipedia-on-ipfs.org 8090224615 Qmc5TBj66MUPgrBejRDRbDr8nWM5yZmrU9q88MwqHc5JxK dev tr.wikipedia-on-ipfs.org 10352722224 QmeLhKEWoUPMcQEb2mR6mpuNXGWHKNQ7E7Bd1rFBvc68Du jbenet slides 14906884444 QmQP99yW82xNKPxXLroxj1rMYMGF6Grwjj2o4svsdmGh7S dev tr.wikipedia-on-ipfs.org 14906884447 QmbpP1RC5q87YyvCAJPJ2KnaV3TNLLMcqupuYLxjs1hhW4 dev tr.wikipedia-on-ipfs.org 14907877597 QmQYGetfzpDR3pPR8mGq78yA87Jpe3RE3tUxp9nhDv36bW dev tr.wikipedia-on-ipfs.org 14907878530 Qme2sLfe9ZMdiuWsEtajWMDzx6B7VbjzpSC2VWhtB6GoB1 dev tr.wikipedia-on-ipfs.org 658038834798 QmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXWo6uco en.wikipedia-on-ipfs.org ```I'll fill in the others in a bit.