search

zendframework / zend-stdlib

Stdlib component from Zend Framework
BSD 3-Clause "New" or "Revised" License
384 stars 76 forks source link

Missing Commit #6

Closed rawkode closed 9 years ago

rawkode commented 9 years ago

Hi,

Our composer install is failing (Using prefer-dist on our build server) and a commit / tgz seems to have disappeared?

  - Installing zendframework/zend-stdlib (2.3.0)
    Downloading: 100%         
    Failed to download zendframework/zend-stdlib from dist: '/Users/rawkode/Development/TeamRock/Blackbird/symfony2-application/application/vendor/zendframework/zend-stdlib/Zend/Stdlib/b6a780f333e513e358f7df0ebdf9c6cb' is not a zip archive.
    Now trying to download from source
  - Installing zendframework/zend-stdlib (2.3.0)
    Cloning d1c481b8a43f2f079b16d3567960ba539e9dacaa
    d1c481b8a43f2f079b16d3567960ba539e9dacaa is gone (history was rewritten?)
Maks3w commented 9 years ago

Please run composer update for update your composer.lock

rawkode commented 9 years ago

Unfortunately another package we use is looking for this specific version

Maks3w commented 9 years ago

@rawkode Does not matter. Just update so your composer.lock takes the new reference (426b5396e89e7da2db9678bc9a0b57865f84fe0f)

rawkode commented 9 years ago

OK - Thanks.

lordelph commented 9 years ago

I'm getting the same issue but composer update doesn't seem to shift it

Installing zendframework/zend-stdlib (2.3.0)
    Downloading: 100%         
    Invalid zip file, retrying...
  - Installing zendframework/zend-stdlib (2.3.0)
    Downloading: 100%         
    Invalid zip file, retrying...
  - Installing zendframework/zend-stdlib (2.3.0)
    Downloading: 100%         
    Failed to download zendframework/zend-stdlib from dist: '/var/www/connect/vendor/zendframework/zend-stdlib/Zend/Stdlib/62f20877145837f45f6036f29d063c52' is not a zip archive.
    Now trying to download from source
  - Installing zendframework/zend-stdlib (2.3.0)
    Cloning d1c481b8a43f2f079b16d3567960ba539e9dacaa
    d1c481b8a43f2f079b16d3567960ba539e9dacaa is gone (history was rewritten?)

  [RuntimeException]                                                                                                                              
  Failed to execute git checkout 'd1c481b8a43f2f079b16d3567960ba539e9dacaa' -- && git reset --hard 'd1c481b8a43f2f079b16d3567960ba539e9dacaa' --  
  fatal: reference is not a tree: d1c481b8a43f2f079b16d3567960ba539e9dacaa
weierophinney commented 9 years ago

Honestly, I'm not sure what's going on here; the sha1 you're attempting to get does not exist anywhere in the tree, and, more interestingly, we imported the old component repository to the legacy branch to ensure all sha1's that existed previously would remain available.

I'd try doing the following:

This should grab the tag correctly at this point. If it still doesn't, I'm out of ideas.

lordelph commented 9 years ago

It's not a cache issue, this occurs during the build of a docker container from scratch.

Looking at my dependencies, I see two packages with zend dependencies

I'm not using that older version of guzzle, that's been pulled in by geoip2/geoip2 - geoip2 has a beta which doesn't have a guzzle dependency, so I've temporarily gone with dev-master. That eliminates the zend-cache/zend-log dependency but that wasn't really the source of my problem.

The master branch of ob/highcharts requires "zendframework/zend-json": "~2.3", rather than "2.3.0", so I went with dev-master there too (at least until that is released). That means I'll pull in zend-json and zend-stdlib 2.5.1. Sure enough, that solved the problem.

lmanzke commented 9 years ago

This problem is very annoying and it is very recent. I have seen it on multiple machines repeatedly. It worked on my machine and a day later crashed on a colleague's one... That is so annoying in fact that im seriously considering arguing to remove the zend dependency completely and switch to different libraries.

Btw: Recommending composer update is a poor recommendation and not always an option. I have come to the conclusion that the repeated occurrence if this is related to zend (because I never ever saw this error with anything else than Zend), not to composer. So fixing this should be up to you! Here the most recent occurrence I encountered (where the composer.lock's age is around 1 day):

[exec]     Failed to download zendframework/zend-stdlib from dist: 'stdlib/Zend/Stdlib/9c5999b1f89b37acb6bd83698b3b86c3' is not a zip archive.
 [exec]     Now trying to download from source
 [exec]   - Installing zendframework/zend-stdlib (2.3.1)
 [exec]     Cloning c1f4830018b5d4f034d32fa01a9e17ea176f56f6
 [exec]     c1f4830018b5d4f034d32fa01a9e17ea176f56f6 is gone (history was rewritten?)
weierophinney commented 9 years ago

@lmanzke Honestly, I cannot, and never have been, able to reproduce it. I can verify that neither sha1 you reference is present, but I can also verify that neither was ever present in the original Component_ZendStdlib repository; the legacy branch of this repo has that entire history, and neither sha1 is present in that branch.

At this point, I've hit a wall; I cannot help you further, as (a) I can verify that those sha1 values are not present, so of course they will not install, and (b) I cannot verify that those sha1 value were ever valid.

What I can note is that it makes no sense that composer is mapping the 2.3.1 release to c1f48300, since that sha1 does not exist; that release tag currently maps to 6079302. This leads me to believe that you have a stale $HOME/.composer/cache and/or a MITM is seeding composer with invalid values. One step you can take is to execute composer clear-cache and attempt to install and/or update after that.

I recognize that you've already said "Recommending composer update is a poor recommendation and not always an option," however, I have no other options to present to you. We do have the entire original history in the legacy branch of this repository (which was done to ensure dist and source installs referenced by sha1 would continue to work); the installs you've shown are referencing sha1 values we do not have, and never have.

akrabat commented 9 years ago

Not sure this helps, but…

The composer.lock that's checked into this zf2_sample_app contains the d1c481b8a43f2f079b16d3567960ba539e9dacaa hash.

The composer.lock that's checked into Piwik-LdapConnection contains the c1f4830018b5d4f034d32fa01a9e17ea176f56f6 hash.

lmanzke commented 9 years ago

@weierophinney I'm quite sure that the error was not on the composer side, as zend is the only library making me constantly hit that error message (and wanting to chuck something out the window because of it). I believe you are right that the hash is not present, but on the same note, you are probably mistaken in saying it never was there. As @akrabat points out, even composer.lock's in the zf2_sample_app contain these hashes.

I think composer is quite right in asking "history was rewritten?", because to my mind, that is the most likely explanation. I am not sure about your processes, but this seems like a history rewrite to me. If that is the case, then of course at some point git will remove certain commits like the ones missing as a part of its garbage collection (As no branch / tag / anything points to the commit anymore, making it look like it was abandoned and not needed anymore). This would explain why you came to the conclusion that it never was there, because your legacy branch would naturally also not contain it.

Please investigate that. Do you have anything rewriting history by any chance? (Maybe automatically?)

weierophinney commented 9 years ago

@Imanzke There's a ton of history behind this.

Prior to 2.5, we were using a manual process to sync the various Component_Zend* repositories from the main zf2 repo at each release. The process would literally go commit by commit on the zf2 repo, sync in the files for the given component, see if any change happened, and, if so, write a new commit referencing the commit on the main zf2 repo; tags were then done based on a combination of sha1 + date. All commits were done only on the master branch; there was only one branch in any of these repositories. Furthermore, we only synced the library code for any given component. As you might imagine, the process was error-prone and untenable long term.

One of the big pushes after 2.4 was to create first-class repositories for each component, complete with tests and full history (not just references to the original repo). I've detailed that process elsewhere. We did this split into new repositories, dropping the Component_ prefix, and normalizing the names. Once the split was complete, we re-pointed packagist at the new repositories.

When we finished, I received a number of reports of people's composer.lock files breaking due to the fact that they were pointing at the Component_Zend* repositories, and updates were grabbing from the new zend-* repos, where the existing sha1 no longer existed. At that time, I copied in the master branch of each Component_Zend* repo into the corresponding zend- repository as a legacy branch to ensure all previously existing sha1s were available. The reports I had after doing this indicated it resolved the issues observed.

So, yes, history was rewritten, but the sha1s from the original component repositories are theoretically all present. I literally have no additional paths to explore at this time; we deleted the Component_* repos to prevent confusion from developers looking for the various component repositories, but only after the importing their master branches into their sibling repositories. We should have all the history; your report indicates we don't, but I have nowhere else to look.