Docker Persona: Image construction within ImageC

[sflxn]

Acceptance Criteria:

• [ ] ImageC calls docker diff code to retrieve all the layer diff tar
• [ ] ImageC properly calculates IDs unless the diff code gives us the IDs
• [ ] ImageC properly assembles history in the form that Docker expects
• [ ] ImageC properly assembles image config in the form that Docker expects

[sgairo]

This now will also include the responsibility of building the image layers.

[sflxn]

The image and manifest building currently works but have some bugs that we're currently working through.

Unit test for the manifest building is not complete. It's quite involved as it requires a lot of mock data in our persona cache and storage portlayer. Some code may need to get refactored.

[sflxn]

We are stopping work on docker push and postponing to VIC 1.3, pending approval from PM. Until we can guarantee the layer archive returns the same digest as Docker, we do not want to put push in VIC.

[chengwang86]

As an update for the current status:

After we push an image to the registry (e.g., harbor) using vic's docker push (if the EOF error is fixed),

• if we pull it again using vanilla docker -- the daemon returns error msg layers from manifest don't match image configuration, and the failure occurs on this line https://github.com/moby/moby/blob/master/distribution/pull_v2.go#L673, which means the layers' diffIDs from the manifest are different from the diffIDs specified in the imageConfig

• if we pull it again using vic's docker pull -- it will succeed since vic's docker pull code does not verify whether the layers' diffIDs match the ones in the imageConfig. -- however, if I further run docker -H... run alpine (taking alpine as an example), I've got this error msg Server error from portlayer: stat /bin/sh: too many levels of symbolic links. This may be addressed after https://github.com/vmware/vic/issues/5779 is done.