rajaganeshh
commented
2 years ago Hi All, Is the above issue has been resolved? if yes, Could you please give some guidance for the fix
Open ghost opened 3 years ago
rajaganeshh
commented
2 years ago Hi All, Is the above issue has been resolved? if yes, Could you please give some guidance for the fix
This issue was originally opened by @bmoyles as hashicorp/packer#10807. It was migrated here as a result of the Packer plugin split. The original body of the issue is below.
Description
When NVMe-capable instances were released, the Amazon chroot builder was changed to take an explicit NVMe path to the real NVMe device and that is checked against the path under
/sys/block. Amazon Linux (and some other distributions) ship with tooling and udev rules that look in the attached NVMe block device's metadata for the block device name that was passed to the EC2 API on attach and set up the appropriate symlinks under/dev/to the real NVMe device (which is fantastic as the NVMe device order is not guaranteed).It would be fantastic (and I would be happy to submit a PR if no one is able to take this on) if we could provide a configuration option to the chroot builder that indicates it should look under
/devfor a symlink to the block device based on the device in the attach request. eg if one asks forvol-deadbeefto be attached to/dev/xvdgin the attach request, there will be a/dev/xvdgsymlink pointing to the real block device.(A step further might be to have Packer itself use
nvme id-ctrlor some Golang-native equivalent do the inspection of the NVMe devices to find the requested volume which would allow this to work on distributions w/o the necessary udev configuration, but that's a much bigger change and I think simply indicating that this NVME-capable instance will have symlinks for Packer is sufficient for a first-pass).Use Case(s)
At Netflix, we have multiple instances baking images in parallel (with multiple processes per-instance) which makes the NVMe device much less predictable, but we DO have the udev configuration rolled into our base images such that we always have the correct symlink for EBS volumes on NVMe instances (whether the request was made for xvd or sd -- we create both symlinks to eliminate confusion).
Potential configuration
I might approach this by simply allowing one to explicitly specify their device-prefix-of-choice via configuration which would short-circuit the code in
devicePrefixhttps://github.com/hashicorp/packer/blob/master/builder/amazon/chroot/device.go#L15 to avoid the scan of/sys/blockfor the block device. This, to me, would indicate that I am asserting my device, when attached, will have a file (symlink) under /dev/ with the appropriate prefix + letter (and/or partition as necessary) so just look for the presence/absence of those links under /dev:Alternatively, we could go down a different path altogether with explicit configuration such as
signaling that a) we're on a NVMe-enabled instance, b) if we request the device be attached to /dev/xvd or /dev/sd, there will be a symlink created when the device is attached, and c) presence or absence of those symlinks is sufficient to determine whether or not our block device is present.
I could approach this via a custom plugin that duplicates the chroot builder and extends the functionality internally, but this seems generally-applicable so figured I'd pose it here before going down that route. If I went that route, I personally would probably lean towards allowing the user to explicitly declare the block-device prefix as it seems like the route with the least moving pieces.