I think this is just a visual issue, but if you select "Install to Hard Drive" and then select "Cancel", the installer appears to continue the installation. In my particular case, I didn't have any storage device connected, other than the bootable USB, so I'm not sure where the installation was trying to write. Fortunately my bootable USB continued to operate, so I don't think the installer tried to overwrite it.
I think this is just a visual issue, but if you select "Install to Hard Drive" and then select "Cancel", the installer appears to continue the installation. In my particular case, I didn't have any storage device connected, other than the bootable USB, so I'm not sure where the installation was trying to write. Fortunately my bootable USB continued to operate, so I don't think the installer tried to overwrite it.