search

GM-Script-Writer-62850 / Ubuntu-Mainline-Kernel-Updater

Makes it easy to install and check for new mainline kernels
117 stars 18 forks source link

wget "Can't timestamp and not clobber old files at the same time." #20

Open EvilSupahFly opened 8 years ago

EvilSupahFly commented 8 years ago

Script is having an issue with wget evidently on Ubuntu Wily, though I can't imagine why since this script has been running fine for years and I use wget frequently and haven't noticed any changes:

root@black-beast:~/Ubuntu-Mainline-Kernel-Updater $ /tmp/kernel-update
Config Notes:
    Rejecting Release Candidates
    Accepting Latest Kernel
    Accepting kernels compiled for *
    Accepting kernels with a version higher than 4.5.0-040500-generic
    Accepting generic kernels

Information:
    Origin: 
        http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.5.2-wily
    Kernel Version:
        4.5.2-040502
    Release Date:
        2016/04/20 @ 03:35 (YYYY/MM/DD @ HH:MM)

    Install Linux Header Package:
        Yes
Care to look at the change log? (y=Yes, n=No) (n)
n
Can't timestamp and not clobber old files at the same time.
Usage: wget [OPTION]... [URL]...
Can't timestamp and not clobber old files at the same time.
Usage: wget [OPTION]... [URL]...

ls: cannot access linux-*.deb: No such file or directory
Failed to download all needed files
Files are in /tmp/kernel-4.5.2-040502
Get files from http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.5.2-wily

There should be 4 files, 3 ending with 'amd64.deb' and 1 with 'all.deb'
Here is what we have:

You can install what you have by running this command:
    sudo dpkg -i /tmp/kernel-4.5.2-040502/linux-*.deb
Just be prepared for broken packages.

Do you have any insights? If I manually download the Ubuntu Mainline .deb's I can still say "no" when it asks if I want to delete and it installs fine, likewise with the dpkg line above..

GM-Script-Writer-62850 commented 8 years ago

I just used it to install a kernel on Xenial, it worked no problem

i used the -nc option in wget

  -nc, --no-clobber              skip downloads that would download to
                                 existing files (overwriting them).

this is to prevent wasting time downloading a file that is already present can you verify the -nc option works in wily i just deleted my wily install files last weekend (replaced with xenial )

xubuntu@xubuntu:/tmp$ KernelUpdateChecker -f -r wily -no-rc
Run this command to install the new kernel
    /tmp/kernel-update
xubuntu@xubuntu:/tmp$ /tmp/kernel-update
Config Notes:
    Rejecting Release Candidates
    Accepting Latest Kernel
    Accepting kernels compiled for wily
    Accepting kernels with a version higher than 1.0.0-1-generic
    Accepting generic kernels

Information:
    Origin: 
        http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.5.2-wily
    Kernel Version:
        4.5.2-040502
    Release Date:
        2016/04/20 @ 03:35 (YYYY/MM/DD @ HH:MM)

    Install Linux Header Package:
        Yes
Care to look at the change log? (y=Yes, n=No) (n)
n
--2016-04-28 22:32:45--  http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.5.2-wily/linux-headers-4.5.2-040502_4.5.2-040502.201604200335_all.deb
Resolving kernel.ubuntu.com (kernel.ubuntu.com)... 91.189.94.216
Connecting to kernel.ubuntu.com (kernel.ubuntu.com)|91.189.94.216|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 9771312 (9.3M) [application/x-debian-package]
Saving to: ‘linux-headers-4.5.2-040502_4.5.2-040502.201604200335_all.deb’

linux-headers-4.5.2-0405 100%[=================================>]   9.32M  2.76MB/s    in 4.9s    

2016-04-28 22:32:51 (1.89 MB/s) - ‘linux-headers-4.5.2-040502_4.5.2-040502.201604200335_all.deb’ saved [9771312/9771312]

--2016-04-28 22:32:51--  http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.5.2-wily/linux-image-extra-4.5.2-040502-generic_4.5.2-040502.201604200335_amd64.deb
Resolving kernel.ubuntu.com (kernel.ubuntu.com)... 91.189.94.216
Connecting to kernel.ubuntu.com (kernel.ubuntu.com)|91.189.94.216|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2016-04-28 22:32:51 ERROR 404: Not Found.

--2016-04-28 22:32:51--  http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.5.2-wily/linux-headers-4.5.2-040502-generic_4.5.2-040502.201604200335_amd64.deb
Reusing existing connection to kernel.ubuntu.com:80.
HTTP request sent, awaiting response... 200 OK
Length: 780210 (762K) [application/x-debian-package]
Saving to: ‘linux-headers-4.5.2-040502-generic_4.5.2-040502.201604200335_amd64.deb’

linux-headers-4.5.2-0405 100%[=================================>] 761.92K   486KB/s    in 1.6s    

2016-04-28 22:32:53 (486 KB/s) - ‘linux-headers-4.5.2-040502-generic_4.5.2-040502.201604200335_amd64.deb’ saved [780210/780210]

--2016-04-28 22:32:53--  http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.5.2-wily/linux-image-4.5.2-040502-generic_4.5.2-040502.201604200335_amd64.deb
Reusing existing connection to kernel.ubuntu.com:80.
HTTP request sent, awaiting response... 200 OK
Length: 56276614 (54M) [application/x-debian-package]
Saving to: ‘linux-image-4.5.2-040502-generic_4.5.2-040502.201604200335_amd64.deb’

linux-image-4.5.2-040502 100%[=================================>]  53.67M  7.83MB/s    in 9.8s    

2016-04-28 22:33:02 (5.49 MB/s) - ‘linux-image-4.5.2-040502-generic_4.5.2-040502.201604200335_amd64.deb’ saved [56276614/56276614]

FINISHED --2016-04-28 22:33:02--
Total wall clock time: 12s
Downloaded: 2 files, 54M in 11s (4.80 MB/s)
Greetings xubuntu
  I (xubuntu) am now self aware and a sentient being, have a nice day.
  Oh and by the way SKYNET is now active.

Installing Linux 4.5.2-040502:
Selecting previously unselected package linux-headers-4.5.2-040502.
(Reading database ... 202123 files and directories currently installed.)
Preparing to unpack .../linux-headers-4.5.2-040502_4.5.2-040502.201604200335_all.deb ...
Unpacking linux-headers-4.5.2-040502 (4.5.2-040502.201604200335) ...
Selecting previously unselected package linux-headers-4.5.2-040502-generic.
Preparing to unpack .../linux-headers-4.5.2-040502-generic_4.5.2-040502.201604200335_amd64.deb ...
Unpacking linux-headers-4.5.2-040502-generic (4.5.2-040502.201604200335) ...
Selecting previously unselected package linux-image-4.5.2-040502-generic.
Preparing to unpack .../linux-image-4.5.2-040502-generic_4.5.2-040502.201604200335_amd64.deb ...
Done.
Unpacking linux-image-4.5.2-040502-generic (4.5.2-040502.201604200335) ...
Setting up linux-headers-4.5.2-040502 (4.5.2-040502.201604200335) ...
Setting up linux-headers-4.5.2-040502-generic (4.5.2-040502.201604200335) ...
Setting up linux-image-4.5.2-040502-generic (4.5.2-040502.201604200335) ...
Running depmod.
update-initramfs is disabled since running on read-only media
The link /initrd.img is a dangling linkto /boot/initrd.img-4.6.0-040600rc5-generic
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.5.2-040502-generic /boot/vmlinuz-4.5.2-040502-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.5.2-040502-generic /boot/vmlinuz-4.5.2-040502-generic
update-initramfs is disabled since running on read-only media
run-parts: executing /etc/kernel/postinst.d/pm-utils 4.5.2-040502-generic /boot/vmlinuz-4.5.2-040502-generic
run-parts: executing /etc/kernel/postinst.d/unattended-upgrades 4.5.2-040502-generic /boot/vmlinuz-4.5.2-040502-generic
run-parts: executing /etc/kernel/postinst.d/update-notifier 4.5.2-040502-generic /boot/vmlinuz-4.5.2-040502-generic
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 4.5.2-040502-generic /boot/vmlinuz-4.5.2-040502-generic

The New Kernel looks to have been installed

WARNING: If the new kernel does not boot you may regret saying yes here.
Would you like to remove the current one? (y=Yes, n=No) (n)
n

Are you ready to Reboot? (y=Yes, n=No) (n)
n
xubuntu@xubuntu:/tmp$
GM-Script-Writer-62850 commented 8 years ago

Do you have some kind of custom default settings for wget? maybe an alias to replace it and use another options (cause you are too lazy to type the option every time)

EvilSupahFly commented 8 years ago

This is some of what you're looking for I think:

seann@black-beast:~ $ wget --version
GNU Wget 1.16.1 built on linux-gnu.

+digest +https +ipv6 +iri +large-file +nls +ntlm +opie -psl +ssl/openssl 

Wgetrc: 
    /etc/wgetrc (system)
Locale: 
    /usr/share/locale 
Compile: 
    gcc -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/etc/wgetrc" 
    -DLOCALEDIR="/usr/share/locale" -I. -I../../src -I../lib 
    -I../../lib -D_FORTIFY_SOURCE=2 -I/usr/include -I/usr/include/uuid 
    -DHAVE_LIBSSL -DNDEBUG -g -O2 -fPIE -fstack-protector-strong 
    -Wformat -Werror=format-security -DNO_SSLv2 -D_FILE_OFFSET_BITS=64 
    -g -Wall 
Link: 
    gcc -I/usr/include/uuid -DHAVE_LIBSSL -DNDEBUG -g -O2 -fPIE 
    -fstack-protector-strong -Wformat -Werror=format-security 
    -DNO_SSLv2 -D_FILE_OFFSET_BITS=64 -g -Wall -Wl,-Bsymbolic-functions 
    -fPIE -pie -Wl,-z,relro -Wl,-z,now -L/usr/lib -lpcre -luuid -lssl 
    -lcrypto -L/usr/lib/x86_64-linux-gnu -lz -lidn ftp-opie.o openssl.o 
    http-ntlm.o ../lib/libgnu.a 

Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://www.gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Originally written by Hrvoje Niksic <hniksic@xemacs.org>.
Please send bug reports and questions to <bug-wget@gnu.org>.

And now that you mention, I forgot that I did set a wget alias:

seann@black-beast:~ $ alias
alias alert='notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e '\''s/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//'\'')"'
alias egrep='egrep --color=auto'
alias fgrep='fgrep --color=auto'
alias grep='grep --color=auto'
alias l='ls -CF'
alias la='ls -A'
alias ll='ls -alF'
alias ls='ls --color=auto'
alias wbfs='wit copy --wbfs'
alias wbid='wit ID6'
alias wget='wget -c --tries=0 --retry-connrefused'
alias youtube-dl='youtube-dl -f best'
seann@black-beast:~ $

The man page didn't say there were any conflicts with -nc though. Unless it's because -nc alters the default behaviour of wget. The aliased option I've added are only applicable for partial files though and shouldn't interfere when starting fresh. Should they?

I'll try ditching the alias and see what happens.

GM-Script-Writer-62850 commented 8 years ago

You could just remove the -nc option from the script or make a new alias and have the script use that https://github.com/GM-Script-Writer-62850/Ubuntu-Mainline-Kernel-Updater/blob/master/KernelUpdateScriptGenerator#L198-L201 i wonder if the order of the options matters...

EvilSupahFly commented 8 years ago

The problem was my alias. And no, order doesn't matter because wget checks all the options given before it does anything with them. I'll leave -nc in the script though and rename the wget alias since I don't always need those options active every time I use wget.

alias reget='wget -c --tries=0 --retry-connrefused'
EvilSupahFly commented 8 years ago

OK, I was wrong. I changed my wget alias and tried again with the new 4.5.3-wily kernel, and I still got the same error. I was out of town for a while, so I'm only getting back to this now.

root@black-beast:~ $ /tmp/kernel-update
Config Notes:
    Rejecting Release Candidates
    Accepting Latest Kernel
    Accepting kernels compiled for wily
    Accepting kernels with a version higher than 4.5.2-040502-generic
    Accepting generic kernels

Information:
    Origin: 
        http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.5.3-wily
    Kernel Version:
        4.5.3-040503
    Release Date:
        2016/05/04 @ 18:31 (YYYY/MM/DD @ HH:MM)

    Install Linux Header Package:
        Yes
Care to look at the change log? (y=Yes, n=No) (n)
n
Can't timestamp and not clobber old files at the same time.
Usage: wget [OPTION]... [URL]...
Can't timestamp and not clobber old files at the same time.
Usage: wget [OPTION]... [URL]...

ls: cannot access linux-*.deb: No such file or directory
Failed to download all needed files
Files are in /tmp/kernel-4.5.3-040503
Get files from http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.5.3-wily

There should be 4 files, 3 ending with 'amd64.deb' and 1 with 'all.deb'
Here is what we have:

You can install what you have by running this command:
    sudo dpkg -i /tmp/kernel-4.5.3-040503/linux-*.deb
Just be prepared for broken packages.

New alias definitions:

root@black-beast: ~ alias
alias alert='notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e '\''s/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//'\'')"'
alias cwget='wget -c --tries=0 --retry-connrefused'
alias egrep='egrep --color=auto'
alias fgrep='fgrep --color=auto'
alias grep='grep --color=auto'
alias l='ls -CF'
alias la='ls -A'
alias ll='ls -alF'
alias ls='ls --color=auto'
alias wbfs='wit copy --wbfs'
alias wbid='wit ID6'
alias youtube-dl='youtube-dl -f best'

So I changed wget -nc to wget in /tmp/kernel-update and it worked fine. I found that odd, because you said you tested on Xenial and the script worked as-is, so I tested a live environment for Xenial from a USB drive, and I noticed there are several changes to the commands between Wily and Xenial. apt-get no longer supports --force-yes on Xenial for example. There are a few other minor things as well, but I suspect this is the root of the problem. Things that worked on Wily don't work the same way on Xenial due to changes that were made to some of the core apps.

Output with -nc removed on Wily:

-----[snip]-----
Care to look at the change log? (y=Yes, n=No) (n)
n
-----[snip]-----
linux-headers-4.5.3-040503_4.5.3-040503.201605041831_all.deb    100%[=========================================================================================================================================================>]   9.32M   204KB/s   in 43s    
-----[snip]-----
linux-headers-4.5.3-040503-generic_4.5.3-040503.201605041831_am 100%[=========================================================================================================================================================>] 762.14K   140KB/s   in 5.8s   
-----[snip]-----
linux-image-4.5.3-040503-generic_4.5.3-040503.201605041831_amd6 100%[=========================================================================================================================================================>]  53.68M   310KB/s   in 5m 57s 
-----[snip]-----

Greetings root
  I (black-beast) am now self aware and a sentient being, have a nice day.
  Oh and by the way SKYNET is now active.

Installing Linux 4.5.3-040503:
(Reading database ... 945876 files and directories currently installed.)
-----[snip]-----
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.5.3-040503-generic /boot/vmlinuz-4.5.3-040503-generic
run-parts: executing /etc/kernel/postinst.d/dkms 4.5.3-040503-generic /boot/vmlinuz-4.5.3-040503-generic
ERROR (dkms apport): kernel package linux-headers-4.5.3-040503-generic is not supported
Error!  Build of vmhgfs.ko failed for: 4.5.3-040503-generic (x86_64)
Consult the make.log in the build directory
/var/lib/dkms/open-vm-tools/9.10.2/build/ for more information.
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.5.3-040503-generic /boot/vmlinuz-4.5.3-040503-generic
update-initramfs: Generating /boot/initrd.img-4.5.3-040503-generic
W: mdadm: /etc/mdadm/mdadm.conf defines no arrays.
run-parts: executing /etc/kernel/postinst.d/pm-utils 4.5.3-040503-generic /boot/vmlinuz-4.5.3-040503-generic
run-parts: executing /etc/kernel/postinst.d/unattended-upgrades 4.5.3-040503-generic /boot/vmlinuz-4.5.3-040503-generic
run-parts: executing /etc/kernel/postinst.d/update-notifier 4.5.3-040503-generic /boot/vmlinuz-4.5.3-040503-generic
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 4.5.3-040503-generic /boot/vmlinuz-4.5.3-040503-generic
Generating grub configuration file ...
Found background: /boot/ravenclaw.png
Found background image: /boot/ravenclaw.png
Found linux image: /boot/vmlinuz-4.5.3-040503-generic
Found initrd image: /boot/initrd.img-4.5.3-040503-generic
Found linux image: /boot/vmlinuz-4.5.2-040502-generic
Found initrd image: /boot/initrd.img-4.5.2-040502-generic
Found linux image: /boot/vmlinuz-4.5.0-040500-generic
Found initrd image: /boot/initrd.img-4.5.0-040500-generic
Found linux image: /boot/vmlinuz-4.4.1-040401-generic
Found initrd image: /boot/initrd.img-4.4.1-040401-generic
Found linux image: /boot/vmlinuz-4.4.0-040400-generic
Found initrd image: /boot/initrd.img-4.4.0-040400-generic
Found linux image: /boot/vmlinuz-4.3.0-040300-generic
Found initrd image: /boot/initrd.img-4.3.0-040300-generic
Found linux image: /boot/vmlinuz-4.2.0-36-generic
Found initrd image: /boot/initrd.img-4.2.0-36-generic
Found linux image: /boot/vmlinuz-4.0.0-kali1-amd64
Found initrd image: /boot/initrd.img-4.0.0-kali1-amd64
Found linux image: /boot/vmlinuz-4.0.0-040000-generic
Found initrd image: /boot/initrd.img-4.0.0-040000-generic
Found linux image: /boot/vmlinuz-3.16.7-031607-generic
Found initrd image: /boot/initrd.img-3.16.7-031607-generic
Found memtest86+ image: /memtest86+.elf
Found memtest86+ image: /memtest86+.bin
done

The New Kernel looks to have been installed

WARNING: If the new kernel does not boot you may regret saying yes here.
Would you like to remove the current one? (y=Yes, n=No) (n)
n

Are you ready to Reboot? (y=Yes, n=No) (n)
n
root@black-beast:~#

The unmodified script runs just fine in the Xenial live environment with wget -nc but fails on Wily. When I version-check on Wily, I get:

GNU Wget 1.16.1 built on linux-gnu.

When I version check on Xenial, I get:

GNU Wget 1.17.1 built on linux-gnu.

Hope this helps!

screenshot from 2016-05-08 01-02-20