Closed thefallenrat closed 7 years ago
i have also worked on func_b recently and added support for pacui's default cache directory for AUR files. the core logic seems to work, but there is something wrong with the output format (using files or variables, lists separated by newlines or spaces) and i am not able to install/remove packages. i still have to investigate this little problem.
i will probably be really busy with real life for the next 2 weeks. when i have some time, i will take a look at this PR and merge your code manually with mine. but when you are in a hurry, you can also manually merge my changes into your code. here is my current func_b, just in case:
function func_b
{
# declare local variables
local {cache,cachePACAUR,pkgR,pkgI,pkgD,line,temp1,temp2,temp3}
# get pacman's cache directory from file /etc/pacman.conf (without any white spaces) and write results to variable "cache":
cache=$( awk -F '=' '/^CacheDir/ {gsub(" ","",$2); print $2}' '/etc/pacman.conf' )
# set "cache" varialbe to default directory, if variable "cache" is empty:
if [[ -z $cache ]] # if "cache" variable is empty
then
cache="/var/cache/pacman/pkg/"
fi
if [[ -e /usr/bin/pacaur ]] # checks, whether file "pacaur" exists, i.e. pacaur is installed
then
# the cache location of pacaur is important for downgrading packages installed from the AUR:
if [[ -z $AURDEST ]] # $AURDEST is environment variable for changing pacaur's default cache directory. check, if "AURDEST" variable is empty
then
cachePACAUR="$HOME/.cache/pacaur/"
else
cachePACAUR="$AURDEST"
fi
fi
# 1. get list of last installs/upgrades/removes from pacman log and display result in fzf. in the end, fzf saves all selected package names (including the words "installed/upgraded/removed/downgraded" to file /tmp/pacui-cache-packages
tail -3000 /var/log/pacman.log | grep "\[ALPM\]" | grep "installed\|removed\|upgraded" | sed 's/\[ALPM\] //g' | fzf-tmux -m -e +s -i -1 --query="$input2" --cycle --tac --reverse --margin=4%,1%,1%,2% --inline-info --header="TAB key to (un)select. ENTER key to roll back. ESC or CTRL+C to quit." --prompt='Enter string to filter displayed list of recent Pacman changes > ' | sed 's/ ([^)]*)//g' | awk '{print $3 " " $4}' > /tmp/pacui-cache-packages
# only run the command inside the if-statement, if file /tmp/pacui-cache-packages is not empty and exists - this happens when fzf is quit with ESC or CTRL+C
if [[ -s /tmp/pacui-cache-packages ]]
then
# 2. in case of conflicting packages, packages have to be first removed (with the force option, because other packages might still depend on them).
#filter /tmp/pacui-cache-packages file for the word "installed" and write package names to variable "pkg"
pkgR=$( awk '/installed/ {print $2}' '/tmp/pacui-cache-packages' )
if [[ -n $pkgR ]] # this if-condition avoids error message when no package gets removed (and $pkgR is empty)
then
sudo pacman -Rdd "$pkgR" --color always --noconfirm
fi
# 3. in case an "upgraded" package needs a package as dependency, the "removed" packages have to be installed.
#filter /tmp/pacui-cache-packages file for the word "removed" and write package names to file /tmp/pacui-cache-install
awk '/removed/ {print $2}' '/tmp/pacui-cache-packages' > /tmp/pacui-cache-install
if [[ -s /tmp/pacui-cache-install ]] # this if-condition avoids error messages when no package gets installed (and /tmp/pacui-cache-install is empty)
then
# the file /tmp/pacui-install will later contain a list of file names of (old versions of) packages in cache. it has to be empty in the beginning!
[[ -e /tmp/pacui-install ]] && rm /tmp/pacui-install # check, whether file exists and delete it.
# read line by line from file /tmp/pacui-cache-install in while loop and save that line to variable $line
while IFS='' read -r line || [[ -n "$line" ]]
do
# write name of latest version in pacman cache into file /tmp/pacui-install ("sort" puts latest version on top, which is then selected):
find "$cache" -name "${line}-[0-9]*[0-9a-z.-_]*.pkg.tar.[gx]z" | sort -r | sed -n '1p' >> /tmp/pacui-install
if [[ -e /usr/bin/pacaur ]] # checks, whether file "pacaur" exists, i.e. pacaur is installed
then
# do the same as above for files from pacaur's cache directory.
# the problem here is that AUR packages are not named/numbered in a constant and easy sortable way. therefore, we search for all files and output their modification date in an easy searchable format (and then, the file name).
# then, "grep" is used to get only package files. then, the list is sorted (by the modification date).
# awk gets rid of the modification date. grep filters for the file name $line. sed only chooses the first/top line.
find "$cachePACAUR" -maxdepth 2 -mindepth 2 -type f -printf "%T+\t%p\n" | grep ".pkg.tar.[gx]z$" | sort -rn | awk '{print $2}' | grep "$line""-" | sed -n '1p' >> /tmp/pacui-temp-downgrade
fi
done < "/tmp/pacui-cache-install"
# write file into variable "pkgI" (otherwise, the "sudo pacman -U" command does not work) and unique sort /tmp/pacui-install (=remove duplicate lines without ordering the output)
pkgI=$( sort -u /tmp/pacui-install )
# install cannot be done as dependency, because sometimes packages are simply replaced by other packages. in this case, installing as dependency would be bad!
sudo pacman -U "$pkgI" --color always --noconfirm
fi
# 4. filter /tmp/pacui-cache-packages file for the word "upgraded" and write package names to file /tmp/pacui-cache-downgrade
# /tmp/pacui-cache-downgrade contains list of package names to be downgraded!
awk '/upgraded/ {print $2}' '/tmp/pacui-cache-packages' > /tmp/pacui-cache-downgrade
if [[ -s /tmp/pacui-cache-downgrade ]] # this if-condition avoids error messages when no package gets downgraded (and /tmp/pacui-cache-downgrade is empty)
then
# first, count the number of times the package name appears in file /tmp/pacui-cache-downgrade:
sort /tmp/pacui-cache-downgrade | uniq -c > /tmp/pacui-cache-downgrade-counted
# first argument in /tmp/pacui-cache-downgrade-counted is the number of times the package name appears and the second is the package name.
# the file /tmp/pacui-downgrade will later contain a list of file names of (old versions of) packages in cache. it has to be empty in the beginning!
[[ -e /tmp/pacui-downgrade ]] && rm /tmp/pacui-downgrade # check, whether file exists and delete it.
# read line by line from file /tmp/pacui-cache-downgrade-counted in while loop and save that line to variable $line
while read -r line && [[ -n "$line" ]]
do
# attention, the following variables can be empty:
temp1=$( echo "$line" | awk '{print $1}' ) # this variable is the no. of times a package has to be downgraded
temp2=$( echo "$line" | awk '{print $2}' ) # this variable is the package name to be downgraded
if [[ -n $temp2 ]] # checks, if variable is not empty
then
# write list with all versions of package in cache into file /tmp/pacui-temp-downgrade (sorted - newest package version is on top).
# this does not work for AUR packages installed by pacaur!
find "$cache" -name "${temp2}-[0-9]*[0-9a-z.-_]*.pkg.tar.[gx]z" | sort -r >> /tmp/pacui-temp-downgrade
if [[ -e /usr/bin/pacaur ]] # checks, whether file "pacaur" exists, i.e. pacaur is installed
then
# do the same as above for files from pacaur's cache directory.
# the problem here is that AUR packages are not named/numbered in a constant and easy sortable way. therefore, we search for all files and output their modification date in an easy searchable format (and then, the file name).
# then, "grep" is used to get only package files. then, the list is sorted (by the modification date).
# awk gets rid of the modification date. grep filters for the file name $temp2.
find "$cachePACAUR" -maxdepth 2 -mindepth 2 -type f -printf "%T+\t%p\n" | grep ".pkg.tar.[gx]z$" | sort -rn | awk '{print $2}' | grep "$temp2""-" >> /tmp/pacui-temp-downgrade
fi
# temp3 is supposed to be "2p" when temp1=1 and "3p" when temp1=2 ... --> needed for "sed" command below
temp3="$(( temp1 + 1 ))p"
# the next line moves the $((temp3-1))-th version below the currently installed package version to file /tmp/pacui-downgrade. if no such old version is available, nothing happens.
# this command determines the currently installed verions of package $temp2: pacman -Q "$temp2" | awk '{print $2}'
grep "$( pacman -Q "$temp2" | awk '{print $2}' )" -A 100 "/tmp/pacui-temp-downgrade" | sed -n "$temp3" >> /tmp/pacui-downgrade
fi
done < "/tmp/pacui-cache-downgrade-counted"
# write file into variable (otherwise, the "sudo pacman -U" command does not work)
pkgD=$( cat /tmp/pacui-downgrade )
# the following if-statement prevents the following error, in case there is no older package version available: "error: no targets specified (use -h for help)"
if [[ -n $pkgD ]] # checks, if variable is not empty
then
# downgrade packages by manually installing them: (sudo pacman -U --noconfirm --color always )
sudo pacman -U "$pkgD" --color always --noconfirm
fi
fi
fi
}
Alright the code are worked 100% even though there are still some temp files that are not yet replaced by variables ( Caused they're in the while and for loop) . If you want to test it just simply copy the func_b
into a new text, then replaces pacman commands :
sudo pacman -Rdd "$pkgR" --color always --noconfirm
into
echo "sudo pacman -Rdd $pkgR --color always --noconfirm"
Run it and then you willl see something like this :
~ >>> ./test
sudo pacman -Rdd aurvote cmake gedit gedit-plugins gspell jsoncpp pcmanfm sakura-gtk2 vim vte xfdesktop yaourt youtube-dl --color always --noconfirm
sudo pacman -U /home/thefallenrat/.pacmancache/gvim-8.0.0851-1-x86_64.pkg.tar.xz /home/thefallenrat/.pacmancache/notepadqq-1.0.1-1-x86_64.pkg.tar.xz /home/thefallenrat/.pacmancache/sakura-gtk2-2.4.2-3-x86_64.pkg.tar.xz /home/thefallenrat/.pacmancache/vim-omnicppcomplete-0.4.1-8-any.pkg.tar.xz /home/thefallenrat/.pacmancache/xed-1.4.6-1-x86_64.pkg.tar.xz --color always --noconfirm
sudo pacman -U /home/thefallenrat/.pacmancache/gdk-pixbuf2-2.36.8-1-x86_64.pkg.tar.xz /home/thefallenrat/.pacmancache/gtk3-3.22.18-1-x86_64.pkg.tar.xz /home/thefallenrat/.pacmancache/gtk-update-icon-cache-3.22.18-1-x86_64.pkg.tar.xz /home/thefallenrat/.pacmancache/linux412-4.12.6-1-x86_64.pkg.tar.xz /home/thefallenrat/.pacmancache/package-query-1.8-1-x86_64.pkg.tar.xz /home/thefallenrat/.pacmancache/wayland-1.13.0-1-x86_64.pkg.tar.xz --color always --noconfirm
Oops accidentally deleted branch and closed the pull request. Sorry about that! Reopening it ...
Also I just wanna say that, func_b
improvement has been 100% completed with full use of variables and mktemp
command for the pacui-tmp-downgrade
that can't be replaced with variable. The code works as expected on my end and you can merge this anytime you want!
i have read your changes and there are some things i have noticed:
is the syntax of trap "rm -rf $pacui_tmp_downgrade" EXIT
right? i have not worked with the trap command before, but this looks to me as you want to remove the output of pacui_tmp_downgrade file and not the file itself. am i wrong about this?
you have commented out the following command (line 605) #find "$cachePACAUR" -maxdepth 2 -mindepth 2 -type f -printf "%T+\t%p\n" | grep ".pkg.tar.[gx]z$" | sort -rn | awk '{print $2}' | grep "$line""-" | sed -n '1p' >> /tmp/pacui-temp-downgrade
(please adjust this command to use variables instead of files, before you start to use it). i would actually like to include AUR packages in this section as well. this would enable the user to "roll back" an AUR package, which he has removed. what do you think about that?
you use a couple of times a strange syntax (example from line 648): find $cache -name "${temp2}-[0-9]*[0-9a-z.-_]*.pkg.tar.[gx]z" | sort -r > ${pacui_tmp_downgrade}
. i think this looks ugly and i would rather prefer this code structure:
pacui_tmp_downgrade=$(find $cache -name "${temp2}-[0-9]*[0-9a-z.-_]*.pkg.tar.[gx]z" | sort -r )
is there a reason why you chose to use that syntax?
your other code looks fine.
Thanks for the reply!
is the syntax of trap "rm -rf $pacui_tmp_downgrade" EXIT right? i have not worked with the trap command before, but this looks to me as you want to remove the output of pacui_tmp_downgrade file and not the file itself. am i wrong about this?
Actually trap
is a conditional command (like if
) that will automatically run the code inside " "
if the process get closed abruptly via terminal close or CTRL+C. In this case, when it does gets closed abruptly, it will automatically delete ${pacui_tmp_downgrade} so there won't be file leftover...
https://stackoverflow.com/questions/687014/removing-created-temp-files-in-unexpected-bash-exit
you have commented out the following command (line 605) #find "$cachePACAUR" -maxdepth 2 -mindepth 2 -type f -printf "%T+\t%p\n" | grep ".pkg.tar.[gx]z$" | sort -rn | awk '{print $2}' | grep "$line""-" | sed -n '1p' >> /tmp/pacui-temp-downgrade (please adjust this command to use variables instead of files, before you start to use it). i would actually like to include AUR packages in this section as well. this would enable the user to "roll back" an AUR package, which he has removed. what do you think about that?
If I enabled it (with variable), it will messed up the first while loop causing only the first line that got out from the loop. But don't worry, I will think out another alternative solution for this...
you use a couple of times a strange syntax (example from line 648):
find $cache -name "${temp2}-[0-9]*[0-9a-z.-_]*.pkg.tar.[gx]z" | sort -r > ${pacui_tmp_downgrade}
. i think this looks ugly and i would rather prefer this code structure:pacui_tmp_downgrade=$(find $cache -name "${temp2}-[0-9]*[0-9a-z.-_]*.pkg.tar.[gx]z" | sort -r )
is there a reason why you chose to use that syntax?
This is important, as it will redirect the output into the file that are set by $pacui_tmp_downgrade}
.
So doing this :
> ${pacui_tmp_downgrade}
Is the same as :
> /tmp/pacui-tmp-downgrade.XXXXXXXX
# Where the repeated X s are random letters or numbers that are set by mktemp command
I just tested with the $()
. It'll make the loop to "run the directory" :
~/RAT's workspaces/pacui >>> ./pacui.bak
aurvote cmake docbook-xml docbook-xsl gedit gedit-code-assistance gedit-plugins git glib2-docs gnome-code-assistance gspell gtk-doc jsoncpp libieee1284 sakura-gtk2 sane sane-openrc source-highlight vim vte xfce4-dev-tools xfdashboard xfdesktop yaourt youtube-dl
/home/thefallenrat/.pacmancache/gvim-8.0.0851-1-x86_64.pkg.tar.xz /home/thefallenrat/.pacmancache/notepadqq-1.0.1-1-x86_64.pkg.tar.xz /home/thefallenrat/.pacmancache/pcmanfm-1.2.5-2-x86_64.pkg.tar.xz /home/thefallenrat/.pacmancache/ratpoison-1.4.9-1-x86_64.pkg.tar.xz /home/thefallenrat/.pacmancache/sakura-gtk2-2.4.2-3-x86_64.pkg.tar.xz /home/thefallenrat/.pacmancache/vim-omnicppcomplete-0.4.1-8-any.pkg.tar.xz /home/thefallenrat/.pacmancache/xed-1.4.6-1-x86_64.pkg.tar.xz
grep: /home/thefallenrat/.pacmancache/artools-base-0.5-2-any.pkg.tar.xz
/home/thefallenrat/.pacmancache/artools-base-0.5.2-1-any.pkg.tar.xz
/home/thefallenrat/.pacmancache/artools-base-0.5-1-any.pkg.tar.xz
/home/thefallenrat/.pacmancache/artools-base-0.5.1-1-any.pkg.tar.xz
/home/thefallenrat/.pacmancache/artools-base-0.4-1-any.pkg.tar.xz
/home/thefallenrat/.pacmancache/artools-base-0.4.1-1-any.pkg.tar.xz: No such file or directory
So doing the strange syntax > ${pacui_tmp_downgrade}
is important.
You can try out my code by just copy func_b
entirely and replace every sudo pacman
command with :
echo " "
echo "$pkgI" | paste -sd " "
i have just merged your PR. thanks a lot for your work!
one thing i noticed when i added some comments:
you use
rm ${pacui_tmp_downgrade}
i was always under the impression that you need root privileges to remove something from the /tmp/ directory. have you actually checked this or simply forgotten?
and one more style question regarding parenthesis: you use often ${variable_name} to get the output of a variable (sometimes it is quoted to prevent bugs from variable names containing non-standard characters). is there any difference to $(variable_name) or simply $variable_name?
Thanks for merging it! Appreciated it as always! :3
one thing i noticed when i added some comments: you use
rm ${pacui_tmp_downgrade}
i was always under the impression that you need root privileges to remove something from the /tmp/ directory. have you actually checked this or simply forgotten?
Yeah actually checked this. You don't need to have a privilege to be able to delete all files that resides in /tmp/
...
Take a look at this example :
~ >>> pacui_tmp_downgrade=$(mktemp /tmp/pacui_tmp_downgrade.XXXXXXXXXX)
~ >>> ls /tmp/
pacui_tmp_downgrade.rehT6ruYm0 pamac-checkdbs-thefallenrat ssh-C3Qw3R5Xw4ua
~ >>> rm ${pacui_tmp_downgrade}
~ >>> ls /tmp
pamac-checkdbs-thefallenrat ssh-C3Qw3R5Xw4ua
and one more style question regarding parenthesis: you use often ${variable_name} to get the output of a variable (sometimes it is quoted to prevent bugs from variable names containing non-standard characters). is there any difference to $(variable_name) or simply $variable_name?
Actually there is according to this, and I use it so that I can add some letter or in this case /n
after it :
~ >>> pacui_test=test
~ >>> echo "$(pacui_test)\nwhat"
zsh: command not found: pacui_test
what
~ >>> echo "${pacui_test}\nwhat"
test
what
Also I noticed that there are still some /tmp/file
that are not yet replaced by variables ( or mktemp ) in your other func
(such as func_r
). Can I help you with that?
thx for your explanations. i just fixed the roll-back option (you were missing a >>) and extended it.
but i am still not able to roll back multiple package updates. you can take a look at this again (it is already late here)? it looks like $pacui_downgrade is fine and contains the list of paths to the files i want to downgrade (separated by a newline). but somehow that list is not converted properly into a space separated list, which pacman can interpret. therefore, pacman gives me the following error (when i try to roll-back the update of "kwrite" and "ktexteditor"):
loading packages...
error: '/var/cache/pacman/pkg/kdoctools-5.36.0-1-x86_64.pkg.tar.xz /var/cache/pacman/pkg/ktexteditor-5.36.0-2-x86_64.pkg.tar.xz': could not find or read package
Also I noticed that there are still some /tmp/file that are not yet replaced by variables ( or mktemp ) in your other func (such as func_r ). Can I help you with that?
are you talking about this file: /tmp/pacui-packages-group
as far as i can see it, it is not security critical, because it is piped into fzf and then stored in a variable. only this variable is fed into pacman with sudo privileges.
am i wrong?
i have just updated my previous post with an error description. i hope this helps. good night.
it looks like $pacui_downgrade is fine and contains the list of paths to the files i want to downgrade (separated by a newline). but somehow that list is not converted properly into a space separated list, which pacman can interpret. therefore, pacman gives me the following error (when i try to roll-back the update of "kwrite" and "ktexteditor"):
Oops! I forgot to edit that part. You need to edit all pacman commands (in func_b
of course):
sudo pacman -Rdd "$pkgR" --color always --noconfirm
into :
sudo pacman -Rdd ${pkgR} --color always --noconfirm
This should work :
~/RAT's workspaces >>> ./pacui.bak
loading packages...
warning: downgrading package artools-base (0.5.4-1 => 0.5.3-1)
warning: downgrading package artools-pkg (0.5.4-1 => 0.5.3-1)
resolving dependencies...
looking for conflicting packages...
Package (2) Old Version New Version Net Change
artools-base 0.5.4-1 0.5.3-1 0.00 MiB
artools-pkg 0.5.4-1 0.5.3-1 0.00 MiB
Total Installed Size: 0.18 MiB
Net Upgrade Size: 0.00 MiB
:: Proceed with installation? [Y/n]
Just opened a pull request for this #15
are you talking about this file: /tmp/pacui-packages-group
as far as i can see it, it is not security critical, because it is piped into fzf and then stored in a variable. only this variable is fed into pacman with sudo privileges.
am i wrong?
But don't they get removed afterwards? What if they want to run func_r
again? Does the file gets replaced or will they gets added from the output?
But don't they get removed afterwards? What if they want to run func_r again? Does the file gets replaced or will they gets added from the output?
func_r is special, but take a look at the func_t, func_l, or func_d. there the file /tmp/pacui-packages-local is created and not removed afterwards. this file contains a list of all packages from the system repository and all locally installed packages and is only created when it does not exist yet. when it already exists, it is simply reused as is and not changed. it never gets recreated, when it already exists, just overwritten as seen by the > symbol in this code line:
pacman -Slq | tr -d " " > /tmp/pacui-packages-local
this is done because of performance reasons and should be noticeable by the user in func_i (where a list of all AUR pacakges gets downloaded).
now, that i think about it, /tmp/pacui-packages-local should be overwritten with the latest list of packages, because the user could have installed packages from the AUR since first using func_t, func_l, or func_d.
do you have other suggestions about the usage of a temporary file here? maybe, we should discuss this in a new issue?
one more comment: people using arch linux only have the AUR package to install pacui. therefore, i think we should stop doing development work (with breakages) in the master branch. i have created a devel branch for this kind of stuff. when we want people to test our stuff, we can move it over to master branch.
you can clone the development branch with (the single-branch argument is optional):
git clone -b devel --single-branch https://github.com/......../pacui.git
Thanks for replies and for merging this!
func_r is special, but take a look at the func_t, func_l, or func_d. there the file /tmp/pacui-packages-local is created and not removed afterwards. this file contains a list of all packages from the system repository and all locally installed packages and is only created when it does not exist yet. when it already exists, it is simply reused as is and not changed. it never gets recreated, when it already exists, just overwritten as seen by the > symbol in this code line:
pacman -Slq | tr -d " " > /tmp/pacui-packages-local
this is done because of performance reasons and should be noticeable by the user in func_i (where a list of all AUR pacakges gets downloaded).
now, that i think about it, /tmp/pacui-packages-local should be overwritten with the latest list of packages, because the user could have installed packages from the AUR since first using func_t, func_l, or func_d.
do you have other suggestions about the usage of a temporary file here? maybe, we should discuss this in a new issue?
Maybe we can do function-wide (?) $pacui_packages_local
that set up said file with mktemp
command, so that if it gets called again from the same or different func it can reuse it maintaining performance, and of course, removed it upon exit. But doing this requires user have to set up pacui-packages-local everytime they launch pacui....
And yeah I guess we should discuss this in a new issue
people using arch linux only have the AUR package to install pacui. therefore, i think we should stop doing development work (with breakages) in the master branch. i have created a devel branch for this kind of stuff. when we want people to test our stuff, we can move it over to master branch.
Oh cool! Should I have a future pull request, I will do this on devel branch!
Hey, I just made a pull request that modifies
func_b
to use variables (mostly sadly) to resolve #12But sadly I got a dead-end in the while loop. In the while loop, I can't make it to store all output into the variable, only the latest output that made it to the variable. So I decide to keep the temporary files until I figured out how to store variables in while loop
P.S Can I add my name in it?