Describe the bug
When attempting to mirror Debian repositories, the Building file list step fails with the following error message:
Traceback (most recent call last):
File "/usr/local/bin/refrapt", line 8, in <module>
sys.exit(main())
File "/var/spool/apt-mirror/.local/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/var/spool/apt-mirror/.local/lib/python3.9/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/var/spool/apt-mirror/.local/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/var/spool/apt-mirror/.local/lib/python3.9/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/var/spool/apt-mirror/.local/lib/python3.9/site-packages/refrapt/refrapt.py", line 166, in main
filesToDownload += ProcessIndex(key, value)
File "/var/spool/apt-mirror/.local/lib/python3.9/site-packages/refrapt/refrapt.py", line 470, in ProcessIndex
if filename.startswith("./"):
UnboundLocalError: local variable 'filename' referenced before assignment
I think this is just a case of the variable name being typo'd? I was able to fix it locally by changing lines 470-471 of refrapt.py to correct the case of the variable name like so:
if fileName.startswith("./"):
fileName = fileName[2:]
Configuration File and Logs
Configuration File
```shell
########################################
## Example Refrapt configuration file ##
########################################
# Inline comments are supported
##############
## SETTINGS ##
##############
# Each of the settings here are based on the default settings in Refrapt. If the values are
# not specified in this file, the default in Refrapt will be used instead.
# This is the default Architecture to be used if one is not specified in for a source.
set architecture = amd64
# For each of the path variables, if not specified, will be found in ~/refrapt
# The root path for log files and the configuration file.
# set rootPath = ""
# The directory where the mirror will be stored. This will be the larget directory, so use a large drive!
set mirrorPath = "/cachetank/apt/mirror"
# Directory used for intermediate Index files. Recommend using an SSD for best performance during the building of file lists.
set skelPath = "/cachetank/apt/skel"
# Directory used for storing log files and lock files. Recommend using an SSD for best performance.
set varPath = "/cachetank/apt/var"
# Specify whether the Contents-[arch].* files should be downloaded.
set contents = True
# The number of threads to use for download and decompression.
# If not specified here, Refrapt will get the number available from the machine.
set threads = 8
# Specify whether to add the --auth-no-challenge parameter to Wget.
set authNoChallenge = False
# Specify whether to add the --no-check-certificate parameter to Wget.
set noCheckCertificate = False
# Specify whether to add the --unlink parameter to Wget.
set unlink = False
# Specify whether you are using a proxy for Wget.
set useProxy = False
# The HTTP proxy to use. useProxy setting must be True.
set httpProxy = ""
# The HTTPS proxy to use. useProxy setting must be True.
set httpsProxy = ""
# The name of the proxy user. useProxy setting must be True.
set proxyUser = ""
# The password of the proxy user. useProxy setting must be True.
set proxyPass = ""
# The SSL certificate to pass to Wget.
set certificate = ""
# The SSL certificate to pass to Wget.
set caCertificate = ""
# The SSL key to pass to Wget.
set privateKey = ""
# Set the --limit-rate value for each Wget instance.
# Note that this will be multipled by the number of threads specified.
# See Wget manpage for correct syntax
set limitRate = 500m # 5 MB
# Limits the download of Translation files for that of your locale. When not specified, Refrapt will default to locale.getdefaultlocale().
# Note that "_XX" languages will be stripped of the locale to collect a wider number of files. Ie "en_GB" becomes "en", which will capture
# files with the same base language, but different locales.
# set language = "en_GB"
# Tell Refrapt to update all files, regardless of whether they are deemed to have changed or not.
# Note that if Wget determines the files are unchanged (via Timestamping), this will have no effect.
# The use of this should be limited for when you know a package has changed, but the size has not
# (which is how Refrapt determines whether the file needs updating)
set forceUpdate = False
# The log level to use for the application. Possible values are:
# DEBUG
# INFO
# WARNING
# ERROR
# CRITICAL
set logLevel = INFO
# Will cause Refrapt to only process the Index files, and determine the list of Binary / Sources files to download,
# and their size, but will not perform the main download. Useful for determining a potential download size before
# committing to it.
set test = False
# Specify whether Refrapt should download the /by-hash/* directories and contents for each available checksum.
set byHash = True
#################
## SOURCE LIST ##
#################
# Define each of the sources which you wish to mirror.
# Multiple architectures can be specified per mirror, as well as multiple components
# Debian Bullseye
deb http://deb.debian.org/debian bullseye main contrib non-free
deb-src http://deb.debian.org/debian bullseye main contrib non-free
deb http://deb.debian.org/debian-security/ bullseye-security main contrib non-free
deb-src http://deb.debian.org/debian-security/ bullseye-security main contrib non-free
deb http://deb.debian.org/debian bullseye-updates main contrib non-free
deb-src http://deb.debian.org/debian bullseye-updates main contrib non-free
# Debian Buster
deb http://deb.debian.org/debian buster main contrib non-free
deb-src http://deb.debian.org/debian buster main contrib non-free
deb http://deb.debian.org/debian buster-backports main contrib non-free
deb-src http://deb.debian.org/debian buster-backports main contrib non-free
deb http://security.debian.org/debian-security buster/updates main
deb-src http://security.debian.org/debian-security buster/updates main
# Debian Stretch
deb http://deb.debian.org/debian stretch main contrib non-free
deb-src http://deb.debian.org/debian stretch main contrib non-free
deb http://security.debian.org/debian-security stretch/updates main
deb-src http://security.debian.org/debian-security stretch/updates main
###################
## DISABLE CLEAN ##
###################
# Cleaning of unrequired files is enabled by default. To prevent cleaning
# a particular directory, add the following command
# clean=False deb http://gb.archive.ubuntu.com/ubuntu
```
System Information:
OS: Debian Buster (10)
Refrapt Version: 0.3.1
Thank you for taking the time to make this awesome tool!
Describe the bug When attempting to mirror Debian repositories, the
Building file list
step fails with the following error message:I think this is just a case of the variable name being typo'd? I was able to fix it locally by changing lines 470-471 of
refrapt.py
to correct the case of the variable name like so:Configuration File and Logs
Configuration File
```shell ######################################## ## Example Refrapt configuration file ## ######################################## # Inline comments are supported ############## ## SETTINGS ## ############## # Each of the settings here are based on the default settings in Refrapt. If the values are # not specified in this file, the default in Refrapt will be used instead. # This is the default Architecture to be used if one is not specified in for a source. set architecture = amd64 # For each of the path variables, if not specified, will be found in ~/refrapt # The root path for log files and the configuration file. # set rootPath = "" # The directory where the mirror will be stored. This will be the larget directory, so use a large drive! set mirrorPath = "/cachetank/apt/mirror" # Directory used for intermediate Index files. Recommend using an SSD for best performance during the building of file lists. set skelPath = "/cachetank/apt/skel" # Directory used for storing log files and lock files. Recommend using an SSD for best performance. set varPath = "/cachetank/apt/var" # Specify whether the Contents-[arch].* files should be downloaded. set contents = True # The number of threads to use for download and decompression. # If not specified here, Refrapt will get the number available from the machine. set threads = 8 # Specify whether to add the --auth-no-challenge parameter to Wget. set authNoChallenge = False # Specify whether to add the --no-check-certificate parameter to Wget. set noCheckCertificate = False # Specify whether to add the --unlink parameter to Wget. set unlink = False # Specify whether you are using a proxy for Wget. set useProxy = False # The HTTP proxy to use. useProxy setting must be True. set httpProxy = "" # The HTTPS proxy to use. useProxy setting must be True. set httpsProxy = "" # The name of the proxy user. useProxy setting must be True. set proxyUser = "" # The password of the proxy user. useProxy setting must be True. set proxyPass = "" # The SSL certificate to pass to Wget. set certificate = "" # The SSL certificate to pass to Wget. set caCertificate = "" # The SSL key to pass to Wget. set privateKey = "" # Set the --limit-rate value for each Wget instance. # Note that this will be multipled by the number of threads specified. # See Wget manpage for correct syntax set limitRate = 500m # 5 MB # Limits the download of Translation files for that of your locale. When not specified, Refrapt will default to locale.getdefaultlocale(). # Note that "_XX" languages will be stripped of the locale to collect a wider number of files. Ie "en_GB" becomes "en", which will capture # files with the same base language, but different locales. # set language = "en_GB" # Tell Refrapt to update all files, regardless of whether they are deemed to have changed or not. # Note that if Wget determines the files are unchanged (via Timestamping), this will have no effect. # The use of this should be limited for when you know a package has changed, but the size has not # (which is how Refrapt determines whether the file needs updating) set forceUpdate = False # The log level to use for the application. Possible values are: # DEBUG # INFO # WARNING # ERROR # CRITICAL set logLevel = INFO # Will cause Refrapt to only process the Index files, and determine the list of Binary / Sources files to download, # and their size, but will not perform the main download. Useful for determining a potential download size before # committing to it. set test = False # Specify whether Refrapt should download the /by-hash/* directories and contents for each available checksum. set byHash = True ################# ## SOURCE LIST ## ################# # Define each of the sources which you wish to mirror. # Multiple architectures can be specified per mirror, as well as multiple components # Debian Bullseye deb http://deb.debian.org/debian bullseye main contrib non-free deb-src http://deb.debian.org/debian bullseye main contrib non-free deb http://deb.debian.org/debian-security/ bullseye-security main contrib non-free deb-src http://deb.debian.org/debian-security/ bullseye-security main contrib non-free deb http://deb.debian.org/debian bullseye-updates main contrib non-free deb-src http://deb.debian.org/debian bullseye-updates main contrib non-free # Debian Buster deb http://deb.debian.org/debian buster main contrib non-free deb-src http://deb.debian.org/debian buster main contrib non-free deb http://deb.debian.org/debian buster-backports main contrib non-free deb-src http://deb.debian.org/debian buster-backports main contrib non-free deb http://security.debian.org/debian-security buster/updates main deb-src http://security.debian.org/debian-security buster/updates main # Debian Stretch deb http://deb.debian.org/debian stretch main contrib non-free deb-src http://deb.debian.org/debian stretch main contrib non-free deb http://security.debian.org/debian-security stretch/updates main deb-src http://security.debian.org/debian-security stretch/updates main ################### ## DISABLE CLEAN ## ################### # Cleaning of unrequired files is enabled by default. To prevent cleaning # a particular directory, add the following command # clean=False deb http://gb.archive.ubuntu.com/ubuntu ```
System Information:
Thank you for taking the time to make this awesome tool!