search

zmanda / amanda

Amanda Network Backup
https://www.zmanda.com/downloads/
Other
214 stars 107 forks source link

Maximum DLEs #243

Open cscottpnw opened 7 months ago

cscottpnw commented 7 months ago

What limits the number of DLEs in a backup set?

I recently began having issues such as no ports available and bad file descriptor errors with a backup with hundreds of DLEs... which has worked up until now. Reducing the DLEs to ~50 mitigated the problem.

konidev20 commented 6 months ago

Hey @cscottpnw,

Please share with me the following information.

The OS, System Specifications and Version of Amanda Server

The OS, System Specification and Version of Amanda Client

What are you trying to do?

What is the expected behavior?

What is happening now?

Which version of the Amanda Server and Client was it working in ?

cscottpnw commented 6 months ago

Please don't treat this like a formal issue/complaint. I'm just curious about the fundamentals involved in limiting DLEs. e.g. file descriptors/ephemeral ports/etc. I couldn't find one.

Log attached

Details:

The OS, System Specifications and Version of Amanda Server:
FreeBSD 12.4 CPU: Intel Core i5-7600 Memory: 32 Amanda 3.5.1

The OS, System Specification and Version of Amanda Client Client and server are running on the same server for this backup set.

What are you trying to do? Backup image/video files

What is the expected behavior? Successful backup across 3 LTO4 tapes

What is happening now? Working properly now with a reduced set of DLEs. I restructured the directory/file structure to allow for fewer DLEs as a test. Sample of the error logs you saw would be helpful (bad file descriptor and port availability issues)

Which version of the Amanda Server and Client was it working in ? Unknow... somewhat recent though. This backup job is run roughly quarterly. The previous attempt was successful with many more DLEs.

konidev20 commented 6 months ago

Hey @cscottpnw,

I am learning about the fundamentals of how Amanda Backup works. I found your issue particularly interesting.

However, I have found something interesting in the documentation. Refer: amanda-auth.7

If there is no auth parameter specified in the amanda.conf(5) it defaults to bsd authentication.

Quoting from the above link:

bsd communication and authentication The authentication is done using .amandahosts file in the Amanda user's home directory. The protocol between Amanda server and client is UDP. The number of disk list entries (DLEs)--number of Amanda clients--is limited by the UDP packet size. This authentication protocol will use a different port for each data stream (see PORT USAGE below)

This is interesting, I want to check how the UDP packet size influences the max number of DLEs. [Edit] Not sure if it is applicable for Local Host communication [/Edit]

PS: The logs you attached are not available. I don't see any link.

cscottpnw commented 6 months ago

dumper.20231201064007002.debug.txt

Not sure what happened to the log file in the last comment. Here it is.

I think you may be on to something regarding BSD auth and packet count/size.

konidev20 commented 6 months ago

dumper.20231201064007002.debug.txt

Not sure what happened to the log file in the last comment. Here it is.

I think you may be on to something regarding BSD auth and packet count/size.

from the logs, looks like you're using bsdtcp authentication. Which would have multiplexed all the data streams to a single port. (There are 2 reserved ports used here)

I will work with my team to get to the end of this. This will help us add a section with respect to limitations if there are any.

[Edit] If you don't mind, can you share the amanda.conf and the xinetd service file as well. Please redact any personal information. [/Edit]