Closed ilg closed 4 months ago
Hi @ilg, it looks like you might be hitting this issue in Python 3.8+. Due to a change in PEP 495, datetime.min
may be producing a value with year 0 because of your systems timezone offset.
You can see this same issue reproduced with this line of just the Python datetime
library:
time = datetime.fromtimestamp(-62135596800)
or
datetime.min.timestamp()
The problem is Python's moving this time out of "year 1" into "year 0" when subtracting the timezone offset and Python doesn't acknowledge a year 0 as being valid input. I'm not sure if we can realistically adjust this in our code base until it's fixed upstream. As a short term workaround, you could potentially add a timedelta
to datetime.min
>= 24 hours. This would look something like this:
IfModifiedSince=(datetime.min + timedelta(days=1))
While not ideal, it works around the current limitation in the standard library and shouldn't interfere with the desired behavior unless you have some very old files stored in S3.
@nateprewitt Thanks for the detailed response. The Python issue link in particular helped me get my head around what's going on.
I still don't understand why I'm not running into the issue on my local dev box, since datetime.min.timestamp()
throws the "year 0" error there, too, but it's clear this isn't a boto3 issue.
Adding a day is basically the workaround I'd put into place, but from the Python issue, it seems like datetime.min.replace(tzinfo=timezone.utc)
might be a better fix.
This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.
Describe the bug
Similar to #3642, but using
get_object
on an S3 client instance and passingIfModifiedSince=datetime.min
. I'm running inside a docker container derived from python:3.11.9 with boto3==1.34.100.When I run this container locally, I do not get the runtime error.
When I run on the same container on AWS EB + EC2, I get the runtime error.
Both systems seem to have their local timezone set to UTC.
(It seems quite possible that this is ultimately a programmer error on my part for passing
datetime.min
, but it had been working on EB+EC2 previously—possibly broken by updating to a newer OS image—and works locally, so maybe there's some issue that could be addressed through boto3 or maybe this will just be a useful note for someone else encountering the same issue in the future.)Expected Behavior
I'd expect the behavior of boto3 to be the same on both platforms and ideally to not raise the
RuntimeError
.Current Behavior
Only when running the container on the EB+EC2 instance, not when running the container locally:
Reproduction Steps
I believe that making an S3 get_object call with
IfModifiedSince=datetime.min
(withfrom datetime import datetime
) should reproduce the issue, but only on some systems.Possible Solution
No response
Additional Information/Context
No response
SDK version used
1.34.100
Environment details (OS name and version, etc.)
Linux ip-10-0-1-32.us-west-2.compute.internal 4.14.301-224.520.amzn2.x86_64 #1 SMP Fri Dec 9 09:57:03 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux