search

juftin / camply

camply, the campsite finder ⛺️ - a tool to find campsites at sold out campgrounds through sites like recreation.gov
https://juftin.com/camply/
MIT License
476 stars 93 forks source link

Add feature to ignore errors instead of stopping the tracking #263

Closed vassoz closed 1 year ago

vassoz commented 1 year ago

Describe the bug

When a provider (ReserveCalifornia in my case) encounters an error the script just exists and stop tracking. The errors are happening very often for ReserveCalifornia so the script become very unreliable.

So I switched to cron. Yet with every time the script encounter an error it sends a notification (which makes script not very useful). Tracking just one date for one campsite I encountered 18 errors over 12 hours run (every 10 minutes). I got 18 notifications over 12 hours which made the notifications more like a spam rather than a useful info.

Original Camply Command (with --debug)

docker run -it \
-e TELEGRAM_BOT_TOKEN=[...] -e TELEGRAM_CHAT_ID=[...] \
juftin/camply camply --provider ReserveCalifornia campsites \
--rec-area 726 --start-date 2023-06-10 --end-date 2023-06-11 \
--notifications telegram \
--search-once

Expected behavior Add 2 options: 1) fail the script after N errors 2) skip the notification on error

Console Output (with --debug)

2023-06-06 05:18:27,786 [    INFO]: Searching across 1 campgrounds
2023-06-06 05:18:27,786 [    INFO]:     ⛰  Sunset SB (#726) - 🏕  South Camp (sites 1-37) (#737)
2023-06-06 05:18:27,786 [    INFO]: Searching South Camp (sites 1-37), Sunset SB (737) for availability: June, 2023
2023-06-06 05:18:27,910 [    INFO]: Exception encountered, emitting notification last gasp.
2023-06-06 05:18:28,494 [  CAMPLY]: Exiting camply 👋
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /usr/local/lib/python3.9/site-packages/camply/providers/usedirect/usedirect.py:328 in            │
│ get_campsites_response                                                                           │
│                                                                                                  │
│   325 │   │   response.raise_for_status()                                                        │
│   326 │   │   response_json = response.json()                                                    │
│   327 │   │   try:                                                                               │
│ ❱ 328 │   │   │   return UseDirectAvailabilityResponse(**response.json())                        │
│   329 │   │   except ValidationError as e:                                                       │
│   330 │   │   │   error_message = (                                                              │
│   331 │   │   │   │   "Error Parsing UseDirect Availability Response "                           │
│                                                                                                  │
│ in pydantic.main.BaseModel.__init__:341                                                          │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValidationError: 10 validation errors for UseDirectAvailabilityResponse
Filters
  none is not an allowed value (type=type_error.none.not_allowed)
EndDate
  none is not an allowed value (type=type_error.none.not_allowed)
TimeZone
  none is not an allowed value (type=type_error.none.not_allowed)
TimeStamp
  none is not an allowed value (type=type_error.none.not_allowed)
UnitSort
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> Name
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> Description
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> Restrictions
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> SeasonDates
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> TrafficStatuses
  none is not an allowed value (type=type_error.none.not_allowed)

The above exception was the direct cause of the following exception:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /usr/local/lib/python3.9/site-packages/camply/search/base_search.py:574 in                       │
│ get_matching_campsites                                                                           │
│                                                                                                  │
│    571 │   │   """                                                                               │
│    572 │   │   if continuous is True or search_once is True:                                     │
│    573 │   │   │   try:                                                                          │
│ ❱  574 │   │   │   │   self._search_campsites_continuous(                                        │
│    575 │   │   │   │   │   log=log,                                                              │
│    576 │   │   │   │   │   verbose=verbose,                                                      │
│    577 │   │   │   │   │   polling_interval=polling_interval,                                    │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/search/base_search.py:508 in                       │
│ _search_campsites_continuous                                                                     │
│                                                                                                  │
│    505 │   │   continuous_search_attempts = 1                                                    │
│    506 │   │   while continuous_search is True:                                                  │
│    507 │   │   │   starting_count = len(self.campsites_found)                                    │
│ ❱  508 │   │   │   self._continuous_search_retry(                                                │
│    509 │   │   │   │   log=log,                                                                  │
│    510 │   │   │   │   verbose=verbose,                                                          │
│    511 │   │   │   │   polling_interval=polling_interval,                                        │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/search/base_search.py:354 in                       │
│ _continuous_search_retry                                                                         │
│                                                                                                  │
│    351 │   │   │   retry=tenacity.retry_if_exception_type(CampsiteNotFoundError),                │
│    352 │   │   │   wait=tenacity.wait.wait_fixed(int(polling_interval_minutes) * 60),            │
│    353 │   │   )                                                                                 │
│ ❱  354 │   │   matching_campsites = retryer.__call__(                                            │
│    355 │   │   │   fn=self._search_matching_campsites_available,                                 │
│    356 │   │   │   log=False,                                                                    │
│    357 │   │   │   verbose=False,                                                                │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/tenacity/__init__.py:379 in __call__                      │
│                                                                                                  │
│   376 │   │                                                                                      │
│   377 │   │   retry_state = RetryCallState(retry_object=self, fn=fn, args=args, kwargs=kwargs)   │
│   378 │   │   while True:                                                                        │
│ ❱ 379 │   │   │   do = self.iter(retry_state=retry_state)                                        │
│   380 │   │   │   if isinstance(do, DoAttempt):                                                  │
│   381 │   │   │   │   try:                                                                       │
│   382 │   │   │   │   │   result = fn(*args, **kwargs)                                           │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/tenacity/__init__.py:314 in iter                          │
│                                                                                                  │
│   311 │   │                                                                                      │
│   312 │   │   is_explicit_retry = fut.failed and isinstance(fut.exception(), TryAgain)           │
│   313 │   │   if not (is_explicit_retry or self.retry(retry_state)):                             │
│ ❱ 314 │   │   │   return fut.result()                                                            │
│   315 │   │                                                                                      │
│   316 │   │   if self.after is not None:                                                         │
│   317 │   │   │   self.after(retry_state)                                                        │
│                                                                                                  │
│ /usr/local/lib/python3.9/concurrent/futures/_base.py:439 in result                               │
│                                                                                                  │
│   436 │   │   │   │   if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]:                     │
│   437 │   │   │   │   │   raise CancelledError()                                                 │
│   438 │   │   │   │   elif self._state == FINISHED:                                              │
│ ❱ 439 │   │   │   │   │   return self.__get_result()                                             │
│   440 │   │   │   │                                                                              │
│   441 │   │   │   │   self._condition.wait(timeout)                                              │
│   442                                                                                            │
│                                                                                                  │
│ /usr/local/lib/python3.9/concurrent/futures/_base.py:391 in __get_result                         │
│                                                                                                  │
│   388 │   def __get_result(self):                                                                │
│   389 │   │   if self._exception:                                                                │
│   390 │   │   │   try:                                                                           │
│ ❱ 391 │   │   │   │   raise self._exception                                                      │
│   392 │   │   │   finally:                                                                       │
│   393 │   │   │   │   # Break a reference cycle with the exception in self._exception            │
│   394 │   │   │   │   self = None                                                                │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/tenacity/__init__.py:382 in __call__                      │
│                                                                                                  │
│   379 │   │   │   do = self.iter(retry_state=retry_state)                                        │
│   380 │   │   │   if isinstance(do, DoAttempt):                                                  │
│   381 │   │   │   │   try:                                                                       │
│ ❱ 382 │   │   │   │   │   result = fn(*args, **kwargs)                                           │
│   383 │   │   │   │   except BaseException:  # noqa: B902                                        │
│   384 │   │   │   │   │   retry_state.set_exception(sys.exc_info())  # type: ignore[arg-type]    │
│   385 │   │   │   │   else:                                                                      │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/search/base_search.py:246 in                       │
│ _search_matching_campsites_available                                                             │
│                                                                                                  │
│    243 │   │   List[AvailableCampsite]                                                           │
│    244 │   │   """                                                                               │
│    245 │   │   matching_campgrounds = []                                                         │
│ ❱  246 │   │   for camp in self.get_all_campsites():                                             │
│    247 │   │   │   if all(                                                                       │
│    248 │   │   │   │   [                                                                         │
│    249 │   │   │   │   │   self._compare_date_overlap(campsite=camp) is True,                    │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/search/search_usedirect.py:128 in                  │
│ get_all_campsites                                                                                │
│                                                                                                  │
│   125 │   │   │   │   │   f"{month.strftime('%B, %Y')}"                                          │
│   126 │   │   │   │   )                                                                          │
│   127 │   │   │   │   end_date = month + relativedelta(months=1) - timedelta(days=1)             │
│ ❱ 128 │   │   │   │   campsites = self.campsite_finder.get_campsites(                            │
│   129 │   │   │   │   │   campground_id=campground.facility_id,                                  │
│   130 │   │   │   │   │   start_date=month,                                                      │
│   131 │   │   │   │   │   end_date=end_date,                                                     │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/providers/usedirect/usedirect.py:385 in            │
│ get_campsites                                                                                    │
│                                                                                                  │
│   382 │   │   List[AvailableCampsite]                                                            │
│   383 │   │   """                                                                                │
│   384 │   │   self.refresh_metadata()                                                            │
│ ❱ 385 │   │   availability_response = self.get_campsites_response(                               │
│   386 │   │   │   campground_id=campground_id,                                                   │
│   387 │   │   │   start_date=start_date,                                                         │
│   388 │   │   │   end_date=end_date,                                                             │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/ratelimit/decorators.py:113 in wrapper                    │
│                                                                                                  │
│   110 │   │   '''                                                                                │
│   111 │   │   while True:                                                                        │
│   112 │   │   │   try:                                                                           │
│ ❱ 113 │   │   │   │   return func(*args, **kargs)                                                │
│   114 │   │   │   except RateLimitException as exception:                                        │
│   115 │   │   │   │   time.sleep(exception.period_remaining)                                     │
│   116 │   return wrapper                                                                         │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/ratelimit/decorators.py:80 in wrapper                     │
│                                                                                                  │
│    77 │   │   │   │   │   │   raise RateLimitException('too many calls', period_remaining)       │
│    78 │   │   │   │   │   return                                                                 │
│    79 │   │   │                                                                                  │
│ ❱  80 │   │   │   return func(*args, **kargs)                                                    │
│    81 │   │   return wrapper                                                                     │
│    82 │                                                                                          │
│    83 │   def __period_remaining(self):                                                          │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/providers/usedirect/usedirect.py:336 in            │
│ get_campsites_response                                                                           │
│                                                                                                  │
│   333 │   │   │   )                                                                              │
│   334 │   │   │   if "Message" in response_json:                                                 │
│   335 │   │   │   │   error_message += " " + response_json["Message"]                            │
│ ❱ 336 │   │   │   raise UseDirectError(error_message) from e                                     │
│   337 │                                                                                          │
│   338 │   def get_campsites(                                                                     │
│   339 │   │   self,                                                                              │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
UseDirectError: Error Parsing UseDirect Availability Response - Facility ID # 737. Invalid FacilityId specified for facility grid search on CALIRDR6

The above exception was the direct cause of the following exception:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /usr/local/bin/camply:8 in <module>                                                              │
│                                                                                                  │
│   5 from camply.cli import cli                                                                   │
│   6 if __name__ == '__main__':                                                                   │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                         │
│ ❱ 8 │   sys.exit(cli())                                                                          │
│   9                                                                                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/cli.py:834 in cli                                  │
│                                                                                                  │
│   831 │   Camply Command Line Utility Wrapper                                                    │
│   832 │   """                                                                                    │
│   833 │   try:                                                                                   │
│ ❱ 834 │   │   camply_command_line()                                                              │
│   835 │   except KeyboardInterrupt:                                                              │
│   836 │   │   logger.debug("Handling Exit Request")                                              │
│   837 │   finally:                                                                               │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1130 in __call__                            │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/rich_click/rich_group.py:21 in main                       │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1055 in main                                │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1657 in invoke                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1404 in invoke                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:760 in invoke                               │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/decorators.py:38 in new_func                        │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/cli.py:726 in campsites                            │
│                                                                                                  │
│   723 │   │   }                                                                                  │
│   724 │   provider_class: Type[BaseCampingSearch] = CAMPSITE_SEARCH_PROVIDER[provider]           │
│   725 │   camping_finder: BaseCampingSearch = provider_class(**provider_kwargs)                  │
│ ❱ 726 │   camping_finder.get_matching_campsites(**search_kwargs)                                 │
│   727                                                                                            │
│   728                                                                                            │
│   729 @camply_command_line.command(cls=RichCommand)                                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/search/base_search.py:584 in                       │
│ get_matching_campsites                                                                           │
│                                                                                                  │
│    581 │   │   │   │   │   search_once=search_once,                                              │
│    582 │   │   │   │   )                                                                         │
│    583 │   │   │   except Exception as e:                                                        │
│ ❱  584 │   │   │   │   self.notifier.last_gasp(error=e)                                          │
│    585 │   │   │   │   raise e                                                                   │
│    586 │   │   else:                                                                             │
│    587 │   │   │   starting_count = len(self.campsites_found)                                    │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/notifications/multi_provider_notifications.py:127  │
│ in last_gasp                                                                                     │
│                                                                                                  │
│   124 │   │   )                                                                                  │
│   125 │   │   for provider in self.providers:                                                    │
│   126 │   │   │   provider.send_message(error_message)                                           │
│ ❱ 127 │   │   raise RuntimeError(error_message) from error                                       │
│   128                                                                                            │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: camply encountered an error and exited 😟 [2023-06-06 05:18:27] - (UseDirectError) Error Parsing UseDirect Availability Response - Facility ID # 737. 
Invalid FacilityId specified for facility grid search on CALIRDR6

Another example:

2023-06-06 05:59:16,527 [  CAMPLY]: camply, the campsite finder ⛺️
2023-06-06 05:59:16,528 [    INFO]: Using Camply Provider: "ReserveCalifornia"
2023-06-06 05:59:16,533 [    INFO]: 1 booking nights selected for search, ranging from 2023-06-10 to 2023-06-10
2023-06-06 05:59:17,078 [    INFO]: Searching for campsites every 10 minutes. 
2023-06-06 05:59:17,078 [    INFO]: Notifications active via: <SilentNotifications>, <TelegramNotifications>
2023-06-06 05:59:17,078 [    INFO]: Searching across 3 campgrounds
2023-06-06 05:59:17,078 [    INFO]:     ⛰  Sunset SB (#726) - 🏕  Pines Hollow and Dunes Camp (Sites 38 – 90) (#734)
2023-06-06 05:59:17,078 [    INFO]:     ⛰  Sunset SB (#726) - 🏕  Sunset Group Area (#735)
2023-06-06 05:59:17,078 [    INFO]:     ⛰  Sunset SB (#726) - 🏕  South Camp (sites 1-37) (#737)
2023-06-06 05:59:17,078 [    INFO]: Searching Pines Hollow and Dunes Camp (Sites 38 – 90), Sunset SB (734) for availability: June, 2023
2023-06-06 05:59:17,113 [    INFO]: Exception encountered, emitting notification last gasp.
2023-06-06 05:59:17,809 [  CAMPLY]: Exiting camply 👋
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /usr/local/lib/python3.9/site-packages/camply/providers/usedirect/usedirect.py:328 in            │
│ get_campsites_response                                                                           │
│                                                                                                  │
│   325 │   │   response.raise_for_status()                                                        │
│   326 │   │   response_json = response.json()                                                    │
│   327 │   │   try:                                                                               │
│ ❱ 328 │   │   │   return UseDirectAvailabilityResponse(**response.json())                        │
│   329 │   │   except ValidationError as e:                                                       │
│   330 │   │   │   error_message = (                                                              │
│   331 │   │   │   │   "Error Parsing UseDirect Availability Response "                           │
│                                                                                                  │
│ in pydantic.main.BaseModel.__init__:341                                                          │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValidationError: 10 validation errors for UseDirectAvailabilityResponse
Filters
  none is not an allowed value (type=type_error.none.not_allowed)
EndDate
  none is not an allowed value (type=type_error.none.not_allowed)
TimeZone
  none is not an allowed value (type=type_error.none.not_allowed)
TimeStamp
  none is not an allowed value (type=type_error.none.not_allowed)
UnitSort
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> Name
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> Description
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> Restrictions
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> SeasonDates
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> TrafficStatuses
  none is not an allowed value (type=type_error.none.not_allowed)

The above exception was the direct cause of the following exception:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /usr/local/lib/python3.9/site-packages/camply/search/base_search.py:574 in                       │
│ get_matching_campsites                                                                           │
│                                                                                                  │
│    571 │   │   """                                                                               │
│    572 │   │   if continuous is True or search_once is True:                                     │
│    573 │   │   │   try:                                                                          │
│ ❱  574 │   │   │   │   self._search_campsites_continuous(                                        │
│    575 │   │   │   │   │   log=log,                                                              │
│    576 │   │   │   │   │   verbose=verbose,                                                      │
│    577 │   │   │   │   │   polling_interval=polling_interval,                                    │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/search/base_search.py:508 in                       │
│ _search_campsites_continuous                                                                     │
│                                                                                                  │
│    505 │   │   continuous_search_attempts = 1                                                    │
│    506 │   │   while continuous_search is True:                                                  │
│    507 │   │   │   starting_count = len(self.campsites_found)                                    │
│ ❱  508 │   │   │   self._continuous_search_retry(                                                │
│    509 │   │   │   │   log=log,                                                                  │
│    510 │   │   │   │   verbose=verbose,                                                          │
│    511 │   │   │   │   polling_interval=polling_interval,                                        │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/search/base_search.py:354 in                       │
│ _continuous_search_retry                                                                         │
│                                                                                                  │
│    351 │   │   │   retry=tenacity.retry_if_exception_type(CampsiteNotFoundError),                │
│    352 │   │   │   wait=tenacity.wait.wait_fixed(int(polling_interval_minutes) * 60),            │
│    353 │   │   )                                                                                 │
│ ❱  354 │   │   matching_campsites = retryer.__call__(                                            │
│    355 │   │   │   fn=self._search_matching_campsites_available,                                 │
│    356 │   │   │   log=False,                                                                    │
│    357 │   │   │   verbose=False,                                                                │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/tenacity/__init__.py:379 in __call__                      │
│                                                                                                  │
│   376 │   │                                                                                      │
│   377 │   │   retry_state = RetryCallState(retry_object=self, fn=fn, args=args, kwargs=kwargs)   │
│   378 │   │   while True:                                                                        │
│ ❱ 379 │   │   │   do = self.iter(retry_state=retry_state)                                        │
│   380 │   │   │   if isinstance(do, DoAttempt):                                                  │
│   381 │   │   │   │   try:                                                                       │
│   382 │   │   │   │   │   result = fn(*args, **kwargs)                                           │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/tenacity/__init__.py:314 in iter                          │
│                                                                                                  │
│   311 │   │                                                                                      │
│   312 │   │   is_explicit_retry = fut.failed and isinstance(fut.exception(), TryAgain)           │
│   313 │   │   if not (is_explicit_retry or self.retry(retry_state)):                             │
│ ❱ 314 │   │   │   return fut.result()                                                            │
│   315 │   │                                                                                      │
│   316 │   │   if self.after is not None:                                                         │
│   317 │   │   │   self.after(retry_state)                                                        │
│                                                                                                  │
│ /usr/local/lib/python3.9/concurrent/futures/_base.py:439 in result                               │
│                                                                                                  │
│   436 │   │   │   │   if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]:                     │
│   437 │   │   │   │   │   raise CancelledError()                                                 │
│   438 │   │   │   │   elif self._state == FINISHED:                                              │
│ ❱ 439 │   │   │   │   │   return self.__get_result()                                             │
│   440 │   │   │   │                                                                              │
│   441 │   │   │   │   self._condition.wait(timeout)                                              │
│   442                                                                                            │
│                                                                                                  │
│ /usr/local/lib/python3.9/concurrent/futures/_base.py:391 in __get_result                         │
│                                                                                                  │
│   388 │   def __get_result(self):                                                                │
│   389 │   │   if self._exception:                                                                │
│   390 │   │   │   try:                                                                           │
│ ❱ 391 │   │   │   │   raise self._exception                                                      │
│   392 │   │   │   finally:                                                                       │
│   393 │   │   │   │   # Break a reference cycle with the exception in self._exception            │
│   394 │   │   │   │   self = None                                                                │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/tenacity/__init__.py:382 in __call__                      │
│                                                                                                  │
│   379 │   │   │   do = self.iter(retry_state=retry_state)                                        │
│   380 │   │   │   if isinstance(do, DoAttempt):                                                  │
│   381 │   │   │   │   try:                                                                       │
│ ❱ 382 │   │   │   │   │   result = fn(*args, **kwargs)                                           │
│   383 │   │   │   │   except BaseException:  # noqa: B902                                        │
│   384 │   │   │   │   │   retry_state.set_exception(sys.exc_info())  # type: ignore[arg-type]    │
│   385 │   │   │   │   else:                                                                      │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/search/base_search.py:246 in                       │
│ _search_matching_campsites_available                                                             │
│                                                                                                  │
│    243 │   │   List[AvailableCampsite]                                                           │
│    244 │   │   """                                                                               │
│    245 │   │   matching_campgrounds = []                                                         │
│ ❱  246 │   │   for camp in self.get_all_campsites():                                             │
│    247 │   │   │   if all(                                                                       │
│    248 │   │   │   │   [                                                                         │
│    249 │   │   │   │   │   self._compare_date_overlap(campsite=camp) is True,                    │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/search/search_usedirect.py:128 in                  │
│ get_all_campsites                                                                                │
│                                                                                                  │
│   125 │   │   │   │   │   f"{month.strftime('%B, %Y')}"                                          │
│   126 │   │   │   │   )                                                                          │
│   127 │   │   │   │   end_date = month + relativedelta(months=1) - timedelta(days=1)             │
│ ❱ 128 │   │   │   │   campsites = self.campsite_finder.get_campsites(                            │
│   129 │   │   │   │   │   campground_id=campground.facility_id,                                  │
│   130 │   │   │   │   │   start_date=month,                                                      │
│   131 │   │   │   │   │   end_date=end_date,                                                     │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/providers/usedirect/usedirect.py:385 in            │
│ get_campsites                                                                                    │
│                                                                                                  │
│   382 │   │   List[AvailableCampsite]                                                            │
│   383 │   │   """                                                                                │
│   384 │   │   self.refresh_metadata()                                                            │
│ ❱ 385 │   │   availability_response = self.get_campsites_response(                               │
│   386 │   │   │   campground_id=campground_id,                                                   │
│   387 │   │   │   start_date=start_date,                                                         │
│   388 │   │   │   end_date=end_date,                                                             │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/ratelimit/decorators.py:113 in wrapper                    │
│                                                                                                  │
│   110 │   │   '''                                                                                │
│   111 │   │   while True:                                                                        │
│   112 │   │   │   try:                                                                           │
│ ❱ 113 │   │   │   │   return func(*args, **kargs)                                                │
│   114 │   │   │   except RateLimitException as exception:                                        │
│   115 │   │   │   │   time.sleep(exception.period_remaining)                                     │
│   116 │   return wrapper                                                                         │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/ratelimit/decorators.py:80 in wrapper                     │
│                                                                                                  │
│    77 │   │   │   │   │   │   raise RateLimitException('too many calls', period_remaining)       │
│    78 │   │   │   │   │   return                                                                 │
│    79 │   │   │                                                                                  │
│ ❱  80 │   │   │   return func(*args, **kargs)                                                    │
│    81 │   │   return wrapper                                                                     │
│    82 │                                                                                          │
│    83 │   def __period_remaining(self):                                                          │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/providers/usedirect/usedirect.py:336 in            │
│ get_campsites_response                                                                           │
│                                                                                                  │
│   333 │   │   │   )                                                                              │
│   334 │   │   │   if "Message" in response_json:                                                 │
│   335 │   │   │   │   error_message += " " + response_json["Message"]                            │
│ ❱ 336 │   │   │   raise UseDirectError(error_message) from e                                     │
│   337 │                                                                                          │
│   338 │   def get_campsites(                                                                     │
│   339 │   │   self,                                                                              │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
UseDirectError: Error Parsing UseDirect Availability Response - Facility ID # 734. Invalid FacilityId specified for facility grid search on CALIRDR6

The above exception was the direct cause of the following exception:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /usr/local/bin/camply:8 in <module>                                                              │
│                                                                                                  │
│   5 from camply.cli import cli                                                                   │
│   6 if __name__ == '__main__':                                                                   │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                         │
│ ❱ 8 │   sys.exit(cli())                                                                          │
│   9                                                                                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/cli.py:834 in cli                                  │
│                                                                                                  │
│   831 │   Camply Command Line Utility Wrapper                                                    │
│   832 │   """                                                                                    │
│   833 │   try:                                                                                   │
│ ❱ 834 │   │   camply_command_line()                                                              │
│   835 │   except KeyboardInterrupt:                                                              │
│   836 │   │   logger.debug("Handling Exit Request")                                              │
│   837 │   finally:                                                                               │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1130 in __call__                            │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/rich_click/rich_group.py:21 in main                       │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1055 in main                                │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1657 in invoke                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1404 in invoke                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:760 in invoke                               │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/decorators.py:38 in new_func                        │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/cli.py:726 in campsites                            │
│                                                                                                  │
│   723 │   │   }                                                                                  │
│   724 │   provider_class: Type[BaseCampingSearch] = CAMPSITE_SEARCH_PROVIDER[provider]           │
│   725 │   camping_finder: BaseCampingSearch = provider_class(**provider_kwargs)                  │
│ ❱ 726 │   camping_finder.get_matching_campsites(**search_kwargs)                                 │
│   727                                                                                            │
│   728                                                                                            │
│   729 @camply_command_line.command(cls=RichCommand)                                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/search/base_search.py:584 in                       │
│ get_matching_campsites                                                                           │
│                                                                                                  │
│    581 │   │   │   │   │   search_once=search_once,                                              │
│    582 │   │   │   │   )                                                                         │
│    583 │   │   │   except Exception as e:                                                        │
│ ❱  584 │   │   │   │   self.notifier.last_gasp(error=e)                                          │
│    585 │   │   │   │   raise e                                                                   │
│    586 │   │   else:                                                                             │
│    587 │   │   │   starting_count = len(self.campsites_found)                                    │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/notifications/multi_provider_notifications.py:127  │
│ in last_gasp                                                                                     │
│                                                                                                  │
│   124 │   │   )                                                                                  │
│   125 │   │   for provider in self.providers:                                                    │
│   126 │   │   │   provider.send_message(error_message)                                           │
│ ❱ 127 │   │   raise RuntimeError(error_message) from error                                       │
│   128                                                                                            │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: camply encountered an error and exited 😟 [2023-06-06 05:59:17] - (UseDirectError) Error Parsing UseDirect Availability Response - Facility ID # 734. 
Invalid FacilityId specified for facility grid search on CALIRDR6

Additional context

< Add any other context about the problem here. >

juftin commented 1 year ago

Apologies for the spam messages, there are certain kind of errors handled gracefully, others aren't. In this case it's an API response validation error which is core to camply and not ignorable unfortunately. 

Filters
  none is not an allowed value (type=type_error.none.not_allowed)
EndDate
  none is not an allowed value (type=type_error.none.not_allowed)
TimeZone
  none is not an allowed value (type=type_error.none.not_allowed)
TimeStamp
  none is not an allowed value (type=type_error.none.not_allowed)
UnitSort
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> Name
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> Description
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> Restrictions
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> SeasonDates
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> TrafficStatuses
  none is not an allowed value (type=type_error.none.not_allowed)

I'm probably being a bit over ambitious in my API response parsing of the ReserveCalifornia API - I can set lots of unnecessary attributes to optional and that should resolve this issue.

juftin commented 1 year ago

Okay, #265 is merged. camply 0.24.3 was just released and you should find camply to be much more stable for ReserveCalifornia.

FWIW, I might recommend running camply inside docker with the restart-mode set to "unless-stopped". This won't solve the problem here with camply being buggy, but it will just restart the same search for you when it does happen. See https://juftin.com/camply/how_to_run/#running-in-docker if you're interested.

Please let me know if you see your issue resolved @vassoz 🤝

vassoz commented 1 year ago

I run it for couple days just for test (same script as above). It's much more stable. Thank you for fixing it that quickly!

I got the following error notification once:

 camply encountered an error and exited 😟 [2023-06-13 18:44:34] - (ValidationError) 3 validation errors for UseDirectAvailabilityResponse
Filters
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> SeasonDates
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> TrafficStatuses
  none is not an allowed value (type=type_error.none.not_allowed)
2023-06-13 18:44:33,808 [  CAMPLY]: camply, the campsite finder ⛺️
2023-06-13 18:44:33,809 [    INFO]: Using Camply Provider: "ReserveCalifornia"
2023-06-13 18:44:33,814 [    INFO]: 1 booking nights selected for search, ranging from 2023-06-17 to 2023-06-17
2023-06-13 18:44:34,483 [    INFO]: Searching for campsites every 10 minutes. 
2023-06-13 18:44:34,483 [    INFO]: Notifications active via: <SilentNotifications>, <TelegramNotifications>
2023-06-13 18:44:34,483 [    INFO]: Searching across 2 campgrounds
2023-06-13 18:44:34,483 [    INFO]:     ⛰  Sunset SB (#726) - 🏕  Pines Hollow and Dunes Camp (Sites 38 – 90) (#734)
2023-06-13 18:44:34,484 [    INFO]:     ⛰  Sunset SB (#726) - 🏕  South Camp (sites 1-37) (#737)
2023-06-13 18:44:34,484 [    INFO]: Searching Pines Hollow and Dunes Camp (Sites 38 – 90), Sunset SB (734) for availability: June, 2023
2023-06-13 18:44:34,516 [    INFO]: Exception encountered, emitting notification last gasp.
2023-06-13 18:44:35,193 [  CAMPLY]: Exiting camply 👋
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /usr/local/lib/python3.9/site-packages/camply/search/base_search.py:574 in                       │
│ get_matching_campsites                                                                           │
│                                                                                                  │
│    571 │   │   """                                                                               │
│    572 │   │   if continuous is True or search_once is True:                                     │
│    573 │   │   │   try:                                                                          │
│ ❱  574 │   │   │   │   self._search_campsites_continuous(                                        │
│    575 │   │   │   │   │   log=log,                                                              │
│    576 │   │   │   │   │   verbose=verbose,                                                      │
│    577 │   │   │   │   │   polling_interval=polling_interval,                                    │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/search/base_search.py:508 in                       │
│ _search_campsites_continuous                                                                     │
│                                                                                                  │
│    505 │   │   continuous_search_attempts = 1                                                    │
│    506 │   │   while continuous_search is True:                                                  │
│    507 │   │   │   starting_count = len(self.campsites_found)                                    │
│ ❱  508 │   │   │   self._continuous_search_retry(                                                │
│    509 │   │   │   │   log=log,                                                                  │
│    510 │   │   │   │   verbose=verbose,                                                          │
│    511 │   │   │   │   polling_interval=polling_interval,                                        │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/search/base_search.py:354 in                       │
│ _continuous_search_retry                                                                         │
│                                                                                                  │
│    351 │   │   │   retry=tenacity.retry_if_exception_type(CampsiteNotFoundError),                │
│    352 │   │   │   wait=tenacity.wait.wait_fixed(int(polling_interval_minutes) * 60),            │
│    353 │   │   )                                                                                 │
│ ❱  354 │   │   matching_campsites = retryer.__call__(                                            │
│    355 │   │   │   fn=self._search_matching_campsites_available,                                 │
│    356 │   │   │   log=False,                                                                    │
│    357 │   │   │   verbose=False,                                                                │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/tenacity/__init__.py:379 in __call__                      │
│                                                                                                  │
│   376 │   │                                                                                      │
│   377 │   │   retry_state = RetryCallState(retry_object=self, fn=fn, args=args, kwargs=kwargs)   │
│   378 │   │   while True:                                                                        │
│ ❱ 379 │   │   │   do = self.iter(retry_state=retry_state)                                        │
│   380 │   │   │   if isinstance(do, DoAttempt):                                                  │
│   381 │   │   │   │   try:                                                                       │
│   382 │   │   │   │   │   result = fn(*args, **kwargs)                                           │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/tenacity/__init__.py:314 in iter                          │
│                                                                                                  │
│   311 │   │                                                                                      │
│   312 │   │   is_explicit_retry = fut.failed and isinstance(fut.exception(), TryAgain)           │
│   313 │   │   if not (is_explicit_retry or self.retry(retry_state)):                             │
│ ❱ 314 │   │   │   return fut.result()                                                            │
│   315 │   │                                                                                      │
│   316 │   │   if self.after is not None:                                                         │
│   317 │   │   │   self.after(retry_state)                                                        │
│                                                                                                  │
│ /usr/local/lib/python3.9/concurrent/futures/_base.py:439 in result                               │
│                                                                                                  │
│   436 │   │   │   │   if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]:                     │
│   437 │   │   │   │   │   raise CancelledError()                                                 │
│   438 │   │   │   │   elif self._state == FINISHED:                                              │
│ ❱ 439 │   │   │   │   │   return self.__get_result()                                             │
│   440 │   │   │   │                                                                              │
│   441 │   │   │   │   self._condition.wait(timeout)                                              │
│   442                                                                                            │
│                                                                                                  │
│ /usr/local/lib/python3.9/concurrent/futures/_base.py:391 in __get_result                         │
│                                                                                                  │
│   388 │   def __get_result(self):                                                                │
│   389 │   │   if self._exception:                                                                │
│   390 │   │   │   try:                                                                           │
│ ❱ 391 │   │   │   │   raise self._exception                                                      │
│   392 │   │   │   finally:                                                                       │
│   393 │   │   │   │   # Break a reference cycle with the exception in self._exception            │
│   394 │   │   │   │   self = None                                                                │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/tenacity/__init__.py:382 in __call__                      │
│                                                                                                  │
│   379 │   │   │   do = self.iter(retry_state=retry_state)                                        │
│   380 │   │   │   if isinstance(do, DoAttempt):                                                  │
│   381 │   │   │   │   try:                                                                       │
│ ❱ 382 │   │   │   │   │   result = fn(*args, **kwargs)                                           │
│   383 │   │   │   │   except BaseException:  # noqa: B902                                        │
│   384 │   │   │   │   │   retry_state.set_exception(sys.exc_info())  # type: ignore[arg-type]    │
│   385 │   │   │   │   else:                                                                      │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/search/base_search.py:246 in                       │
│ _search_matching_campsites_available                                                             │
│                                                                                                  │
│    243 │   │   List[AvailableCampsite]                                                           │
│    244 │   │   """                                                                               │
│    245 │   │   matching_campgrounds = []                                                         │
│ ❱  246 │   │   for camp in self.get_all_campsites():                                             │
│    247 │   │   │   if all(                                                                       │
│    248 │   │   │   │   [                                                                         │
│    249 │   │   │   │   │   self._compare_date_overlap(campsite=camp) is True,                    │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/search/search_usedirect.py:139 in                  │
│ get_all_campsites                                                                                │
│                                                                                                  │
│   136 │   │   │   │   │   f"{month.strftime('%B, %Y')}"                                          │
│   137 │   │   │   │   )                                                                          │
│   138 │   │   │   │   end_date = month + relativedelta(months=1) - timedelta(days=1)             │
│ ❱ 139 │   │   │   │   campsites = self.campsite_finder.get_campsites(                            │
│   140 │   │   │   │   │   campground_id=campground.facility_id,                                  │
│   141 │   │   │   │   │   start_date=month,                                                      │
│   142 │   │   │   │   │   end_date=end_date,                                                     │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/providers/usedirect/usedirect.py:388 in            │
│ get_campsites                                                                                    │
│                                                                                                  │
│   385 │   │   List[AvailableCampsite]                                                            │
│   386 │   │   """                                                                                │
│   387 │   │   self.refresh_metadata()                                                            │
│ ❱ 388 │   │   availability_response = self.get_campsites_response(                               │
│   389 │   │   │   campground_id=campground_id,                                                   │
│   390 │   │   │   start_date=start_date,                                                         │
│   391 │   │   │   end_date=end_date,                                                             │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/ratelimit/decorators.py:113 in wrapper                    │
│                                                                                                  │
│   110 │   │   '''                                                                                │
│   111 │   │   while True:                                                                        │
│   112 │   │   │   try:                                                                           │
│ ❱ 113 │   │   │   │   return func(*args, **kargs)                                                │
│   114 │   │   │   except RateLimitException as exception:                                        │
│   115 │   │   │   │   time.sleep(exception.period_remaining)                                     │
│   116 │   return wrapper                                                                         │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/ratelimit/decorators.py:80 in wrapper                     │
│                                                                                                  │
│    77 │   │   │   │   │   │   raise RateLimitException('too many calls', period_remaining)       │
│    78 │   │   │   │   │   return                                                                 │
│    79 │   │   │                                                                                  │
│ ❱  80 │   │   │   return func(*args, **kargs)                                                    │
│    81 │   │   return wrapper                                                                     │
│    82 │                                                                                          │
│    83 │   def __period_remaining(self):                                                          │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/providers/usedirect/usedirect.py:330 in            │
│ get_campsites_response                                                                           │
│                                                                                                  │
│   327 │   │   response.raise_for_status()                                                        │
│   328 │   │   response_json = response.json()                                                    │
│   329 │   │   try:                                                                               │
│ ❱ 330 │   │   │   return UseDirectAvailabilityResponse(**response.json())                        │
│   331 │   │   except ValidationError as e:                                                       │
│   332 │   │   │   raise                                                                          │
│   333 │   │   │   error_message = (                                                              │
│                                                                                                  │
│ in pydantic.main.BaseModel.__init__:341                                                          │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValidationError: 3 validation errors for UseDirectAvailabilityResponse
Filters
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> SeasonDates
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> TrafficStatuses
  none is not an allowed value (type=type_error.none.not_allowed)

The above exception was the direct cause of the following exception:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /usr/local/bin/camply:8 in <module>                                                              │
│                                                                                                  │
│   5 from camply.cli import cli                                                                   │
│   6 if __name__ == '__main__':                                                                   │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                         │
│ ❱ 8 │   sys.exit(cli())                                                                          │
│   9                                                                                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/cli.py:834 in cli                                  │
│                                                                                                  │
│   831 │   Camply Command Line Utility Wrapper                                                    │
│   832 │   """                                                                                    │
│   833 │   try:                                                                                   │
│ ❱ 834 │   │   camply_command_line()                                                              │
│   835 │   except KeyboardInterrupt:                                                              │
│   836 │   │   logger.debug("Handling Exit Request")                                              │
│   837 │   finally:                                                                               │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1130 in __call__                            │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/rich_click/rich_group.py:21 in main                       │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1055 in main                                │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1657 in invoke                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1404 in invoke                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:760 in invoke                               │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/decorators.py:38 in new_func                        │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/cli.py:726 in campsites                            │
│                                                                                                  │
│   723 │   │   }                                                                                  │
│   724 │   provider_class: Type[BaseCampingSearch] = CAMPSITE_SEARCH_PROVIDER[provider]           │
│   725 │   camping_finder: BaseCampingSearch = provider_class(**provider_kwargs)                  │
│ ❱ 726 │   camping_finder.get_matching_campsites(**search_kwargs)                                 │
│   727                                                                                            │
│   728                                                                                            │
│   729 @camply_command_line.command(cls=RichCommand)                                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/search/base_search.py:584 in                       │
│ get_matching_campsites                                                                           │
│                                                                                                  │
│    581 │   │   │   │   │   search_once=search_once,                                              │
│    582 │   │   │   │   )                                                                         │
│    583 │   │   │   except Exception as e:                                                        │
│ ❱  584 │   │   │   │   self.notifier.last_gasp(error=e)                                          │
│    585 │   │   │   │   raise e                                                                   │
│    586 │   │   else:                                                                             │
│    587 │   │   │   starting_count = len(self.campsites_found)                                    │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/notifications/multi_provider_notifications.py:127  │
│ in last_gasp                                                                                     │
│                                                                                                  │
│   124 │   │   )                                                                                  │
│   125 │   │   for provider in self.providers:                                                    │
│   126 │   │   │   provider.send_message(error_message)                                           │
│ ❱ 127 │   │   raise RuntimeError(error_message) from error                                       │
│   128                                                                                            │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: camply encountered an error and exited 😟 [2023-06-13 18:44:34] - (ValidationError) 3 validation errors for UseDirectAvailabilityResponse
Filters
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> SeasonDates
  none is not an allowed value (type=type_error.none.not_allowed)
Facility -> TrafficStatuses
  none is not an allowed value (type=type_error.none.not_allowed)

Also I got multiple following errors (no notifications though):

2023-06-13 10:21:36,508 [  CAMPLY]: camply, the campsite finder ⛺️
2023-06-13 10:21:36,509 [    INFO]: Using Camply Provider: "ReserveCalifornia"
2023-06-13 10:21:36,514 [    INFO]: 1 booking nights selected for search, ranging from 2023-06-17 to 2023-06-17
2023-06-13 10:37:45,694 [  CAMPLY]: Exiting camply 👋
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py:790 in urlopen                  │
│                                                                                                  │
│    787 │   │   │   response_conn = conn if not release_conn else None                            │
│    788 │   │   │                                                                                 │
│    789 │   │   │   # Make the request on the HTTPConnection object                               │
│ ❱  790 │   │   │   response = self._make_request(                                                │
│    791 │   │   │   │   conn,                                                                     │
│    792 │   │   │   │   method,                                                                   │
│    793 │   │   │   │   url,                                                                      │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py:491 in _make_request            │
│                                                                                                  │
│    488 │   │   │   │   new_e, (OSError, NewConnectionError, TimeoutError, SSLError)              │
│    489 │   │   │   ) and (conn and conn.proxy and not conn.has_connected_to_proxy):              │
│    490 │   │   │   │   new_e = _wrap_proxy_error(new_e, conn.proxy.scheme)                       │
│ ❱  491 │   │   │   raise new_e                                                                   │
│    492 │   │                                                                                     │
│    493 │   │   # conn.request() calls http.client.*.request, not the method in                   │
│    494 │   │   # urllib3.request. It also calls makefile (recv) on the socket.                   │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py:467 in _make_request            │
│                                                                                                  │
│    464 │   │   try:                                                                              │
│    465 │   │   │   # Trigger any extra validation we need to do.                                 │
│    466 │   │   │   try:                                                                          │
│ ❱  467 │   │   │   │   self._validate_conn(conn)                                                 │
│    468 │   │   │   except (SocketTimeout, BaseSSLError) as e:                                    │
│    469 │   │   │   │   self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)           │
│    470 │   │   │   │   raise                                                                     │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py:1092 in _validate_conn          │
│                                                                                                  │
│   1089 │   │                                                                                     │
│   1090 │   │   # Force connect early to allow us to validate the connection.                     │
│   1091 │   │   if conn.is_closed:                                                                │
│ ❱ 1092 │   │   │   conn.connect()                                                                │
│   1093 │   │                                                                                     │
│   1094 │   │   if not conn.is_verified:                                                          │
│   1095 │   │   │   warnings.warn(                                                                │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/urllib3/connection.py:635 in connect                      │
│                                                                                                  │
│   632 │   │   │   │   SystemTimeWarning,                                                         │
│   633 │   │   │   )                                                                              │
│   634 │   │                                                                                      │
│ ❱ 635 │   │   sock_and_verified = _ssl_wrap_socket_and_match_hostname(                           │
│   636 │   │   │   sock=sock,                                                                     │
│   637 │   │   │   cert_reqs=self.cert_reqs,                                                      │
│   638 │   │   │   ssl_version=self.ssl_version,                                                  │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/urllib3/connection.py:774 in                              │
│ _ssl_wrap_socket_and_match_hostname                                                              │
│                                                                                                  │
│   771 │   │   if is_ipaddress(normalized):                                                       │
│   772 │   │   │   server_hostname = normalized                                                   │
│   773 │                                                                                          │
│ ❱ 774 │   ssl_sock = ssl_wrap_socket(                                                            │
│   775 │   │   sock=sock,                                                                         │
│   776 │   │   keyfile=key_file,                                                                  │
│   777 │   │   certfile=cert_file,                                                                │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/urllib3/util/ssl_.py:459 in ssl_wrap_socket               │
│                                                                                                  │
│   456 │   except NotImplementedError:  # Defensive: in CI, we always have set_alpn_protocols     │
│   457 │   │   pass                                                                               │
│   458 │                                                                                          │
│ ❱ 459 │   ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)           │
│   460 │   return ssl_sock                                                                        │
│   461                                                                                            │
│   462                                                                                            │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/urllib3/util/ssl_.py:503 in _ssl_wrap_socket_impl         │
│                                                                                                  │
│   500 │   │   SSLTransport._validate_ssl_context_for_tls_in_tls(ssl_context)                     │
│   501 │   │   return SSLTransport(sock, ssl_context, server_hostname)                            │
│   502 │                                                                                          │
│ ❱ 503 │   return ssl_context.wrap_socket(sock, server_hostname=server_hostname)                  │
│   504                                                                                            │
│                                                                                                  │
│ /usr/local/lib/python3.9/ssl.py:501 in wrap_socket                                               │
│                                                                                                  │
│    498 │   │   │   │   │   server_hostname=None, session=None):                                  │
│    499 │   │   # SSLSocket class handles server_hostname encoding before it calls                │
│    500 │   │   # ctx._wrap_socket()                                                              │
│ ❱  501 │   │   return self.sslsocket_class._create(                                              │
│    502 │   │   │   sock=sock,                                                                    │
│    503 │   │   │   server_side=server_side,                                                      │
│    504 │   │   │   do_handshake_on_connect=do_handshake_on_connect,                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/ssl.py:1041 in _create                                                  │
│                                                                                                  │
│   1038 │   │   │   │   │   if timeout == 0.0:                                                    │
│   1039 │   │   │   │   │   │   # non-blocking                                                    │
│   1040 │   │   │   │   │   │   raise ValueError("do_handshake_on_connect should not be specifie  │
│ ❱ 1041 │   │   │   │   │   self.do_handshake()                                                   │
│   1042 │   │   │   except (OSError, ValueError):                                                 │
│   1043 │   │   │   │   self.close()                                                              │
│   1044 │   │   │   │   raise                                                                     │
│                                                                                                  │
│ /usr/local/lib/python3.9/ssl.py:1310 in do_handshake                                             │
│                                                                                                  │
│   1307 │   │   try:                                                                              │
│   1308 │   │   │   if timeout == 0.0 and block:                                                  │
│   1309 │   │   │   │   self.settimeout(None)                                                     │
│ ❱ 1310 │   │   │   self._sslobj.do_handshake()                                                   │
│   1311 │   │   finally:                                                                          │
│   1312 │   │   │   self.settimeout(timeout)                                                      │
│   1313                                                                                           │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /usr/local/lib/python3.9/site-packages/requests/adapters.py:486 in send                          │
│                                                                                                  │
│   483 │   │   │   timeout = TimeoutSauce(connect=timeout, read=timeout)                          │
│   484 │   │                                                                                      │
│   485 │   │   try:                                                                               │
│ ❱ 486 │   │   │   resp = conn.urlopen(                                                           │
│   487 │   │   │   │   method=request.method,                                                     │
│   488 │   │   │   │   url=url,                                                                   │
│   489 │   │   │   │   body=request.body,                                                         │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py:844 in urlopen                  │
│                                                                                                  │
│    841 │   │   │   elif isinstance(new_e, (OSError, HTTPException)):                             │
│    842 │   │   │   │   new_e = ProtocolError("Connection aborted.", new_e)                       │
│    843 │   │   │                                                                                 │
│ ❱  844 │   │   │   retries = retries.increment(                                                  │
│    845 │   │   │   │   method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2]       │
│    846 │   │   │   )                                                                             │
│    847 │   │   │   retries.sleep()                                                               │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/urllib3/util/retry.py:470 in increment                    │
│                                                                                                  │
│   467 │   │   elif error and self._is_read_error(error):                                         │
│   468 │   │   │   # Read retry?                                                                  │
│   469 │   │   │   if read is False or method is None or not self._is_method_retryable(method):   │
│ ❱ 470 │   │   │   │   raise reraise(type(error), error, _stacktrace)                             │
│   471 │   │   │   elif read is not None:                                                         │
│   472 │   │   │   │   read -= 1                                                                  │
│   473                                                                                            │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/urllib3/util/util.py:38 in reraise                        │
│                                                                                                  │
│   35 ) -> typing.NoReturn:                                                                       │
│   36 │   try:                                                                                    │
│   37 │   │   if value.__traceback__ is not tb:                                                   │
│ ❱ 38 │   │   │   raise value.with_traceback(tb)                                                  │
│   39 │   │   raise value                                                                         │
│   40 │   finally:                                                                                │
│   41 │   │   value = None  # type: ignore[assignment]                                            │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py:790 in urlopen                  │
│                                                                                                  │
│    787 │   │   │   response_conn = conn if not release_conn else None                            │
│    788 │   │   │                                                                                 │
│    789 │   │   │   # Make the request on the HTTPConnection object                               │
│ ❱  790 │   │   │   response = self._make_request(                                                │
│    791 │   │   │   │   conn,                                                                     │
│    792 │   │   │   │   method,                                                                   │
│    793 │   │   │   │   url,                                                                      │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py:491 in _make_request            │
│                                                                                                  │
│    488 │   │   │   │   new_e, (OSError, NewConnectionError, TimeoutError, SSLError)              │
│    489 │   │   │   ) and (conn and conn.proxy and not conn.has_connected_to_proxy):              │
│    490 │   │   │   │   new_e = _wrap_proxy_error(new_e, conn.proxy.scheme)                       │
│ ❱  491 │   │   │   raise new_e                                                                   │
│    492 │   │                                                                                     │
│    493 │   │   # conn.request() calls http.client.*.request, not the method in                   │
│    494 │   │   # urllib3.request. It also calls makefile (recv) on the socket.                   │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py:467 in _make_request            │
│                                                                                                  │
│    464 │   │   try:                                                                              │
│    465 │   │   │   # Trigger any extra validation we need to do.                                 │
│    466 │   │   │   try:                                                                          │
│ ❱  467 │   │   │   │   self._validate_conn(conn)                                                 │
│    468 │   │   │   except (SocketTimeout, BaseSSLError) as e:                                    │
│    469 │   │   │   │   self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)           │
│    470 │   │   │   │   raise                                                                     │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py:1092 in _validate_conn          │
│                                                                                                  │
│   1089 │   │                                                                                     │
│   1090 │   │   # Force connect early to allow us to validate the connection.                     │
│   1091 │   │   if conn.is_closed:                                                                │
│ ❱ 1092 │   │   │   conn.connect()                                                                │
│   1093 │   │                                                                                     │
│   1094 │   │   if not conn.is_verified:                                                          │
│   1095 │   │   │   warnings.warn(                                                                │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/urllib3/connection.py:635 in connect                      │
│                                                                                                  │
│   632 │   │   │   │   SystemTimeWarning,                                                         │
│   633 │   │   │   )                                                                              │
│   634 │   │                                                                                      │
│ ❱ 635 │   │   sock_and_verified = _ssl_wrap_socket_and_match_hostname(                           │
│   636 │   │   │   sock=sock,                                                                     │
│   637 │   │   │   cert_reqs=self.cert_reqs,                                                      │
│   638 │   │   │   ssl_version=self.ssl_version,                                                  │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/urllib3/connection.py:774 in                              │
│ _ssl_wrap_socket_and_match_hostname                                                              │
│                                                                                                  │
│   771 │   │   if is_ipaddress(normalized):                                                       │
│   772 │   │   │   server_hostname = normalized                                                   │
│   773 │                                                                                          │
│ ❱ 774 │   ssl_sock = ssl_wrap_socket(                                                            │
│   775 │   │   sock=sock,                                                                         │
│   776 │   │   keyfile=key_file,                                                                  │
│   777 │   │   certfile=cert_file,                                                                │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/urllib3/util/ssl_.py:459 in ssl_wrap_socket               │
│                                                                                                  │
│   456 │   except NotImplementedError:  # Defensive: in CI, we always have set_alpn_protocols     │
│   457 │   │   pass                                                                               │
│   458 │                                                                                          │
│ ❱ 459 │   ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)           │
│   460 │   return ssl_sock                                                                        │
│   461                                                                                            │
│   462                                                                                            │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/urllib3/util/ssl_.py:503 in _ssl_wrap_socket_impl         │
│                                                                                                  │
│   500 │   │   SSLTransport._validate_ssl_context_for_tls_in_tls(ssl_context)                     │
│   501 │   │   return SSLTransport(sock, ssl_context, server_hostname)                            │
│   502 │                                                                                          │
│ ❱ 503 │   return ssl_context.wrap_socket(sock, server_hostname=server_hostname)                  │
│   504                                                                                            │
│                                                                                                  │
│ /usr/local/lib/python3.9/ssl.py:501 in wrap_socket                                               │
│                                                                                                  │
│    498 │   │   │   │   │   server_hostname=None, session=None):                                  │
│    499 │   │   # SSLSocket class handles server_hostname encoding before it calls                │
│    500 │   │   # ctx._wrap_socket()                                                              │
│ ❱  501 │   │   return self.sslsocket_class._create(                                              │
│    502 │   │   │   sock=sock,                                                                    │
│    503 │   │   │   server_side=server_side,                                                      │
│    504 │   │   │   do_handshake_on_connect=do_handshake_on_connect,                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/ssl.py:1041 in _create                                                  │
│                                                                                                  │
│   1038 │   │   │   │   │   if timeout == 0.0:                                                    │
│   1039 │   │   │   │   │   │   # non-blocking                                                    │
│   1040 │   │   │   │   │   │   raise ValueError("do_handshake_on_connect should not be specifie  │
│ ❱ 1041 │   │   │   │   │   self.do_handshake()                                                   │
│   1042 │   │   │   except (OSError, ValueError):                                                 │
│   1043 │   │   │   │   self.close()                                                              │
│   1044 │   │   │   │   raise                                                                     │
│                                                                                                  │
│ /usr/local/lib/python3.9/ssl.py:1310 in do_handshake                                             │
│                                                                                                  │
│   1307 │   │   try:                                                                              │
│   1308 │   │   │   if timeout == 0.0 and block:                                                  │
│   1309 │   │   │   │   self.settimeout(None)                                                     │
│ ❱ 1310 │   │   │   self._sslobj.do_handshake()                                                   │
│   1311 │   │   finally:                                                                          │
│   1312 │   │   │   self.settimeout(timeout)                                                      │
│   1313                                                                                           │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ProtocolError: ('Connection aborted.', TimeoutError(110, 'Connection timed out'))

During handling of the above exception, another exception occurred:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /usr/local/bin/camply:8 in <module>                                                              │
│                                                                                                  │
│   5 from camply.cli import cli                                                                   │
│   6 if __name__ == '__main__':                                                                   │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                         │
│ ❱ 8 │   sys.exit(cli())                                                                          │
│   9                                                                                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/cli.py:834 in cli                                  │
│                                                                                                  │
│   831 │   Camply Command Line Utility Wrapper                                                    │
│   832 │   """                                                                                    │
│   833 │   try:                                                                                   │
│ ❱ 834 │   │   camply_command_line()                                                              │
│   835 │   except KeyboardInterrupt:                                                              │
│   836 │   │   logger.debug("Handling Exit Request")                                              │
│   837 │   finally:                                                                               │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1130 in __call__                            │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/rich_click/rich_group.py:21 in main                       │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1055 in main                                │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1657 in invoke                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1404 in invoke                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/core.py:760 in invoke                               │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/click/decorators.py:38 in new_func                        │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/cli.py:725 in campsites                            │
│                                                                                                  │
│   722 │   │   │   "search_once": search_once,                                                    │
│   723 │   │   }                                                                                  │
│   724 │   provider_class: Type[BaseCampingSearch] = CAMPSITE_SEARCH_PROVIDER[provider]           │
│ ❱ 725 │   camping_finder: BaseCampingSearch = provider_class(**provider_kwargs)                  │
│   726 │   camping_finder.get_matching_campsites(**search_kwargs)                                 │
│   727                                                                                            │
│   728                                                                                            │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/search/search_usedirect.py:96 in __init__          │
│                                                                                                  │
│    93 │   │   │   )                                                                              │
│    94 │   │   │   sys.exit(1)                                                                    │
│    95 │   │   if self._campground_ids:                                                           │
│ ❱  96 │   │   │   self.campgrounds = self.campsite_finder.find_campgrounds(                      │
│    97 │   │   │   │   campground_id=self._campground_ids,                                        │
│    98 │   │   │   │   verbose=False,                                                             │
│    99 │   │   │   )                                                                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/providers/usedirect/usedirect.py:201 in            │
│ find_campgrounds                                                                                 │
│                                                                                                  │
│   198 │   │   │   sys.exit(1)                                                                    │
│   199 │   │   if search_string is not None:                                                      │
│   200 │   │   │   logger.info(f'Searching for Campgrounds: "{search_string}"')                   │
│ ❱ 201 │   │   self.refresh_metadata()                                                            │
│   202 │   │   found_campgrounds = self._search_for_campgrounds(                                  │
│   203 │   │   │   campground_id=campground_id,                                                   │
│   204 │   │   │   rec_area_id=rec_area_id,                                                       │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/providers/usedirect/usedirect.py:120 in            │
│ refresh_metadata                                                                                 │
│                                                                                                  │
│   117 │   │   None                                                                               │
│   118 │   │   """                                                                                │
│   119 │   │   if self.metadata_refreshed is False:                                               │
│ ❱ 120 │   │   │   self._get_campground_metadata()                                                │
│   121 │   │   │   self._get_city_parks()                                                         │
│   122 │   │   │   self._get_places()                                                             │
│   123 │   │   │   self._get_facilities()                                                         │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/camply/providers/usedirect/usedirect.py:520 in            │
│ _get_campground_metadata                                                                         │
│                                                                                                  │
│   517 │   │   if campground_metadata is None:                                                    │
│   518 │   │   │   self.offline_cache_dir.mkdir(parents=True, exist_ok=True)                      │
│   519 │   │   │   url = f"{self.base_url}/{self.rdr_path}/{UseDirectConfig.METADATA_PREFIX}"     │
│ ❱ 520 │   │   │   resp = self.session.get(url=url)                                               │
│   521 │   │   │   resp.raise_for_status()                                                        │
│   522 │   │   │   campground_metadata = resp.json()                                              │
│   523 │   │   │   metadata_file.write_text(json.dumps(campground_metadata, indent=2))            │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/requests/sessions.py:602 in get                           │
│                                                                                                  │
│   599 │   │   """                                                                                │
│   600 │   │                                                                                      │
│   601 │   │   kwargs.setdefault("allow_redirects", True)                                         │
│ ❱ 602 │   │   return self.request("GET", url, **kwargs)                                          │
│   603 │                                                                                          │
│   604 │   def options(self, url, **kwargs):                                                      │
│   605 │   │   r"""Sends a OPTIONS request. Returns :class:`Response` object.                     │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/requests/sessions.py:589 in request                       │
│                                                                                                  │
│   586 │   │   │   "allow_redirects": allow_redirects,                                            │
│   587 │   │   }                                                                                  │
│   588 │   │   send_kwargs.update(settings)                                                       │
│ ❱ 589 │   │   resp = self.send(prep, **send_kwargs)                                              │
│   590 │   │                                                                                      │
│   591 │   │   return resp                                                                        │
│   592                                                                                            │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/requests/sessions.py:703 in send                          │
│                                                                                                  │
│   700 │   │   start = preferred_clock()                                                          │
│   701 │   │                                                                                      │
│   702 │   │   # Send the request                                                                 │
│ ❱ 703 │   │   r = adapter.send(request, **kwargs)                                                │
│   704 │   │                                                                                      │
│   705 │   │   # Total elapsed time of the request (approximately)                                │
│   706 │   │   elapsed = preferred_clock() - start                                                │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/requests/adapters.py:501 in send                          │
│                                                                                                  │
│   498 │   │   │   )                                                                              │
│   499 │   │                                                                                      │
│   500 │   │   except (ProtocolError, OSError) as err:                                            │
│ ❱ 501 │   │   │   raise ConnectionError(err, request=request)                                    │
│   502 │   │                                                                                      │
│   503 │   │   except MaxRetryError as e:                                                         │
│   504 │   │   │   if isinstance(e.reason, ConnectTimeoutError):                                  │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ConnectionError: ('Connection aborted.', TimeoutError(110, 'Connection timed out'))
Liang-KS commented 1 year ago

I have also encountered the issue today. never hit the same issue with recreationdotgov.

see attached txt file

camply_error.txt

vassoz commented 1 year ago

BTW: the command is:

docker run -it \
-e TELEGRAM_BOT_TOKEN=... -e TELEGRAM_CHAT_ID=.. \
juftin/camply camply --provider ReserveCalifornia campsites \
--rec-area 726 --campground 734 --campground 737 \
--start-date 2023-06-17 --end-date 2023-06-18 \
--notifications telegram \
--search-once
juftin commented 1 year ago

Hey @vassoz @Liang-KS I published a new release yesterday, you should notice more stability and less crashes 🤞 please let me know if you see these errors resolving

Liang-KS commented 1 year ago

Hey @vassoz @Liang-KS I published a new release yesterday, you should notice more stability and less crashes 🤞 please let me know if you see these errors resolving

so far so good, will let you know after a few days running. thanks for your time to fix this!!

juftin commented 1 year ago

Any issues being raised @Liang-KS?

Liang-KS commented 1 year ago

Any issues being raised @Liang-KS?

No issues, searching on ReserveCalifornia has been very stable since Jun 19th. Kudos!

vassoz commented 1 year ago

Looks good to me as well. Great work! Thank you!

I believe it exited couple times on me due to network issues. But I was not able to catch the logs.

Is it possible to add a notification for abnormal exists so you know that the tracking has stopped (e.g. via a watchdog process)?

juftin commented 1 year ago

Nice 🎉 !

Yeah, @vassoz there should be a "last gasp" notification sent out when camply exits. If you had a networking issue like an internet outage that prevents camply from accessing reservecalifornia.com AND from sending a notification (like through the Slack API) this wouldn't work.

When camply encounters an error from the provider API, like a 504 error, it should retry a few times before finally giving up and raising an error. Let me look into this and make sure we're giving these services enough retries to accommodate site downtime and stuff like that.

juftin commented 1 year ago

Alright, camply wasn't actually handling server exit codes at all for UseDirect providers (ReserveCalifornia and more). I've released camply v0.29.0 which should make ReserveCalifornia a whole lot more reliable. When it encounters a server status code, like a 503 response, it will retry for up to 100 minutes using exponential backoff.

This should be good to go🤞

juftin commented 1 year ago

It's been a little while since this issue was active. I will close this for now. Feel free to re-open if you still believe this is an issue.