Filtering Campsites

[stjohnjohnson]

I’m looking to a filter the alerts I get to ones that meet my requirements (e.g. trailer size).

How would you recommend I implement this?

[juftin]

Hey @stjohnjohnson apologies for the late reply here, I'm just getting back into things after a camping trip actually. I've been digging into this more and here's what I've found:

• The existing endpoint camply uses to search for campsite availabilities is https://www.recreation.gov/api/camps/availability/campground/<campground_id>/month?start_date=<start_date>

  • Here's an example
  • This endpoint only accepts a single Query String Parameter, start_date, and fails when you give it more than one param. The data returned also doesn't tell us anything about what equipment is allowed at each individual spot - so we can't leverage this particular endpoint for our purpose unfortunately.

• The place where we do get information about what equipment is allowed is when we retrieve metadata for the campsites themselves.

  • I've seen this metadata in two places, from the RIDB API (open source, camply uses an API token for Auth to this data) and also from the Recreation.gov API (mentioned above, it's open-sourceness is a little less clear and camply spoofs a web browser's user-agent in order to access it). The data retruned from these two
    • RIDB API: https://ridb.recreation.gov/docs#/Campsites/getFacilityCampsites
      • I'd prefer to use RIDB over Recreation.gov whenever possible but I've hitten some 404 errors when paginating campsites when experimenting with this locally
    • Recreation.gov API: https://www.recreation.gov/api/search/campsites?start=0&size=1000&fq=asset_id%3A231939&fq=campsite_equipment_name%3ATent&include_non_site_specific_campsites=true
      • I discovered this when playing around with Chrome Developer tools on this page. There are also cool ways to perform aggregate SQL queries on the underlying database with that endpoint too

So here's my idea: when camply validates the underlying campgrounds it's going to be searching through it will also make an additional call to retrieve information about all of the related Equipment / Attributes of the underlying campsites it searches. Subsequently, when it finds available campsites within those campgrounds, it will look up that metadata and join it on the Campsite object. Once that data is included on the campsite object it will be relatively simple to filter on it during the search process.

I have code I've been experimenting with, I will open up a branch and share that shortly. Hopefully we'll have a solution to this very soon.

[juftin]

Okay, following up. I've been working on it over here and I'm close: https://github.com/juftin/camply/pull/88

The trickiest part has been modifying how the underlying Data Containers handle hashing when some of their properties are lists (like attributes permitted equipment). Now that that is out of the way the last bits of work are in the details of the Implementation.

Here are some remaining questions:

• How should we standardize on equipment names. Here are the various names I've found so far after scaning ~3000 campsites:
  • 'Tent', 'RV', 'Trailer', 'Pickup Camper', 'Pop up', 'Vehicle', 'Large Tent Over 9X12`', 'Small Tent', 'Caravan/Camper Van', 'Car', 'Hammock', 'RV/Motorhome', 'Fifth Wheel', 'Horse', nan, 'Boat'
    • I'm thinking that we want to include a few generic categories like Tent, RV, Trailer and roll these up into one of those

What should the user interface look like? Will someone only specify that they have a trailer/tent/RV or should they also specify the maximum length?

[stjohnjohnson]

Awesome! I would use the standard filters that Recreation.gov has: vehicle length, site type, amenities, allowable equipment, and electrical hookup

[juftin]

Okay @stjohnjohnson I've made a ton of progress on this and just pushed the latest feature, the --equipment flag. Read more about it here: https://juftin.com/camply/README.html#searching-for-a-campsite-that-fits-your-equipment

As part of this I've surfaced two new attributes on the AvailableCampsite object, campsite_attributes and permitted_equipment which will be useful for expanding the filtering potential of campsites.

Check out some of the data here:

❯ export LOG_LEVEL=debug
❯ camply campsites \
    --campground 232338 \
    --start-date 2022-09-09 \
    --end-date 2022-09-17 \
    --nights 3 \
    --equipment RV 75 \
    --continuous \
    --notifications silent
[2022-06-30 15:27:37] CAMPLY   camply, the campsite finder ⛺️
[2022-06-30 15:27:37] INFO     8 booking nights selected for search, ranging from 2022-09-09 to 2022-09-16
[2022-06-30 15:27:37] INFO     Searching for availabilities with 3 consecutive night stays.
[2022-06-30 15:27:37] DEBUG    Starting new HTTPS connection (1): ridb.recreation.gov:443
[2022-06-30 15:27:39] DEBUG    https://ridb.recreation.gov:443 "GET /api/v1/facilities/232338?full=True HTTP/1.1" 200 None
[2022-06-30 15:27:39] INFO     1 Matching Campgrounds Found
[2022-06-30 15:27:39] INFO     ⛰  Rio Grande National Forest, CO (#2018) - 🏕  Thirty Mile (#232338)
[2022-06-30 15:27:39] INFO     Filtering Campsites based on Equipment: RV
[2022-06-30 15:27:39] INFO     Searching for campsites every 10 minutes.
[2022-06-30 15:27:39] INFO     Notifications active via: <SilentNotifications>
[2022-06-30 15:27:39] INFO     Only <SilentNotifications> enabled. I hope you're watching these logs.
[2022-06-30 15:27:39] INFO     Searching across 1 campgrounds
[2022-06-30 15:27:39] DEBUG    Starting new HTTPS connection (1): www.recreation.gov:443
[2022-06-30 15:27:39] DEBUG    https://www.recreation.gov:443 "GET /api/search/campsites?start=0&size=1000&fq=asset_id%3A232338&include_non_site_specific_campsites=True HTTP/1.1" 200 None
[2022-06-30 15:27:39] INFO     Metadata fetched for 35 campsites
[2022-06-30 15:27:39] INFO     Searching Thirty Mile, Rio Grande National Forest, CO (232338) for availability: September, 2022
[2022-06-30 15:27:39] DEBUG    Starting new HTTPS connection (1): www.recreation.gov:443
[2022-06-30 15:27:39] DEBUG    https://www.recreation.gov:443 "GET /api/camps/availability/campground/232338/month?start_date=2022-09-01T00%3A00%3A00.000Z HTTP/1.1" 200 None
[2022-06-30 15:27:39] INFO             ⛺️      175 total sites found in month of September
[2022-06-30 15:27:39] INFO     ⛺️ ⛺️ ⛺️ ⛺️ 1 Reservable Campsites Matching Search Preferences
[2022-06-30 15:27:39] INFO     📅 Fri, September 09 🏕  1 sites
[2022-06-30 15:27:39] INFO             ⛰️  Rio Grande National Forest, CO  🏕  Thirty Mile: ⛺ 1 sites
[2022-06-30 15:27:39] INFO                     🔗 https://www.recreation.gov/camping/campsites/38346 (3 nights)
[2022-06-30 15:27:39] INFO     1 New Campsites Found.
[2022-06-30 15:27:39] DEBUG    SilentNotification:
                                       • 2022-09-09 - 2022-09-12
                                       • STANDARD NONELECTRIC
                                       • 012
                                       • Rio Grande National Forest, CO
                                       • Thirty Mile
                                       • https://www.recreation.gov/camping/campsites/38346
[2022-06-30 15:27:39] DEBUG    Campsite Info: {'availability_status': 'Available',
                                'booking_date': Timestamp('2022-09-09 00:00:00'),
                                'booking_end_date': Timestamp('2022-09-12 00:00:00'),
                                'booking_nights': 3,
                                'booking_url': 'https://www.recreation.gov/camping/campsites/38346',
                                'campsite_attributes': [{'attribute_category': 'site_details',
                                                         'attribute_id': 56,
                                                         'attribute_name': 'Min Num of People',
                                                         'attribute_value': '1'},
                                                        {'attribute_category': 'site_details',
                                                         'attribute_id': 0,
                                                         'attribute_name': 'Campfire Allowed',
                                                         'attribute_value': 'Yes'},
                                                        {'attribute_category': 'site_details',
                                                         'attribute_id': 12,
                                                         'attribute_name': 'Checkout Time',
                                                         'attribute_value': '12:00 PM'},
                                                        {'attribute_category': 'site_details',
                                                         'attribute_id': 77,
                                                         'attribute_name': 'Shade',
                                                         'attribute_value': 'Yes'},
                                                        {'attribute_category': 'site_details',
                                                         'attribute_id': 52,
                                                         'attribute_name': 'Max Num of People',
                                                         'attribute_value': '16'},
                                                        {'attribute_category': 'site_details',
                                                         'attribute_id': 10,
                                                         'attribute_name': 'Capacity/Size Rating',
                                                         'attribute_value': 'Double'},
                                                        {'attribute_category': 'site_details',
                                                         'attribute_id': 11,
                                                         'attribute_name': 'Checkin Time',
                                                         'attribute_value': '2:00 PM'},
                                                        {'attribute_category': 'equipment_details',
                                                         'attribute_id': 23,
                                                         'attribute_name': 'Driveway Entry',
                                                         'attribute_value': 'Pull-Through'},
                                                        {'attribute_category': 'equipment_details',
                                                         'attribute_id': 54,
                                                         'attribute_name': 'Max Vehicle Length',
                                                         'attribute_value': '80'},
                                                        {'attribute_category': 'equipment_details',
                                                         'attribute_id': 0,
                                                         'attribute_name': 'Is Equipment Mandatory',
                                                         'attribute_value': 'false'},
                                                        {'attribute_category': 'equipment_details',
                                                         'attribute_id': 53,
                                                         'attribute_name': 'Max Num of Vehicles',
                                                         'attribute_value': '2'},
                                                        {'attribute_category': 'equipment_details',
                                                         'attribute_id': 26,
                                                         'attribute_name': 'Driveway Surface',
                                                         'attribute_value': 'Gravel'},
                                                        {'attribute_category': 'amenities',
                                                         'attribute_id': 301,
                                                         'attribute_name': 'Map Y Coordinate',
                                                         'attribute_value': '725.36'},
                                                        {'attribute_category': 'amenities',
                                                         'attribute_id': 300,
                                                         'attribute_name': 'Map X Coordinate',
                                                         'attribute_value': '576.64'},
                                                        {'attribute_category': 'amenities',
                                                         'attribute_id': 314,
                                                         'attribute_name': 'Placed on Map',
                                                         'attribute_value': '1'}],
                                'campsite_id': 38346,
                                'campsite_loop_name': 'THIR',
                                'campsite_occupancy': (1, 16),
                                'campsite_site_name': '012',
                                'campsite_type': 'STANDARD NONELECTRIC',
                                'campsite_use_type': 'Overnight',
                                'facility_id': 232338,
                                'facility_name': 'Thirty Mile',
                                'permitted_equipment': [{'equipment_name': 'Tent', 'max_length': 80.0},
                                                        {'equipment_name': 'RV', 'max_length': 80.0},
                                                        {'equipment_name': 'Trailer', 'max_length': 80.0}],
                                'recreation_area': 'Rio Grande National Forest, CO',
                                'recreation_area_id': 2018}
[2022-06-30 15:27:39] CAMPLY   Exiting camply 👋

[juftin]

@stjohnjohnson one more follow up. Will you test the Telegram notifications for me on camply 0.4.2? I tried to standardize notifications but haven't been able to test that Telegram notifications still look the same.

[ArtixZ]

A related question here: how can I filter by site type as shown in the screenshot above? e.g. Tent Only / Boat in / Hike to

[juftin]

Hey @ArtixZ those properties are campsite_attributes properties on the AvailableCampsite objects - right now I only support filtering on the permitted_equipment properties using the --equipment option: https://juftin.com/camply/README.html#searching-for-a-campsite-that-fits-your-equipment

Filtering on attributes is coming as soon as I get some time to code it up

[ArtixZ]

@juftin Thanks a lot working on this! This project is really a high quality stuff. I'm an outdoor hiking/backpacking lover, and a SDE. If I can contribute anything on this, do let me know.

[rexxars]

@juftin Is there a way to filter on campsite type? Or an arbitrary filter string that includes/exclude on campsite name? I'm getting a lot of "boat in only" results ('campsite_type': 'BOAT IN') that would be amazing to exclude.

[juftin]

Not yet, I'll find a way to eventually incorporate that into the campsite filtering feature.

[rtmlp]

Hi. Thanks for creating this awesome repository. I am interested in the campsite_attributes feature as well. Let me know if you need any help in implementation or testing?