elasticmachine
commented
2 years ago Pinging @elastic/kibana-gis (Team:Geo)
Closed jeffvestal closed 2 years ago
elasticmachine
commented
2 years ago Pinging @elastic/kibana-gis (Team:Geo)
nreese
commented
2 years ago @jeffvestal can you provide an explicit example. Runtime fields are working with sample data web logs. For example, create map and add sample web logs. Style color by hour_of_day, which is a runtime field. Everything renders as expected.
jeffvestal
commented
2 years ago @nreese
One example of a runtime field was to take a string price field, remove the $ and , then create a double field.
I can ping you with creds to my deployment so you can check it out. @nickpeihl checked it out last week also.
if (doc['price'].size() != 0){
String tmp=dissect('$%{price}').extract(doc["price"].value)?.price;
tmp = tmp.replace(',','');
def pn = Double.parseDouble( tmp );
if (pn != null) emit(pn);
}
else { emit(0)}
Is your example for a sample data set? If not, do you mind providing an index mapping and sample document?
jeffvestal
commented
2 years ago This is the airbnb chicago listings
mappings
"airbnb-chicago-listings": {
"mappings": {
"_meta": {
"created_by": "file-data-visualizer"
},
"properties": {
"@timestamp": {
"type": "date"
},
"accommodates": {
"type": "long"
},
"amenities": {
"type": "text"
},
"availability_30": {
"type": "long"
},
"availability_365": {
"type": "long"
},
"availability_60": {
"type": "long"
},
"availability_90": {
"type": "long"
},
"bathrooms_text": {
"type": "keyword"
},
"bedrooms": {
"type": "long"
},
"beds": {
"type": "long"
},
"calculated_host_listings_count": {
"type": "long"
},
"calculated_host_listings_count_entire_homes": {
"type": "long"
},
"calculated_host_listings_count_private_rooms": {
"type": "long"
},
"calculated_host_listings_count_shared_rooms": {
"type": "long"
},
"calendar_last_scraped": {
"type": "date",
"format": "iso8601"
},
"description": {
"type": "text"
},
"first_review": {
"type": "date",
"format": "iso8601"
},
"has_availability": {
"type": "keyword"
},
"host_about": {
"type": "text"
},
"host_acceptance_rate": {
"type": "keyword"
},
"host_has_profile_pic": {
"type": "keyword"
},
"host_id": {
"type": "keyword"
},
"host_identity_verified": {
"type": "keyword"
},
"host_is_superhost": {
"type": "keyword"
},
"host_listings_count": {
"type": "long"
},
"host_location": {
"type": "text"
},
"host_name": {
"type": "keyword"
},
"host_neighbourhood": {
"type": "keyword"
},
"host_picture_url": {
"type": "keyword"
},
"host_response_rate": {
"type": "keyword"
},
"host_response_time": {
"type": "keyword"
},
"host_since": {
"type": "date",
"format": "iso8601"
},
"host_thumbnail_url": {
"type": "keyword"
},
"host_total_listings_count": {
"type": "long"
},
"host_url": {
"type": "keyword"
},
"host_verifications": {
"type": "text"
},
"id": {
"type": "keyword"
},
"instant_bookable": {
"type": "keyword"
},
"last_review": {
"type": "date",
"format": "iso8601"
},
"last_scraped": {
"type": "date",
"format": "iso8601"
},
"latitude": {
"type": "double"
},
"license": {
"type": "text"
},
"listing_url": {
"type": "keyword"
},
"location": {
"type": "geo_point"
},
"longitude": {
"type": "double"
},
"maximum_maximum_nights": {
"type": "long"
},
"maximum_minimum_nights": {
"type": "long"
},
"maximum_nights": {
"type": "long"
},
"maximum_nights_avg_ntm": {
"type": "double"
},
"minimum_maximum_nights": {
"type": "long"
},
"minimum_minimum_nights": {
"type": "long"
},
"minimum_nights": {
"type": "long"
},
"minimum_nights_avg_ntm": {
"type": "double"
},
"name": {
"type": "text"
},
"neighborhood_overview": {
"type": "text"
},
"neighbourhood": {
"type": "keyword"
},
"neighbourhood_cleansed": {
"type": "keyword"
},
"number_of_reviews": {
"type": "long"
},
"number_of_reviews_l30d": {
"type": "long"
},
"number_of_reviews_ltm": {
"type": "long"
},
"picture_url": {
"type": "keyword"
},
"price": {
"type": "keyword"
},
"property_type": {
"type": "keyword"
},
"review_scores_accuracy": {
"type": "double"
},
"review_scores_checkin": {
"type": "double"
},
"review_scores_cleanliness": {
"type": "double"
},
"review_scores_communication": {
"type": "double"
},
"review_scores_location": {
"type": "double"
},
"review_scores_rating": {
"type": "double"
},
"review_scores_value": {
"type": "double"
},
"reviews_per_month": {
"type": "double"
},
"room_type": {
"type": "keyword"
},
"scrape_id": {
"type": "date",
"format": "yyyyMMddHHmmss"
}
}
}
}
}```
sample doc {
"_index": "airbnb-chicago-listings",
"_id": "QrTBh4EB8MMny-skTcLX",
"_score": 1,
"_source": {
"host_verifications": "['email', 'phone', 'facebook', 'reviews', 'kba']",
"picture_url": "https://a0.muscache.com/pictures/e32ef032-c20b-4f07-8cf0-30fa3d94f4bb.jpg",
"calendar_last_scraped": "2022-03-31",
"first_review": "2019-02-15",
"neighborhood_overview": "*Highly sought after neighborhood in Chicago! <br />*Within a 10 min walk of hundreds of incredible restaurants, cafes( including a Starbucks and a local roasters with the best coffee in the Midwest) clothing boutiques, grocery stores, convenience stores, killer taco joints, bars, live music venues and much more! <br />*It's easy to get around with the blue line train (which takes you to/from airport) a 5 min walk away plus tons of busses and easy access to the major highways! Although, you don't really need to leave the neighborhood. <br />*I have lived all over the city and this is hands down the best location! Whether you want the real Chicago experience or you're working and need easy access to downtown or any other parts of the city, this is where you need to be! <br />*Street parking is available for free on many of the side streets in this neighborhood. But, select blocks are permit parking after 6pm. Please make sure to read the signs and not park in a residential permit zone. ",
"number_of_reviews": 7,
"review_scores_cleanliness": 5,
"review_scores_checkin": 5,
"host_location": "Chicago, Illinois, United States",
"neighbourhood": "Chicago, Illinois, United States",
"price": "$226.00",
"reviews_per_month": 0.18,
"review_scores_location": 5,
"id": "20054600",
"host_identity_verified": "t",
"longitude": -87.67084,
"review_scores_accuracy": 4.86,
"host_response_time": "N/A",
"review_scores_value": 4.86,
"bedrooms": 2,
"license": "Registration Number: R17000017884",
"minimum_nights_avg_ntm": 2.3,
"name": "Beautiful Condo w/ Skyline Views & Private Decks",
"availability_60": 0,
"host_thumbnail_url": "https://a0.muscache.com/im/pictures/user/071b942e-a7fc-450e-9345-8d8c0f94f3cf.jpg?aki_policy=profile_small",
"host_neighbourhood": "Ukrainian Village",
"minimum_maximum_nights": 1125,
"beds": 3,
"scrape_id": "20220317023945",
"instant_bookable": "f",
"calculated_host_listings_count_entire_homes": 3,
"amenities": """["Essentials", "Patio or balcony", "Iron", "Refrigerator", "Hot water", "First aid kit", "Air conditioning", "Microwave", "Hangers", "Coffee maker", "Cable TV", "Kitchen", "Dishwasher", "Fire extinguisher", "Private entrance", "Carbon monoxide alarm", "Cooking basics", "TV with standard cable", "Luggage dropoff allowed", "Smart lock", "Stove", "Heating", "Smoke alarm", "Wifi", "Dishes and silverware", "Dedicated workspace", "Free street parking", "Oven", "Shampoo", "BBQ grill", "Indoor fireplace", "Hair dryer"]""",
"host_url": "https://www.airbnb.com/users/show/38111959",
"maximum_maximum_nights": 1125,
"neighbourhood_cleansed": "West Town",
"minimum_minimum_nights": 2,
"latitude": 41.90239,
"maximum_nights_avg_ntm": 1125,
"description": """-Penthouse with private rooftop deck with incredible views of Chicago skyline <br />-Beautiful home with modern furnishings, professional-grade appliances, and three private decks<br />-Sleeps 5 in 3 beds (king bed in master BR, air mattress in adjacent loft, non-pull-out couch in living room)<br />-Blocks from Blue Line train<br />-Blocks from ride share bike station<br />-15 minute ride to Michigan Avenue<br /><br /><b>The space</b><br />*Large open floor plan with beautiful hardwood and marble floors<br />*Brand new, modern kitchen with professional-grade appliances and breakfast bar<br />*Huge master suite with fancy king-sized bed and en-suite bathroom with double vanities, water closet, soaking tub with TV and spa shower, and sliding doors opening to second floor private balcony<br />*Loft space off master bedroom functions as additional bedroom with room for 1 queen sized air mattresses<br />*Living room features 52" 4K flat screen tv and large windows and doors opening on to pr""",
"review_scores_communication": 5,
"has_availability": "t",
"property_type": "Entire condo",
"host_response_rate": "N/A",
"availability_90": 0,
"calculated_host_listings_count_shared_rooms": 0,
"calculated_host_listings_count_private_rooms": 0,
"availability_365": 0,
"availability_30": 0,
"listing_url": "https://www.airbnb.com/rooms/20054600",
"maximum_minimum_nights": 3,
"maximum_nights": 15,
"number_of_reviews_ltm": 0,
"host_picture_url": "https://a0.muscache.com/im/pictures/user/071b942e-a7fc-450e-9345-8d8c0f94f3cf.jpg?aki_policy=profile_x_medium",
"host_has_profile_pic": "t",
"bathrooms_text": "1 bath",
"number_of_reviews_l30d": 0,
"host_id": "38111959",
"host_listings_count": 3,
"host_total_listings_count": 3,
"host_acceptance_rate": "N/A",
"review_scores_rating": 5,
"@timestamp": "2022-03-17T02:39:45.000-05:00",
"minimum_nights": 2,
"last_review": "2020-02-04",
"calculated_host_listings_count": 3,
"accommodates": 5,
"host_is_superhost": "t",
"location": "41.90239,-87.67084",
"host_since": "2015-07-10",
"last_scraped": "2022-03-31",
"host_name": "Joanna (Dba Blue Sky Home Rentals)",
"room_type": "Entire home/apt"
}
}Thanks, I am able to re-create the problem on main. I have updated issue comment with more detailed test instructions
nreese
commented
2 years ago I have isolated the problem. The % character in the runtime field is causing decodeURIComponent call to throw URI malformed exception.
Kibana version: 8.3.0
Elasticsearch version: 8.3.0
Original install method (e.g. download page, yum, from source, etc.): Elastic Cloud
Describe the bug: When a Documents layer with vector scaling is created from a data view that contains a runtime field, the layer does not render.
Steps to reproduce:
Run console commands to create
testindextestdata view.Add runtime field
price_as_numberDocumentslayer withtestdata viewUse vector tilesis selected for ScalingExpected behavior: Map will render documents' geo_point
Screenshots (if relevant):
Errors in browser console (if relevant): Error example in layer editor (documents -> vector)