search

leonard-IMBERT / StarlightMap

This is an interactive map for the role play forum isle of starlight
http://hexagrid.xyz
MIT License
0 stars 2 forks source link

Shielding every field of the crawler against malformed data #15

Closed KingMarth closed 6 years ago

KingMarth commented 6 years ago

This approach should make sure the crawler always gets us some data, even if not sensible data.

KingMarth commented 6 years ago

I haven't looked over your crafting tree parsing code closely; it'll be tricky since the item names don't show up anywhere in the description (since they're in the images), so you'd need to manually map each one, at which point there's little reason to scrape the page. I like the idea of having the image for an item pop up when hovering over that item, perhaps augmented by the rules text of that item.

I ended up adding a bunch of new parser features too. Tracking classes as well as declared classes/professions is helpful, but putting an item count is surprisingly useful when dealing with Gatherers (and our new structures with their 20-50 endurance).

leonard-IMBERT commented 6 years ago

Nice ! I've begun parsing the parsing but I was indeed thinking about file describing the tech an used the order of appearance in the post. I've got another idea with image recognition but I need to deep more in it.

Otherwise the pull request seems well done, thanks :)