search

openaustralia / morph

Take the hassle out of web scraping
https://morph.io
GNU Affero General Public License v3.0
461 stars 74 forks source link

PHP version of scraperwiki fails silently when using camel case columns #1137

Open wfdd opened 7 years ago

wfdd commented 7 years ago

Camel case column names are automatically converted to snake case upon creation of the table using the RedBean API. Inserts bypass the API and fail as there is no column with the originally-provided camel case name. This failure is silent for reasons beyond my mental grasp.

LoveMyData commented 7 years ago

Current version of RedBean is pretty old, if you have the background with RedBean. Can you put something together and PR on the changes is highly welcome.

wfdd commented 7 years ago

Sorry, I don't use PHP with morph - I'd just been asked by an acquaintance to look into why his scraper was failing.

chris48s commented 7 years ago

@wfdd - I don't use PHP on morph either, but if I want to use a more recent version of a python package in morph, I can just specify a newer version in my scraper's requirements.txt and the more recent package will be used instead of the globally installed one. Can your friend fix his scraper by pulling in a more recent version of the library in his scraper's composer.json ?

wfdd commented 7 years ago

No, RedBeans is bundled with scraperwiki and has not received an update in 5 years. Also scraperwiki is the 'blessed' scraping lib here on morph so it ought to receive bug fixes from upstream.