Closed lhl2617 closed 4 years ago
I just need this for an ad-hoc projects scrape, so this is code that works for me
const scrapProjectsPanel = async (page) => {
const openingButton = await page.$(
`.pv-accomplishments-block.projects button`,
);
if (openingButton) {
await openingButton.click();
await new Promise((resolve) => {
setTimeout(() => {
resolve();
}, 500);
});
let showMoreButton;
while (showMoreButton = await page.$(`.pv-profile-section__see-more-inline`)) {
await showMoreButton.click();
await new Promise((resolve) => {
setTimeout(() => {
resolve();
}, 250);
});
}
const selectors = await page.$$(`.pv-accomplishment-entity`)
const got = await Promise.all(selectors.map((selector) =>
Object.keys(template.projects.fields)
.reduce(scrapSelectorFields(selector, template.projects), Promise.resolve({}))
))
return got;
}
}
I will PR when I have time - but if others have time-critical uses for this, please go ahead and PR for me
Hi,
When scraping for accomplishment sections for detailed accomplishments (added in #89, by @EmaSuriano), only the first item is returned. Was wondering if you are seeing the same bug.
Thanks